v2.0.0-beta

⚠️ Breaking-Changes ⚠️

This release contains important changes in the database structure and is therefore not compatible with previous releases. A full re-synchronization of the index will be needed.

Note also that any intermediate work from master isn't guaranteed to be compatible with v2.0.0. Should you have been using intermediate edge versions, you will need, in all likelihood, to drop and reconstruct the database as well.

Added

• 📌 #28 - Support for synchronization through the Babbage's era, including capturing inline-datums & reference scripts.

• 📌 #17 - New command-line flag: --prune-utxo. When set, inputs that are spent on-chain will be removed from the index. Once-synced,
  the index therefore only contain the current ledger UTxO set. When not set, spent inputs are kept in the index but are now marked accordingly to record if and when they've spent.

  HTTP endpoints for /matches (& the like) can also now accept either optional query-flag ?spent or ?unspent to filter matches depending on whether they've been spent.

  Consequently, there's also a new (possibly null) field spent_at returned for each match result. When set, it indicates the slot in which the input was found being spent.

  • GET /matches[?(spent|unspent)] → 📖 API Reference
  • GET /matches/{pattern-fragment}[?(spent|unspent)] → 📖 API Reference
  • GET /matches/{pattern-fragment}/{pattern-fragment}[?(spent|unspent)] → 📖 API Reference

• 📌 #21 New HTTP endpoint to retrieve Plutus' datum pre-image from a datum hash digest. Behind the scene, Kupo now track any datum found in transactions' witnesses set or output (inline datums). Note that, datums that aren't associated to any existing pattern matches are eventually garbage-collected.
  • GET /datums/{datum-hash} → 📖 API Reference

• 📌 #21 New HTTP endpoint to retrieve native & Plutus' script pre-image from a script hash digest. Behind the scene, Kupo now track any script found in transactions' witnesses set, auxiliary data and/or outputs (reference scripts).
  • GET /scripts/{script-hash} → 📖 API Reference

• 📌 #40 (🔨 @nielstron) New HTTP endpoint to retrieve patterns that includes a given pattern. Useful to check if an address is matched by a given configuration.
  • GET /patterns/{pattern-fragment}[/{pattern-fragment}] → 📖 API Reference

• 📌 #40 New optional command-line option --gc-interval to tweak the interval between database's garbage collection. Each garbage collection takes a bit of time (few seconds) and pauses the indexer while doing so; A too short interval may have a strong impact on overall syncing time. A too long interval may increase the time needed for collecting garbage. Optimal values depends on your use-case, but the default (10 minutes) is seemingly a sensible default. While syncing from scratch with very permissive patterns, you may want to increase this value (e.g. to 60 minutes) to avoid needlessly pausing the synchronization.

• 📌 #24 - New HTTP endpoint to retrieve a point on-chain from a given slot. The endpoint is flexible and allows for retrieving the ancestor
  of a known point very easily. This is handy in combination with other protocols that leverage on-chain points and intersections (like Ogmios' chain-sync).

  • GET /checkpoints/{slot-no}[?strict] → 📖 API Reference

• 📌 #35 - New HTTP header X-Most-Recent-Checkpoint to every (successful) response. It contains the slot number of the current database most recent checkpoint. This allows client to know which slot a certain query is accurate of.

Changed

• 📌 #17 - The slot_no and header_hash fields are no longer accessible on top-level match result objects. Instead, they're now nested under a created_at field, analogous to how spent_at has been introduced.

• 📌 #24 - Fixed a bug where listing checkpoints would sometimes return duplicate entries.

• 📌 #39 - Inserting a new pattern (i.e. PUT /patterns/{pattern-fragment}) now requires to provide a rollback point, to which the server will rollback and start synchronizing again. The old behavior can be recovered by simply passing the most recent checkpoint as a rollback point. Note that, you may add an already existing pattern if you only need, for some reason, to rollback the indexer to some previous point in time. See the 📖 API Reference for details.

• 📌 #48 - (Massively) improved performance of query by stake credential. This used to be in linear time in the size of the UTxO set, and is now performed in logarithmic time, same as query by address. Querying by payment credentials is still performed in linear times though so this is probably something you want to avoid doing on permissive patterns (e.g. * or */*).

• API endpoints are no longer versioned (i.e. prefixed with v1). However, providing v1 will still work and route requests all-the-same to ensure backward-compatibility. The rationale being that, since kupo is a local service (and thus, clients decide when they want to upgrade), there's no particular need to version the API in the request path.

• Fixed a bug where the server would systematically reject any request to dynamically remove a pattern (because deemed overlapping with existing patterns).

Removed

• N/A

Snapshots

In the release artifacts, you can find two snapshots (kupo.sqlite3-{testnet, mainnet}.tar.gz). These snapshots were obtained from the following options:

• --match *
• --since origin
• --prune-utxo

They can be used as starting points to get started quickly.

Dependencies

Dependency	Tested With
cardano-node	1.35.x
ogmios	v5.5.x

Details

v1.0.1...v2.0.0-beta

Acknowledgments

Thanks a lot to @Quantumplation, @MartinSchere, @bakon11, @waalge, @baymac and @mmahut for their feedback and ideas.