v2.9.0

Warning

Artifact names have changed in the release. They are now zip archives, and architectures are x86_64 and aarch64 instead of amd64 and arm64.

Added

v2.9.0

• tip is now supported as a value for the --since option. When provided, the indexing process will start wherever the blockchain source (node, ogmios, ...) is at.

Changed

v2.9.0

• Integrate with cardano-node==9.0.0.

• Integrate with hydra-node==0.16.0.

Removed

v2.9.0

N/A

Benchmarks

See benchmarks/README.md.

Snapshots

See Snapshots from v2.5

Dependencies

Dependency	Tested With
cardano-node	9.0.0
ogmios	v6.*.*
hydra-node	0.16.0

Details

v2.8.0...v2.9.0

v2.8.0

Added

v2.8.0

• (💡 @scarmuega) A new mode --read-only which can be used to boot-up an HTTP server with only read access to the underlying database. This option comes as an alternative to the other options for chain producers (e.g. --node-socket and --node-config). The replica can only reply successfully to GET queries with the exception of queries under /metadata. The latter must go through the master server.

• Automatic restart and setup indexes when --defer-db-indexes is provided and the tip of the chain is reached.

• A new field configuration.indexes to the health object that indicates whether the server is still deferring the installation of lookup indexes or has already installed them.

Changed

v2.8.0

• (🔨 @klarkc + @sorki) Integrate with cardano-node==8.7.2 including the (preliminary) Conway era.

• (💡 @eMCeee89) The server now returns:

  • 503 from /health if the connection_status is "disconnected";
  • 202 if the server is connected but far away from the node's tip (i.e. still syncing)

• Reinstate WAL journal mode for SQLite main writer. This should allow setting up more concurrent readers on top of the same database.

• Fine-tune some internal constraints around database connections management for better performances. In particular, the maximum number of concurrent readers have been increased.

Removed

v2.8.0

• 'Content-Length' header on some of the server responses which turned out to be already handled by the underlying server when necessary (according to the original HTTP specification).

Benchmarks

See benchmarks/README.md.

Snapshots

In the release artifacts, you can find multiple snapshots (kupo.sqlite3-{mainnet, preview, preprod}.tar.gz). These snapshots were obtained from the following options:

• --match *
• --since origin
• --prune-utxo

They can be used as starting points to get started quickly. Note that large archives are split in multiple parts; the original archive can be recovered using cat. For example, for mainnet:

❯ cat kupo.sqlite3-mainnet.tar.gz.part_* > kupo.sqlite3-mainnet.tar.gz

It is recommended to also check the archive's integrity using md5. It should match the digests below depending on the network. For example, for mainnet:

❯ md5 kupo.sqlite3-mainnet.tar.gz
MD5 (kupo.sqlite3-mainnet.tar.gz) = 972da57c519a281561c4d6b904a82301

Network	MD5 Digest
preview	d67726a64b5714fdd1dd68790762659e
preprod	57d11cdf4461dfc8f24fbbcc108542ed
mainnet	972da57c519a281561c4d6b904a82301

Docker image(s)

cardanosolutions/kupo:v2.8.0

Dependencies

Dependency	Tested With
cardano-node	8.7.2, 8.7.3
ogmios	v6.0.3
hydra-node	0.13.0

Details

v2.7.2...v2.8.0

v2.7.2

Added

v2.7.0

• (🔨 @ch1bo, 🔨 @v0d1ch) Support for indexing a Hydra head.

  Use --hydra-host and --hydra-port to connect kupo to a hydra-node.

  Note

  Hydra heads do not actually form a 'chain'; Hydra doesn't have blocks, but snapshots which are akin to blocks so we treat them as such. The block header hash we opted for is arbitrary and is a hash of all the transaction id present in the snapshot.

  It also comes with diminished capabilities since Hydra doesn't have any protocol to query metadata for example and Kupo does not store them. So unlike Ogmios or Cardano-node, metadata cannot be retrieved when Hydra is used as a chain producer.

• Added TLS certificates to the Docker image so that remote connections to Ogmios or Hydra instances behind TLS can mostly work out-of-the-box

v2.7.2

• (🐛 @nielstron) Add missing index on inputs' datum (speed up garbage collection). See #151.
• (🔨 @angerman) Automated aarch64 static builds for Linux.

Changed

v2.7.1

• Upgraded internal dependencies to cardano-node==8.6.0.

• Fixed Kupo choking on mainnet pointer addresses following ledger internal updates in cardano-node>=8.1.2. See also: input-output-hk/cardano-ledger#3898.

• Fixed Kupo not indexing inline-scripts in output until they were used in a transaction. See also #148.

• Binary data is now pruned incrementally to avoid blocking the main writer thread for too long, especially in scenario where the garbage collection has been 'snoozed' many times. See also #143

• Fixed wrong metadata hash being reported for metadata containing native or Plutus scripts.

v2.7.2

• Fix missing runtime flags on pre-compiled binary. See #152.

Removed

N/A

Benchmarks

See benchmarks/README.md.

Snapshots

See Snapshots from v2.5

Dependencies

Dependency	Tested With
cardano-node	8.1.2, 8.6.0
ogmios	v6.0.0
hydra-node	0.13.0

Details

v2.6.0...v2.7.2

v2.6.1

Added

v2.6.0

• Simple internal retrying strategy on HTTP request, this should make the experience with the API server smoother as request would fail less often. Transient failures would now likely be resolved internally.

Changed

v2.6.0

• Support for (and require) Ogmios >= v6.0.0 when using Ogmios as a blockchain provider.

• Use biggger cache and database page size. This makes queries 10-15% faster on average, at the expense of some extra memory usage during period of heavy loads. This new setting is a trial and may be made configurable if deemed necessary.

• Rollbacks are now done incrementally. This doesn't really affect normal operations as rollbacks are usually small already, but it does impact the dynamic addition of patterns with rollback points far in the past.

• Re-implemented a fix from v2.3.4 that got lost in translation regarding restart on failures.

• Fixed / reworked ANSI logs to be more compatible with fonts that do not support fancy chevron characters. Colors have also been adjusted for better readibility.

• Kupo now has a top-level panic hook acting as a 'catch all' exception. So unexpected failures will now be properly logged as error and cause the application to exit with an exit code 1.

v2.6.1

• (🐛 @mpizenberg, @scarmuega) Fixed 'not enough bytes' exception sometimes raised when establishing a connection over TLS.
• Fixed connection failures for fetching metadata when the connection is configured over TLS.

Removed

v2.6.0

• --max-concurrency command-line option is gone; Kupo now use sensible defaults based on the machine's detected capabilities.

Benchmarks

See benchmarks/README.md.

Snapshots

See Snapshots from v2.5

Docker image(s)

cardanosolutions/kupo:v2.6.1

Dependencies

Dependency	Tested with
cardano-node	8.1.2
ogmios	v6.0.0

Note
Kupo requires either cardano-node or ogmios as a blockchain entrypoint, but not both.

Details

v2.5.0...v2.6.1

v2.5.1

Added

v2.5.0

• (:bulb: @jeremybarnes2342) Allow connection to remote Ogmios hosts behind TLS. This is now possible by prefixing the hostname with wss://.

• (:bulb: @juliusfrost) The HTTP server now replies to pre-flight requests (i.e. OPTIONS).

Changed

v2.5.0

• (:bulb: @juliusfrost) The HTTP server is now accessible cross-origins by default (Access-Control-Allow-Origin: *).

• (:bug: @ray-robot) Increased logs surrounding rollbacks, and adjusted the management of internal indexes related to rollbacks. This should lead to a decrease of I/O activity during rollbacks, without noticeable impact on the application (provided that initial synchronization is done with --defer-db-indexes)

v2.5.1

• (:bug: @mpizenberg, @scarmuega) Fixed 'not enough bytes' exception sometimes raised when establishing a connection over TLS. (backported from v2.6.1)
• Fixed connection failures for fetching metadata when the connection is configured over TLS. (backported from v2.6.1)
• Fixed / reworked ANSI logs to be more compatible with fonts that do not support fancy chevron characters. Colors have also been adjusted for better readibility. (backported from v2.6.0)
• Re-implemented a fix from v2.3.4 that got lost in translation regarding restart on failures.

Removed

N/A

Benchmarks

See benchmarks/README.md.

Snapshots

In the release artifacts, you can find multiple snapshots (kupo.sqlite3-{mainnet, preview, preprod}.tar.gz). These snapshots were obtained from the following options:

• --match *
• --since origin
• --prune-utxo

They can be used as starting points to get started quickly. Note that large archives are split in multiple parts; the original archive can be recovered using cat. For example, for mainnet:

❯ cat kupo.sqlite3-mainnet.tar.gz.part_* > kupo.sqlite3-mainnet.tar.gz

It is recommended to also check the archive's integrity using md5. It should match the digests below depending on the network. For example, for mainnet:

❯ md5 kupo.sqlite3-mainnet.tar.gz
MD5 (kupo.sqlite3-mainnet.tar.gz) = b79c6855aa3f7dc552f5d7844256f79b

Network	MD5 Digest
preview	1157ac86f741082f8ffe823f9b9e70be
preprod	57d11cdf4461dfc8f24fbbcc108542ed
mainnet	b79c6855aa3f7dc552f5d7844256f79b

Docker image(s)

cardanosolutions/kupo:v2.5.1

Dependencies

Dependency	Tested With
cardano-node	1.35.4, 1.35.7, 8.1.0, 8.1.1, 8.1.2
ogmios	v5.6.0, v5.7.0, v5.8.0

Details

v2.4.1...v2.5.1

v2.4.1

Added

v2.4.0

• 📌 #108 Introduce a new indexing pattern by transaction metadata tag. This pattern will only index outputs of a transaction that (a) has metadata, (b) has one metadatum labelled with the provided tag. This pattern is however (at least for now) only usable for indexing, not querying.

  → 📖 API Reference

• Provide a new command copy to conveniently clone an existing database into a smaller subset. This is particularly useful to quickly fork new instances of a parent index without having to resynchronize the entire chain. The main use case being a scenario where one maintains a global index (i.e. matching *) and needs to create on-demand indexes using more restrictive patterns. The copy commands accepts one or many patterns and copies (even large) indexes in a matter of seconds.

  Usage: kupo copy --from DIR --into DIR [--match PATTERN]
  
    Copy from a source database into another, while applying the provided pattern
    filters.
  
  Available options:
    -h,--help                Show this help text
    --from DIR               Working directory to copy from.
    --into DIR               Working directory to copy into.
    --match PATTERN          A pattern to match on. Can be provided multiple times (as a logical disjunction, i.e. 'or')
  

• The GET /health endpoint now returns the version as part of the (JSON) response.

Changed

v2.4.1

• Fix ANSI colors of logs in TTY terminals

• Fix internal dependency causing build issues for ouroboros-consensus

v2.4.0

• The server now implements some basic retry mechanism for some transient errors that can occur under heavy load (e.g. failing to open a database connection). This is mostly transparent for clients but should result in less 503 errors by providing a first retryable layer directly in the server.

• Add missing (optional) index on policies table to speed up queries by policy id and asset id.

• Executables from master will now return nightly as a version number, with a git commit hash. This is done in hope to reduce confusion when figuring out what version is a particular binary built against.

Removed

N/A

Benchmarks

See benchmarks/README.md.

snapshots

See Snapshots from v2.3.4

Dependencies

Dependency	Tested With
cardano-node	1.35.x
ogmios	v5.6.x

Details

v2.3.4...v2.4.1

v2.3.4

Added

v2.3.0

• Support for ETag/If-None-Match standard HTTP caching. ETag in kupo matches the most recent block header hash. This allows clients to perform efficient polling and caching on their end. This also comes as an additional protection for rollbacks as one can control the ETag between two requests and assess whether a rollback happened.

• 📌 #96 (:bulb: @cortsf) Matches can now be fetched in a paginated fashion, using slot ranges. Ranges can be made on either created_at or spent_at fields, and are inclusive. Besides, clients have two ways to define ranges:

  • by absolute slot number;
  • by point (slot number + block header hash).

  The latter performs an extra check and will fail should the provided point not exist. This is handy to fetch collections over multiple pages while ensuring that the underlying data doesn't change due to a new fork of the chain being adopted behind the scene. Here are some examples of queries with (valid) ranges:

  /matches?created_after=1234
  

  /matches?created_after=1234&created_before=5678
  

  /matches?spent&spent_before=1234
  

  /matches?spent&created_after=1234.4675360c80235b60b127222702b6e9b2b5c20dee7115acfc46eb6f3e9fd97ff0&spent_before=5678
  

  See:

  • GET /matches → 📖 API Reference
  • GET /matches/{pattern} → 📖 API Reference

Changed

v2.3.4

• (:bug: @mmahut) Fixed negotiation of the intersection point after loosing (and recovering) connection from its block provider (cardano-node or ogmios). Kupo would restart synchronizing back to the checkpoint known at startup instead of the latest known checkpoint in the indexer.

v2.3.3

• (:bug: @Quantumplation) Fixed a bug where Kupo would prune inputs at regular interval even if --prune-utxo isn't set. This only occurs when matching patterns are neither * nor */*. Related, Kupo would also never prune binary data as it should; even with --prune-utxo enabled.

v2.3.2

• (:bug: @mmahut @scarmuega) Bumped version of the embedded SQLite stack to v3.40.1 and changed the main writer process journaling mode to TRUNCATE. These two changes come as a mitigation of an SQLite bug that resulted in sometimes throwing:

  Error while attempting to perform step: cannot rollback - no transaction is active.

  The root cause for this error is hard to identify and may be due to a full disk space; It's been also linked to the write-ahead logging journal mode; which Kupo benefits only a little and only during syncing.

v2.3.1

• 📌 #101 (:bug: @jy14898) Starting from the Babbage era, transactions that fail phase-2 validations (i.e. script execution) can use a collateral return output where to send change from a lost collateral. This makes collateral management easier for client applications and result in a special kind of UTxO in the ledger. Kupo has been wrongly attributing an index of 0 to those outputs, whereas the ledger indicates the following:

  Note that the new $collOuts$ function generates a single output with an index $|txOuts_{txb}|$.

  This is now fixed. Database patches are provided for preview, preprod with the release notes of v2.3.1 to update already indexed collateral returns if any.

  Note

  There's no patch for mainnet because, to this date, there hasn't been any phase-2 failure on mainnet making use of collateral returns.

• (:bug: @mmahut) Make internal db garbage-collection more incremental. This has two main benefits:

  1. It allows the consumer to preempt the database connection for writing blocks while a GC is happening. This is crucial to not start lagging behind because of a long-running (several minutes) garbage collection.
  2. It breaks the inputs pruning over multiple smaller transaction, which are much faster to process than a single (sometimes excessively large) transaction by SQLite under the hood; considerably reducing the garbage-collection delays.

• Passing --defer-db-indexes now also removes query indexes if present.

  Prior to this commit, passing --defer-db-indexes on a database with already present indexes would only raise a warning and there was no practical way to remove indexes from the database (other than doing it manually). Now, --defer-db-indexes will systematically make sure that indexes are dropped when set. Thus, this flag effectively controls the existence (or more specifically, the absence) of query indexes in the database. Starting kupo without this flag still creates the indexes should they not exist.

v2.3.0

• 📌 #94 (:bulb: @mpizenberg) Improved user-experience on start-up when providing invalid or missing working directory. Kupo will now recursively create the working directory if it's missing and otherwise provide a more informative error if it can't (e.g. because the directory already exists and is a file or because of a lack of permissions).

• 📌 #95 (:bulb: @Quantumplation) The behavior associated with rollback_to (when dynamically inserting new patterns) has been slightly altered. Before, it used to fail when providing only a slot number not associated with any on-chain point. Now, Kupo will rollback to the closest ancestor of the given slot number. When there's an existing point, or when a header-hash is provided alonside the slot number; the behavior remains unchanged.

• 📌 #98 Logs are now shown in a human-friendly way when stdout is an ANSI-capable terminal. When sent to a file or to a non-terminal, the behaviour is unchanged (i.e. structured JSON).

• (:bug: @scarmuega) Kupo will no longer crash when loosing connection with the underlying cardano-node or Ogmios server. Instead, it'll recover from the issue and try to reconnect automatically.

Removed

v2.3.0

• The /patterns family of endpoints no longer return a X-Most-Recent-Checkpoint for it doesn't make much sense for these endpoints. Indeed, they may change regardless of what is processed by the indexer.

Benchmarks

See benchmarks/README.md.

Snapshots

In the release artifacts, you can find multiple snapshots (kupo.sqlite3-{mainnet, preview, preprod}.tar.gz). These snapshots were obtained from the following options:

• --match *
• --since origin
• --prune-utxo

They can be used as starting points to get started quickly. Note that the mainnet archive is split in multiple parts; the original archive can be recovered as such:

❯ cat kupo.sqlite3-mainnet.tar.gz.part_* > kupo.sqlite3-mainnet.tar.gz

Then, verify the archive's integrity using md5, it should match the checksum below:

❯ md5 kupo.sqlite3-mainnet.tar.gz
MD5 (kupo.sqlite3-mainnet.tar.gz) = dba114c96a81be73c7fbccb596bcfe60

Dependencies

Dependency	Tested With
cardano-node	1.35.x
ogmios	v5.5.x

Details

v2.2.1...v2.3.4

v2.2.1

Added

v2.2.0

• (:bulb: @alessandrokonrad) Results now contain a new field datum_type only present when datum_hash is not null. It indicates whether the datum in the output is inline or only a hash reference.

  See GET /matches/{pattern} → 📖 API Reference

Changed

v2.2.1

• 📌 #86 (:bug: @nielstron) - Fix start-up configuration check to not fail when providing twice the same pattern on restart.

• Fix Ogmios' parser for non-Babbage eras in the presence of collaterals without collateral return.

• Provide better error message when starting Kupo with a misconfigured network. This is a pretty common case and the message used to be a bit cryptic. Kupo now detects that and inform the end-user properly.

v2.2.0

• (:bug: @errfrom) Fixed a bug where utxo entries from collateral returns (on phase-2 failed transaction) would be missing from the index. In fact, Kupo does not index outputs from failed transaction, but since the Babbage era, failed transactions may contain an extra field "collateral return" which becomes a legitimate transaction output on-chain. These are now properly indexed as well.

Removed

N/A

Benchmarks

See benchmarks for v2.1.0.

Snapshots

In the release artifacts, you can find two snapshots (kupo.sqlite3-{mainnet}.tar.gz). These snapshots were obtained from the following options:

• --match *
• --since origin
• --prune-utxo

They can be used as starting points to get started quickly. Note that the mainnet archive is split in two parts; the original archive can be recovered as such:

❯ cat kupo.sqlite3-mainnet.tar.gz.part_* > kupo.sqlite3-mainnet.tar.gz

Then, verify the archive's integrity using md5, it should match the checksum below:

❯ md5 kupo.sqlite3-mainnet.tar.gz
MD5 (kupo.sqlite3-mainnet.tar.gz) = TODO

Dependencies

Dependency	Tested With
cardano-node	1.35.x
ogmios	v5.5.x

Details

v2.1.0...v2.2.1

v2.1.0

Warning
Internal Breaking-Change

This release internally reworks the output_reference column of the inputs table.
There's no direct upgrade from an existing index since this requires additional information not present in the database; the only possible migration is therefore to drop the entire index and force a resync from scratch.

Added

• 📌 #25 - New pattern format to match results by transaction id or by output reference. Also added two new HTTP query parameters for filtering matches: transaction_id & output_index. See more information in the 📖 API Reference.

  See syntax

    ┏━━━━━━━━━━━━━━┓ ╭───╮ ┏━━━━━━━━━━━━━━━━┓
  ╾─┫ OUTPUT_INDEX ┣─┤ @ ├─┫ TRANSACTION_ID ┣─╼
    ┗━━━━━━━━━━━━━━┛ ╰───╯ ┗━━━━━━━━━━━━━━━━┛
  

  Note
  Filtering is done by scanning linearly over all results, whereas matching is much faster and leverages the database's internal indexes. One can however combine filters and matches in all kind of ways.

• 📌 #56 (:bulb: @1000101) - New pattern format to match results by policy id or asset id. Also added two new HTTP query parameters for filtering matches: policy_id & asset_name. See more information in the 📖 API Reference.

  See syntax

    ┏━━━━━━━━━━━┓ ╭───╮ ┏━━━━━━━━━━━━┓
  ╾─┫ POLICY_ID ┣─┤ . ├─┫ ASSET_NAME ┣─╼
    ┗━━━━━━━━━━━┛ ╰───╯ ┗━━━━━━━━━━━━┛
  

  Note
  Filtering is done by scanning linearly over all results, whereas matching is much faster and leverages the database's internal indexes. One can however combine filters and matches in all kind of ways.

• 📌 #51 (:bulb: @everestada, 💬 @bakon11) - New endpoint to retrieve transaction metadata by slot number, possibly filtered by transaction id or output reference.

  • GET /metadata/{slot-no} → 📖 API Reference

• 📌 #63 (:hammer: @sorki) - Prometheus-compatible format can now be returned by GET /health. This occurs when Accept: text/plain is provided as request header. Otherwise, it defaults to application/json as before.

• 📌 #65 (:bulb: @1000101) - Added transaction_index to each result, referring to the index in the block of the transaction carrying the matched output.

• 📌 #72 (:bulb: @Quantumplation) - Clients can now bulk-add many patterns at once.

  • PUT /patterns → 📖 API Reference

• New command-line flag --defer-db-indexes to be given during the very first synchronization to postpone the creation of some non-essential database indexes. Non-essential refers to indexes that are needed to speed up some queries (e.g. querying by payment credentials) but that do not play a role in synchronization. In fact, maintaining such indexes during synchronizations (millions of blocks) adds a lot of unnecessary overhead. Indexes can be created after the facts in a couple of minutes on a standard laptop. This flag therefore comes in handy to speed up an initial synchronization. Once synchronized, the server can be restarted without the flag to automatically create the necessary indexes.

• The server now makes use of an internal resource pool when it comes to database connections. This pool can be configured using a new command-line option --max-concurrency. Users running an instance of Kupo on a capable machine (e.g. 32-cores) may want to increase the default of 50.

Changed

• 📌 #65 (:bulb: @1000101) - Results are now ordered by descending slots, transaction index and output index by default. They used to be ordered by descending slot number only; and then arbitrarily ordered from within a same slot. The order is now fully stable.

• 📌 #58 (:bulb: @nielstron) - Improved performances of query by payment credentials from linear times to logarithmic times. Most patterns now benefits from database indexes and are blazing fast.

• (:hammer: @mmahut, @1000101) Tuned various runtime and internal parameters to optimize query performances. The indexer now makes better use of available machine resources -- in particular CPU.

• Request log messages of the HTTP servers have been split in two; the server is now logging request and responses independently with the time taken to process the response as part of the log message.

• The server now returns an error 503 Service Unavailable when too many requests are pilling up. Note that the server can handle a relatively heavy load for most patterns but, for large addresses (such as the contract addresses of big marketplaces or DEXes), a single query can take a few seconds. If the server has exhausted all its available resources to serve additional requests, it'll fail gracefully with an error 503 and let the client handle requeuing of the request if necessary.

Removed

• stack is no longer supported as a development tool / build option.

• (:bug: @Quantumplation) A bug in the PUT /patterns/{pattern-fragment} endpoint which would cause the server to take a very long time to reply when already synchronized. Adding a pattern is now instantenous when connected through Ogmios and effective as soon as a next block is visible when connected through cardano-node.

Benchmarks

See benchmarks.

Snapshots

In the release artifacts, you can find two snapshots (kupo.sqlite3-{testnet, mainnet}.tar.gz). These snapshots were obtained from the following options:

• --match *
• --since origin
• --prune-utxo

They can be used as starting points to get started quickly. Note that the mainnet archive is split in two parts; the original archive can be recovered as such:

❯ cat kupo.sqlite3-mainnet.tar.gz.part_* > kupo.sqlite3-mainnet.tar.gz

Then, verify the archive's integrity using md5, it should match the checksum below:

❯ md5 kupo.sqlite3-mainnet.tar.gz
MD5 (kupo.sqlite3-mainnet.tar.gz) = a869ce18da5e028d80d9ad6d40f20897

Dependencies

Dependency	Tested With
cardano-node	1.35.x
ogmios	v5.5.x

Details

v2.0.0-beta...v2.1.0

v2.0.0-beta

⚠️ Breaking-Changes ⚠️

This release contains important changes in the database structure and is therefore not compatible with previous releases. A full re-synchronization of the index will be needed.

Note also that any intermediate work from master isn't guaranteed to be compatible with v2.0.0. Should you have been using intermediate edge versions, you will need, in all likelihood, to drop and reconstruct the database as well.

Added

• 📌 #28 - Support for synchronization through the Babbage's era, including capturing inline-datums & reference scripts.

• 📌 #17 - New command-line flag: --prune-utxo. When set, inputs that are spent on-chain will be removed from the index. Once-synced,
  the index therefore only contain the current ledger UTxO set. When not set, spent inputs are kept in the index but are now marked accordingly to record if and when they've spent.

  HTTP endpoints for /matches (& the like) can also now accept either optional query-flag ?spent or ?unspent to filter matches depending on whether they've been spent.

  Consequently, there's also a new (possibly null) field spent_at returned for each match result. When set, it indicates the slot in which the input was found being spent.

  • GET /matches[?(spent|unspent)] → 📖 API Reference
  • GET /matches/{pattern-fragment}[?(spent|unspent)] → 📖 API Reference
  • GET /matches/{pattern-fragment}/{pattern-fragment}[?(spent|unspent)] → 📖 API Reference

• 📌 #21 New HTTP endpoint to retrieve Plutus' datum pre-image from a datum hash digest. Behind the scene, Kupo now track any datum found in transactions' witnesses set or output (inline datums). Note that, datums that aren't associated to any existing pattern matches are eventually garbage-collected.
  • GET /datums/{datum-hash} → 📖 API Reference

• 📌 #21 New HTTP endpoint to retrieve native & Plutus' script pre-image from a script hash digest. Behind the scene, Kupo now track any script found in transactions' witnesses set, auxiliary data and/or outputs (reference scripts).
  • GET /scripts/{script-hash} → 📖 API Reference

• 📌 #40 (🔨 @nielstron) New HTTP endpoint to retrieve patterns that includes a given pattern. Useful to check if an address is matched by a given configuration.
  • GET /patterns/{pattern-fragment}[/{pattern-fragment}] → 📖 API Reference

• 📌 #40 New optional command-line option --gc-interval to tweak the interval between database's garbage collection. Each garbage collection takes a bit of time (few seconds) and pauses the indexer while doing so; A too short interval may have a strong impact on overall syncing time. A too long interval may increase the time needed for collecting garbage. Optimal values depends on your use-case, but the default (10 minutes) is seemingly a sensible default. While syncing from scratch with very permissive patterns, you may want to increase this value (e.g. to 60 minutes) to avoid needlessly pausing the synchronization.

• 📌 #24 - New HTTP endpoint to retrieve a point on-chain from a given slot. The endpoint is flexible and allows for retrieving the ancestor
  of a known point very easily. This is handy in combination with other protocols that leverage on-chain points and intersections (like Ogmios' chain-sync).

  • GET /checkpoints/{slot-no}[?strict] → 📖 API Reference

• 📌 #35 - New HTTP header X-Most-Recent-Checkpoint to every (successful) response. It contains the slot number of the current database most recent checkpoint. This allows client to know which slot a certain query is accurate of.

Changed

• 📌 #17 - The slot_no and header_hash fields are no longer accessible on top-level match result objects. Instead, they're now nested under a created_at field, analogous to how spent_at has been introduced.

• 📌 #24 - Fixed a bug where listing checkpoints would sometimes return duplicate entries.

• 📌 #39 - Inserting a new pattern (i.e. PUT /patterns/{pattern-fragment}) now requires to provide a rollback point, to which the server will rollback and start synchronizing again. The old behavior can be recovered by simply passing the most recent checkpoint as a rollback point. Note that, you may add an already existing pattern if you only need, for some reason, to rollback the indexer to some previous point in time. See the 📖 API Reference for details.

• 📌 #48 - (Massively) improved performance of query by stake credential. This used to be in linear time in the size of the UTxO set, and is now performed in logarithmic time, same as query by address. Querying by payment credentials is still performed in linear times though so this is probably something you want to avoid doing on permissive patterns (e.g. * or */*).

• API endpoints are no longer versioned (i.e. prefixed with v1). However, providing v1 will still work and route requests all-the-same to ensure backward-compatibility. The rationale being that, since kupo is a local service (and thus, clients decide when they want to upgrade), there's no particular need to version the API in the request path.

• Fixed a bug where the server would systematically reject any request to dynamically remove a pattern (because deemed overlapping with existing patterns).

Removed

• N/A

Snapshots

In the release artifacts, you can find two snapshots (kupo.sqlite3-{testnet, mainnet}.tar.gz). These snapshots were obtained from the following options:

• --match *
• --since origin
• --prune-utxo

They can be used as starting points to get started quickly.

Dependencies

Dependency	Tested With
cardano-node	1.35.x
ogmios	v5.5.x

Details

v1.0.1...v2.0.0-beta

Acknowledgments

Thanks a lot to @Quantumplation, @MartinSchere, @bakon11, @waalge, @baymac and @mmahut for their feedback and ideas.