Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add syncing documentation #23344

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 93 additions & 0 deletions docs/learn/advanced/18-syncing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
---
sidebar_position: 1
---

# Syncing

Syncing is the process of downloading the blockchain and state from a remote node.
ziscky marked this conversation as resolved.
Show resolved Hide resolved

There are two types of syncing:

1. Blockchain syncing.
With block sync a node is downloading all of the data of an application from genesis and verifying it.
2. State syncing.
With state sync your node will download data related to the head or near the head of the chain and verify the data. This leads to drastically shorter times for joining a network.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
There are two types of syncing:
1. Blockchain syncing.
With block sync a node is downloading all of the data of an application from genesis and verifying it.
2. State syncing.
With state sync your node will download data related to the head or near the head of the chain and verify the data. This leads to drastically shorter times for joining a network.
There are few types to sync a node:
1. Genesis Sync: operator downloads genesis, sets peers and syncs the blockchain by executing all blocks since genesis.
NOTE: requires a peer node that doesn't prune blocks (CometBFT state).
1. State Sync: operator need to set `trust_height` and `trust_hash` of a block that s/he trusts as well as `trust_period` - the time how long the operator will trust the given block height. The app will query peer nodes to stream state from that blocks as well as all following blocks. Once the state is downloaded, the node will start executing all blocks since the `trust_height`.
NOTE: requires a peer node that provides a state sync and is trusted by the validator.
1. Snapshot Sync: operator downloads a snapshot from snapshot provider (usually validators) s/he trusts. From there the operator needs to unpack the snapshot to the node directory, update config, set peers, set moniker, setup validator keys. Once the data directory is properly configured, the validator can start a node. The node will query blocks since the snapshot and execute them.
NOTE: requires a snapshot directory that is trusted by the validator.
Genesis Sync has zero trust assumptions, but it's the most resource heavy. It also requires a node with all blocks - many validators prune the blocks to optimize space.
Snapshot sync is the fastest and the most efficient: it downloads snapshot using a content delivery network or a protocol optimized for data dalivery, rather than streaming from existing nodes. Once you have a snapshot you can easily resync from the same snapshot if you have problems. State sync might be more complicated.



## Observing syncing progress

> ### Note: This section applies to comet users.
ziscky marked this conversation as resolved.
Show resolved Hide resolved

### Block Sync Metrics

- `blocksync_syncing`: Indicates whether a node is currently block syncing.
- `blocksync_num_txs`: Number of transactions in the latest block.
- `blocksync_total_txs`: Total number of transactions.
- `blocksync_block_size_bytes`: Size of the latest block in bytes.
- `blocksync_latest_block_height`: Height of the latest block.
ziscky marked this conversation as resolved.
Show resolved Hide resolved

### Block sync configuration

```toml
blocksync:
version: "v0"
```

- `version`: The version of the block sync protocol to use.

### State Sync Metrics

- `statesync_syncing`: Indicates whether a node is currently state syncing.

### State sync configuration

```toml
statesync:
enable: true
rpc_servers: ""
trust_height: 0
trust_hash: ""
trust_period: 168h0m0s
discovery_time: 15s
temp_dir: ""
chunk_request_timeout: 10s
chunk_fetchers: "4"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not a toml syntax. The config snippet should be updated to a proper toml snippet.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, same for "Block sync configuration" section earlier on

```

- `enable`: Set to true to enable state sync.
- `rpc_servers`: Comma-separated list of RPC servers for state sync.
- `trust_height`: Block height to trust for state sync.
- `trust_hash`: Block hash to trust for state sync.
- `trust_period`: Trust period for light client verification.
- `discovery_time`: Time to spend discovering snapshots before picking one.
- `temp_dir`: Directory for temporary state sync files.
- `chunk_request_timeout`: Timeout for chunk requests.
- `chunk_fetchers`: Number of concurrent chunk fetchers.
ziscky marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of creating another section to explain params, let's explain them in the config code using comments:

[statesync]
// Comma-separated list of RPC servers for state sync.
rpc_servers = ...


### Checking if sync is complete

Query for the node status using the REST or GRPC API:

REST example:
```bash
curl http://localhost:1317/cosmos/base/tendermint/v1beta1/syncing
```
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can also check with app q consensus comet syncing


Expected response:
```json
{
"syncing": false
}
```

GRPC example:
```bash
grpcurl -plaintext localhost:9090 cosmos.base.tendermint.v1beta1.Service/GetSyncing
```

The response includes `SyncInfo.CatchingUp` field
Syncing is complete when this field is `false`
ziscky marked this conversation as resolved.
Show resolved Hide resolved




Loading