docs(blockchain-scanner): Update documentation for the new `zebra-sca…

…nner` binary (#8675)

* validate zebrad cache dir

* update scanner documentation

* clippy

* Apply suggestions from code review

Co-authored-by: Arya <[email protected]>

* add info level by default to scanner application

* remove `RUST_LOG=info` from some of the test examples on how to run

* remove `RUST_LOG=info` from the book sample

---------

Co-authored-by: Arya <[email protected]>

Cargo.lock

@@ -5995,7 +5995,7 @@ dependencies = [
  "thiserror",
  "tokio",
  "tokio-stream",
- "tonic 0.11.0",
+ "tonic",
  "tonic-build 0.11.0",
  "tonic-reflection",
  "tower",
@@ -6039,7 +6039,7 @@ dependencies = [
  "tempfile",
  "tokio",
  "toml 0.8.14",
- "tonic 0.11.0",
+ "tonic",
  "tower",
  "tracing",
  "tracing-subscriber",

book/src/user/shielded-scan-grpc-server.md

@@ -4,14 +4,12 @@
 
 ### Setup
 
-After setting up [Zebra Shielded Scanning](https://zebra.zfnd.org/user/shielded-scan.html), add a `listen_addr` field to the shielded-scan configuration:
+After setting up [Zebra Shielded Scanning](https://zebra.zfnd.org/user/shielded-scan.html), you can add a `listen-addr` argument to the scanner binary:
 
-```toml
-[shielded_scan]
-listen_addr = "127.0.0.1:8231"
-```
 
-Then, run `zebrad` to start the scan gRPC server.
+```bash
+zebra-scanner --sapling-keys-to-scan '{"key":"zxviews1q0duytgcqqqqpqre26wkl45gvwwwd706xw608hucmvfalr759ejwf7qshjf5r9aa7323zulvz6plhttp5mltqcgs9t039cx2d09mgq05ts63n8u35hyv6h9nc9ctqqtue2u7cer2mqegunuulq2luhq3ywjcz35yyljewa4mgkgjzyfwh6fr6jd0dzd44ghk0nxdv2hnv4j5nxfwv24rwdmgllhe0p8568sgqt9ckt02v2kxf5ahtql6s0ltjpkckw8gtymxtxuu9gcr0swvz", "birthday_height": 419200}' --zebrad-cache-dir /media/alfredo/stuff/chain/zebra --zebra-rpc-listen-addr '127.0.0.1:8232' --listen-addr '127.0.0.1:8231'
+```
 
 Making requests to the server will also require a gRPC client, the examples here use `grpcurl`, though any gRPC client should work.
 

book/src/user/shielded-scan.md

@@ -1,44 +1,49 @@
 # Zebra Shielded Scanning
 
-This document describes Zebra's shielded scanning from users' perspective.
+The `zebra-scanner` binary is a standalone application that utilizes Zebra libraries to scan for transactions associated with specific Sapling viewing keys. It stores the discovered transactions and scanning progress data in a RocksDB database.
 
+For this application to function, it requires access to a Zebra node's RPC server and state cache.
+
 For now, we only support Sapling, and only store transaction IDs in the scanner results database.
+
 Ongoing development is tracked in issue [#7728](https://github.com/ZcashFoundation/zebra/issues/7728).
 
 ## Important Security Warning
 
 Zebra's shielded scanning feature has known security issues. It is for experimental use only.
 
-Do not use regular or sensitive viewing keys with Zebra's experimental scanning
-feature. Do not use this feature on a shared machine. We suggest generating new
-keys for experimental use or publicly known keys.
+Do not use regular or sensitive viewing keys with Zebra's experimental scanning feature. Do not use this feature on a shared machine. We suggest generating new keys for experimental use or using publicly known keys.
 
 ## Build & Install
 
-Use [Zebra 1.6.0](https://github.com/ZcashFoundation/zebra/releases/tag/v1.6.0)
-or greater, or the `main` branch to get the latest features, and enable the
-`shielded-scan` feature during the build. You can also use Rust's `cargo` to
-install the latest release:
+Use [Zebra 1.9.0](https://github.com/ZcashFoundation/zebra/releases/tag/v1.9.0) or greater, or the `main` branch to get the latest features.
+
+You can also use Rust's `cargo` to install `zebra-scanner` from the latest release Zebra repository:
 
 ```bash
-cargo install --features shielded-scan --locked --git https://github.com/ZcashFoundation/zebra zebrad
+cargo install --locked --git https://github.com/ZcashFoundation/zebra zebra-scan
 ```
 
-Zebra binary will be at `~/.cargo/bin/zebrad`, which should be in your `PATH`.
+The scanner binary will be at `~/.cargo/bin/zebra-scanner`, which should be in your `PATH`.
 
-## Configuration
+## Arguments
 
-Generate a configuration file with the default settings:
+Retrieve the binary arguments with:
 
 ```bash
-zebrad generate -o ~/.config/zebrad.toml
+zebra-scanner --help
 ```
 
+## Scanning the Block Chain
+
+Before starting, ensure a `zebrad` node is running locally with the RPC endpoint open. Refer to the [lightwalletd zebrad setup](https://zebra.zfnd.org/user/lightwalletd.html#configure-zebra-for-lightwalletd) or [zebrad mining setup](https://zebra.zfnd.org/user/mining.html#configure-zebra-for-mining) for instructions.
+
+To initiate the scanning process, you need the following:
 
-In the generated `zebrad.toml` file, use:
+- A zebrad cache state directory. This can be obtained from the running zebrad configuration file, under the `state` section in the `cache_dir` field.
+- A key to scan with, optionally including a birthday height, which specifies the starting height for the scanning process for that key.
+- A zebrad RPC endpoint address. This can be found in the running zebrad configuration file, under the `rpc` section in the `listen_addr` field.
 
-- the `[shielded_scan]` table for database settings, and
-- the `[shielded_scan.sapling_keys_to_scan]` table for diversifiable full viewing keys.
 
 Sapling diversifiable/extended full viewing keys strings start with `zxviews` as
 described in
@@ -47,44 +52,29 @@ described in
 For example, to scan the block chain with the [public ZECpages viewing
 key](https://zecpages.com/boardinfo), use:
 
-```toml
-[shielded_scan.sapling_keys_to_scan]
-"zxviews1q0duytgcqqqqpqre26wkl45gvwwwd706xw608hucmvfalr759ejwf7qshjf5r9aa7323zulvz6plhttp5mltqcgs9t039cx2d09mgq05ts63n8u35hyv6h9nc9ctqqtue2u7cer2mqegunuulq2luhq3ywjcz35yyljewa4mgkgjzyfwh6fr6jd0dzd44ghk0nxdv2hnv4j5nxfwv24rwdmgllhe0p8568sgqt9ckt02v2kxf5ahtql6s0ltjpkckw8gtymxtxuu9gcr0swvz" = 419200
-```
-
-Where the number 419200 is the birthday of the key:
-- birthday lower than the Sapling activation height defaults to Sapling activation height.
-- birthday greater or equal than Sapling activation height will start scanning at provided height, improving scanner speed.
-
-## Scanning the Block Chain
-
-Simply run
-
 ```bash
-zebrad
+RUST_LOG=info zebra-scanner --sapling-keys-to-scan '{"key":"zxviews1q0duytgcqqqqpqre26wkl45gvwwwd706xw608hucmvfalr759ejwf7qshjf5r9aa7323zulvz6plhttp5mltqcgs9t039cx2d09mgq05ts63n8u35hyv6h9nc9ctqqtue2u7cer2mqegunuulq2luhq3ywjcz35yyljewa4mgkgjzyfwh6fr6jd0dzd44ghk0nxdv2hnv4j5nxfwv24rwdmgllhe0p8568sgqt9ckt02v2kxf5ahtql6s0ltjpkckw8gtymxtxuu9gcr0swvz", "birthday_height": 419200}' --zebrad-cache-dir /media/alfredo/stuff/chain/zebra --zebra-rpc-listen-addr '127.0.0.1:8232'
 ```
 
-The scanning will start once Zebra syncs its state past the Sapling activation
-height. Scanning a synced state takes between 12 and 24 hours. The scanner looks
-for transactions containing Sapling notes with outputs decryptable by the
-provided viewing keys.
+- A birthday lower than the Sapling activation height defaults to Sapling activation height.
+- A birthday greater or equal than Sapling activation height will start scanning at provided height, improving scanner speed.
+
+The scanning process begins once Zebra syncs its state past the Sapling activation height. Scanning a synced state takes between 12 and 24 hours. The scanner searches for transactions containing Sapling notes with outputs decryptable by the provided viewing keys.
 
-You should see log messages in the output every 10 000 blocks scanned, similar
-to:
+You will see log messages in the output every 10,000 blocks scanned, similar to:
 
 ```
-2023-12-16T12:14:41.526740Z  INFO zebra_scan::storage::db: Last scanned height for key number 0 is 435000, resuming at 435001
-2023-12-16T12:14:41.526745Z  INFO zebra_scan::storage::db: loaded Zebra scanner cache
 ...
-2023-12-16T12:15:19.063796Z  INFO {zebrad="39830b0" net="Main"}: zebra_scan::scan: Scanning the blockchain for key 0, started at block 435001, now at block 440000, current tip 2330550
+2024-07-13T16:07:47.944309Z  INFO zebra_scan::service::scan_task::scan: Scanning the blockchain for key 0, started at block 571001, now at block 580000, current tip 2556979
+2024-07-13T16:08:07.811013Z  INFO zebra_scan::service::scan_task::scan: Scanning the blockchain for key 0, started at block 571001, now at block 590000, current tip 2556979
 ...
 ```
 
-The Zebra scanner will resume the task if your Zebra instance went down for any
-reason. In a new start, Zebra will display:
+If your Zebra instance goes down for any reason, the Zebra scanner will resume the task. Upon a new start, Zebra will display:
 
 ```
-Last scanned height for key number 0 is 1798000, resuming at 1798001
+2024-07-13T16:07:17.700073Z  INFO zebra_scan::storage::db: Last scanned height for key number 0 is 590000, resuming at 590001
+2024-07-13T16:07:17.706727Z  INFO zebra_scan::service::scan_task::scan: got min scan height start_height=Height(590000)
 ```
 
 ## Displaying Scanning Results

zebra-scan/src/bin/scanner/main.rs

@@ -10,7 +10,7 @@ use zebra_chain::{block::Height, parameters::Network};
 use zebra_state::SaplingScanningKey;
 
 use core::net::SocketAddr;
-use std::path::PathBuf;
+use std::path::{Path, PathBuf};
 
 /// A structure with sapling key and birthday height.
 #[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize)]
@@ -34,15 +34,22 @@ impl std::str::FromStr for SaplingKey {
 #[tokio::main]
 /// Runs the zebra scanner binary with the given arguments.
 async fn main() -> Result<(), Box<dyn std::error::Error>> {
-    // Display all logs from the zebra-scan crate.
+    // Display logs with `info` level by default.
+    let tracing_filter: String = match std::env::var("RUST_LOG") {
+        Ok(val) if !val.is_empty() => val,
+        _ => "info".to_string(),
+    };
+
     tracing_subscriber::fmt::fmt()
-        .with_env_filter(tracing_subscriber::EnvFilter::from_default_env())
+        .with_env_filter(tracing_filter)
         .init();
 
     // Parse command line arguments.
     let args = Args::from_args();
 
     let zebrad_cache_dir = args.zebrad_cache_dir;
+    validate_dir(&zebrad_cache_dir)?;
+
     let scanning_cache_dir = args.scanning_cache_dir;
     let mut db_config = zebra_scan::Config::default().db_config;
     db_config.cache_dir = scanning_cache_dir;
@@ -142,3 +149,19 @@ pub struct Args {
     #[structopt(long)]
     pub listen_addr: Option<SocketAddr>,
 }
+
+/// Create an error message is a given directory does not exist or we don't have access to it for whatever reason.
+fn validate_dir(dir: &Path) -> Result<(), Box<dyn std::error::Error>> {
+    match dir.try_exists() {
+        Ok(true) => Ok(()),
+        Ok(false) => {
+            let err_msg = format!("directory {} does not exist.", dir.display());
+            error!("{}", err_msg);
+            Err(std::io::Error::new(std::io::ErrorKind::NotFound, err_msg).into())
+        }
+        Err(e) => {
+            error!("directory {} could not be accessed: {:?}", dir.display(), e);
+            Err(e.into())
+        }
+    }
+}

zebra-scan/tests/scanner.rs

@@ -38,7 +38,7 @@ fn scanner_help() -> eyre::Result<()> {
 /// To run it locally, one way is:
 ///
 /// ```
-/// RUST_LOG=info cargo test scan_binary_starts -- --include-ignored --nocapture
+/// cargo test scan_binary_starts -- --nocapture
 /// ```
 #[tokio::test]
 #[cfg(not(target_os = "windows"))]
@@ -126,7 +126,7 @@ async fn scan_binary_starts() -> Result<()> {
 /// Needs a cache state close to the tip. A possible way to run it locally is:
 ///
 /// export ZEBRA_CACHED_STATE_DIR="/path/to/zebra/state"
-/// RUST_LOG=info cargo test scan_start_where_left -- --ignored --nocapture
+/// cargo test scan_start_where_left -- --ignored --nocapture
 ///
 /// The test will run zebrad with a key to scan, scan the first few blocks after sapling and then stops.
 /// Then it will restart zebrad and check that it resumes scanning where it was left.