Testnet update

MystenLabs · Oct 9, 2024 · c425f4b · c425f4b
1 parent 552b353
commit c425f4b
Show file tree

Hide file tree

Showing 15 changed files with 189 additions and 39 deletions.
diff --git a/contracts/blob_store/README.md b/contracts/blob_store/README.md
@@ -1,9 +1,10 @@
-# Walrus Devnet Move contracts
+# Walrus Testnet Move contracts
+ <!-- Update latest package ID -->
 
-This is the Move source code for the Walrus Devnet instance. We provide this so developers can
+This is the Move source code for the Walrus Testnet instance. We provide this so developers can
 experiment with building Walrus apps that require Move extensions. This code is published on Sui
 Testnet at package ID `0x7e12d67a52106ddd5f26c6ff4fe740ba5dea7cfc138d5b1d33863ba9098aa6fe`.
 
-**A word of caution:** Walrus Testnet will use new Move packages with struct layouts and function
+**A word of caution:** Walrus Mainnet will use new Move packages with struct layouts and function
 signatures that may not be compatible with this package. Move code that builds against this package
-will need to rewritten.
+will need to adapted.
diff --git a/docs/README.md b/docs/README.md
@@ -11,19 +11,19 @@ Walrus behind the scenes. See the [Walrus Sites chapter](./walrus-sites/intro.md
 details on how this works.
 ```
 
-```admonish danger title="Disclaimer about the Walrus developer preview"
-The current Devnet release of Walrus and Walrus Sites is a developer preview intended to showcase
-the technology and solicit feedback from builders. All storage nodes and aggregators are operated by
-Mysten Labs and all transactions are executed on the Sui Testnet and use Testnet SUI which has no
+```admonish danger title="Disclaimer about the Walrus Testnet"
+The current Testnet release of Walrus and Walrus Sites is a preview intended to showcase
+the technology and solicit feedback from builders, users and storage node operators.
+All transactions are executed on the Sui Testnet and use Testnet WAL and SUI which has no
 value. The state of the store **can and will be wiped**, at any point and possibly with no warning.
-Do not rely on this developer preview for any production purposes, it comes with no availability or
+Do not rely on this Testnet for any production purposes, it comes with no availability or
 persistence guarantees.
 
 Furthermore, encodings and blob IDs may be incompatible with the future Testnet and Mainnet and
-developers will be responsible for migrating any Devnet applications and data to Testnet. Detailed
-migration guides will be provided when Testnet becomes available.
+developers will be responsible for migrating any Testnet applications and data to Mainnet. Detailed
+migration guides will be provided when Mainnet becomes available.
 
-Also see the [Devnet terms of service](../tos.md) under which this developer preview is made
+Also see the [Testnet terms of service](../testnet_tos.md) under which this Testnet is made
 available.
 ```
 
@@ -49,7 +49,14 @@ confidentiality.
   for coordination, attesting availability, and payments. Storage space is represented as a resource
   on Sui, which can be owned, split, merged, and transferred. Stored blobs are also represented by
   objects on Sui, which means that smart contracts can check whether a blob is available and for how
-  long.
+  long, extend its lifetime or optionally delete it.
+
+- **Epochs, Tokenomics and Delegated Proof of Stake** Walrus is operated by a committee of storage
+  nodes that evolve between epochs. A native token, WAL (and its subdivision FROST), is used
+  to delegate stake to storage nodes, and those with high stake become part of the epoch committee.
+  The WAL token is also used for payments for storage. At the end of each epoch rewards for
+  selecting storage nodes, storing and serving blobs are distributed to storage nodes and whose that
+  stake with them. All these processes are mediated by smart contracts on the Sui platform.
 
 - **Flexible access:** Users can interact with Walrus through a command-line interface (CLI),
   software development kits (SDKs), and web2 HTTP technologies. Walrus is designed to work well

diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md
@@ -12,6 +12,7 @@
   - [Announcing Walrus](./blog/01_announcing_walrus.md)
   - [2024-08-12 Devnet Update](./blog/02_devnet_update.md)
   - [Announcing the Walrus Whitepaper](./blog/03_whitepaper.md)
+  - [2024-10-17 Testnet Update](./blog/04_testnet_update.md)
 
 ---
 
@@ -64,4 +65,5 @@
 ---
 
 [Glossary](./glossary.md)
-[Devnet terms of service](./tos.md)
+[Devnet terms of service](./devnet_tos.md)
+[Testnet terms of service](./testnet_tos.md)
diff --git a/docs/blog/04_testnet_update.md b/docs/blog/04_testnet_update.md
@@ -0,0 +1,9 @@
+# 2024-10-17 Testnet Update
+<!-- TODO write this section -->
+
+* Deletable blobs and reclaiming storage
+* External Storage nodes
+* WAL Token used for payments and WAL <> SUI faucet
+* Staking and Unstaking with Staking App
+* Epoch change and shard migration
+* Explorer
diff --git a/docs/dev-guide/dev-guide.md b/docs/dev-guide/dev-guide.md
@@ -15,17 +15,17 @@ This developer guide describes the following:
 Refer again to the [glossary](../glossary.md) of terms as a reference.
 
 ```admonish danger title="Disclaimer about the Walrus developer preview"
-The current Devnet release of Walrus and Walrus Sites is a developer preview intended to showcase
-the technology and solicit feedback from builders. All storage nodes and aggregators are operated by
-Mysten Labs and all transactions are executed on the Sui Testnet and use Testnet SUI which has no
+The current Testnet release of Walrus and Walrus Sites is a preview intended to showcase
+the technology and solicit feedback from builders, users and storage node operators.
+All transactions are executed on the Sui Testnet and use Testnet WAL and SUI which has no
 value. The state of the store **can and will be wiped**, at any point and possibly with no warning.
-Do not rely on this developer preview for any production purposes, it comes with no availability or
+Do not rely on this Testnet for any production purposes, it comes with no availability or
 persistence guarantees.
 
 Furthermore, encodings and blob IDs may be incompatible with the future Testnet and Mainnet and
-developers will be responsible for migrating any Devnet applications and data to Testnet. Detailed
-migration guides will be provided when Testnet becomes available.
+developers will be responsible for migrating any Testnet applications and data to Mainnet. Detailed
+migration guides will be provided when Mainnet becomes available.
 
-Also see the [Devnet terms of service](../tos.md) under which this developer preview is made
+Also see the [Testnet terms of service](../testnet_tos.md) under which this Testnet is made
 available.
 ```
diff --git a/docs/dev-guide/dev-operations.md b/docs/dev-guide/dev-operations.md
@@ -39,9 +39,9 @@ maximum blob size is currently 957&nbsp;MiB. You may store larger blobs by split
 smaller chunks.
 
 Blobs are stored for a certain number of *epochs*, as specified at the time they were stored. Walrus
-storage nodes ensure that within these epochs a read succeeds. The Walrus Devnet only uses a single
-epoch today, and blobs uploaded will be available in that single epoch (until the Devnet is wiped).
-Future devnets may span across multiple epochs.
+storage nodes ensure that within these epochs a read succeeds. The current Testnet uses a short
+epoch duration of one day for testing purposes, but Mainnet epochs are likely to be longer such as
+many weeks each.
 
 ## Read
 

diff --git a/docs/dev-guide/sui-struct.md b/docs/dev-guide/sui-struct.md
@@ -7,7 +7,7 @@ querying or executing transactions on Sui directly. However, Walrus uses Sui to
 and smart contract developers can read information about the Walrus system, as well as stored blobs,
 on Sui.
 
-The Move code of the Walrus Devnet contracts is available at
+The Move code of the Walrus Testnet contracts is available at
 <https://github.com/MystenLabs/walrus-docs/blob/main/contracts/blob_store>. An example package using
 the Walrus contracts is available at
 <https://github.com/MystenLabs/walrus-docs/blob/main/examples/move>.
@@ -16,7 +16,7 @@ The following sections provide further insights into the contract and an overvie
 Walrus objects in your own Sui smart contracts.
 
 ```admonish danger title="A word of caution"
-Walrus Testnet will use new Move packages with struct layouts and function signatures that may not
+Walrus Mainnet will use new Move packages with struct layouts and function signatures that may not
 be compatible with this package. Move code that builds against this package will need to rewritten.
 ```
 

diff --git a/docs/tos.md → docs/devnet_tos.md b/docs/tos.md → docs/devnet_tos.md
diff --git a/docs/operator-guide/aggregator.md b/docs/operator-guide/aggregator.md
@@ -1,4 +1,5 @@
 # Operating an aggregator
+<!-- (TODO - with example cache setup) -->
 
 Below is an example of an aggregator node which hosts a HTTP endpoint that can be used
 to fetch data from Walrus over the web.

diff --git a/docs/operator-guide/operator-guide.md b/docs/operator-guide/operator-guide.md
@@ -1,7 +1,8 @@
 # Operator guide
 
 This chapter introduces all the concepts needed for operators of the different components that make
-up the Walrus system. It is currently a work in progress and will be updated as the platform grows.
+up the Walrus system. It is currently a work in progress and will be updated as the platform
+matures.
 
 Specifically, this guide describes the following:
 

diff --git a/docs/operator-guide/storage-node.md b/docs/operator-guide/storage-node.md
@@ -1,7 +1,8 @@
 # Operating a storage node
+<!-- (TODO - update with operator instructions) -->
 
-The binary of the storage node is not yet publicly available. It will be made available in September
-to operators for Testnet nodes. Prior to official network launch the code will be open-sourced.
+The binary of the storage node is not yet publicly available. Prior to official network launch the
+code will be open-sourced.
 
 A basic systemd service running the Storage Node could look like this:
 

diff --git a/docs/testnet_tos.md b/docs/testnet_tos.md
@@ -0,0 +1,50 @@
+<!-- markdownlint-disable proper-names -->
+# TESTNET TERMS OF SERVICE - WALRUS
+<!-- Update for testnet -->
+
+Last updated: June 13, 2024
+
+By using Mysten Labs Devnet software, technologies, tools, and other services (collectively
+“Devnet”), you agree to the general Terms of Service and these additional Devnet Terms of Service
+(together, the “Terms”). If you do not agree, do not participate in Devnet. If you are using Devnet
+on behalf of an organization, you represent and warrant that you are an authorized representative of
+that organization and have the authority to bind that business or entity to the Terms.
+
+## Eligibility Criteria
+
+You may use Devnet only if you:
+
+- Are 18 years or older and capable of forming a binding contract with us.
+- Are not otherwise barred from participating in Devnet under applicable law.
+
+We may, at our discretion, introduce new or change existing eligibility criteria or conditions we
+deem appropriate. Devnet may operate in certain phases, and your participation in any one phase of
+Devnet does not guarantee that you will be selected for any other phases of Devnet.
+
+## Duration
+
+Devnet will commence on the date we prescribe and continue until terminated at our discretion. We
+may change, discontinue, or wipe, temporarily or permanently, all or any part of Devnet, at any
+time and without notice at our discretion, including, without limitation, the modification of the
+presence, amounts, or any other conditions applicable to data you have stored within Devnet, without
+any liability to you or other Devnet users.
+
+## No Warranty
+
+Mysten Labs provides the Devnet platform solely as a developer preview. Devnet is provided "as is"
+and "with all faults." We make no warranties, express or implied, regarding the reliability,
+accuracy, performance, or fitness for a particular purpose of the service provided. You accept all
+risks associated with the use of Devnet and agree that Mysten Labs, its affiliates, and its
+employees shall not be liable for any damages, whether direct, indirect, incidental, special,
+consequential, or punitive, arising out of the use or inability to use the service, including but
+not limited to lost profits, loss of business, or data loss.
+
+No employee or representative of Mysten Labs is authorized to make any warranties or representations
+beyond those stated in this agreement. Any statements made by employees or representatives of Mysten
+Labs regarding the service shall not be construed as warranties or representations, and customers
+agree to indemnify and hold harmless Mysten Labs from any such statements.
+
+Any deficiencies or errors in the Devnet platform shall not constitute a breach of this agreement,
+and customers agree to waive any right to seek a refund or compensation based on such deficiencies
+or errors. This "as is, no warranty" provision shall survive the termination or expiration of any
+other agreements between you and Mysten Labs.
diff --git a/docs/usage/client-cli.md b/docs/usage/client-cli.md
@@ -15,12 +15,13 @@ their meaning.
 ## Walrus system information
 
 Information about the Walrus system is available through the `walrus info` command. For example,
+<!-- (TODO - update with final) -->
 
 ```console
 $ walrus info
 
 Walrus system information
-Current epoch: 0
+Current epoch: 54
 
 Storage nodes
 Number of nodes: 10
@@ -31,18 +32,19 @@ Maximum blob size: 13.3 GiB (14,273,391,930 B)
 Storage unit: 1.00 KiB
 
 Approximate storage prices per epoch
-Price per encoded storage unit: 50 MIST
-Price to store metadata: 0.0031 SUI
-Marginal price per additional 1 MiB (w/o metadata): 241,950 MIST
+Price per encoded storage unit: 5 FROST
+Price to store metadata: 0.0003 WAL
+Marginal price per additional 1 MiB (w/o metadata): 24,195 FROST
 
 Total price for example blob sizes
-16.0 MiB unencoded (135 MiB encoded): 0.0069 SUI per epoch
-512 MiB unencoded (2.33 GiB encoded): 0.122 SUI per epoch
-13.3 GiB unencoded (60.5 GiB encoded): 3.174 SUI per epoch
+16.0 MiB unencoded (135 MiB encoded): 0.0007 WAL per epoch
+512 MiB unencoded (2.33 GiB encoded): 0.012 WAL per epoch
+13.3 GiB unencoded (60.5 GiB encoded): 0.317 WAL per epoch
+
 ```
 
 gives an overview of the number of storage nodes and shards in the system, the maximum blob size,
-and the current cost in (Testnet) SUI for storing blobs.
+and the current cost in (Testnet) WAL for storing blobs. (Note: 1 WAL = 1 000 000 000 FROST)
 
 Additional information such as encoding parameters and sizes, BFT system information, and
 information on the storage nodes and their shard distribution can be viewed with the `--dev`
@@ -94,6 +96,50 @@ By default the blob data is written to the standard output. The `--out <OUT>` CL
 can be used to specify an output file name. The `--rpc-url <URL>` (or `-r`) may be used to specify
 a Sui RPC node to use instead of the one set in the wallet configuration or the default one.
 
+## Reclaiming space via deletable blobs
+
+By default `walrus store` uploads a blob and Walrus will keep it available until after its expiry
+epoch. Not even the uploader may delete it beforehand. However, optionally, the store command
+may be invoked with the `--deletable` flag, to indicate the blob may be deleted before its expiry
+by the owner of the Sui blob object representing the blob. Deletable blobs are indicated as such
+in the Sui events that certify them, and should not be relied upon for availability by others.
+
+A deletable blob may be deleted with the command:
+```
+walrus delete --blob-id <BLOB_ID>
+```
+Optionally the delete command can be invoked by specifying a `--file <PATH>` option, to derive the
+blob ID from a file, or `--object-id <SUI_ID>` to delete the blob in the Sui blob object specified.
+
+The `delete` command reclaims the storage object associated with the deleted blob, which is
+re-used to store new blobs. The delete operation provides
+flexibility around managing storage costs and re-using storage.
+
+The delete operation has limited utility for privacy: It only deletes slivers from the current
+epoch storage nodes, and subsequent epoch storage nodes, if no other user has uploaded a copy of
+the same blob. If another copy of the same blob exists in Walrus the delete operation will not
+make the blob unavailable for download, and `walrus read` invocations will download it. Copies of
+the public blob may be cached or downloaded by users, and these copies are not deleted.
+
+```admonish danger title="Delete reclaims space only"
+**All blobs stored in Walrus are public and discoverable by all.** The `delete` command will
+not delete slivers if other copies of the blob are stored on Walrus possibly by other users.
+It does not delete blobs from caches, slivers from past storage nodes, or copies
+that could have been made by users before the blob was deleted.
+```
+
+## Blob ID utilities
+
+The `walrus blob-id <FILE>` may be used to derive the blob ID of any file. The blob ID is a
+commitment to the file, and any blob with the same ID will decode to the same content. The blob
+ID is a 256 bit number and represented on some Sui explorer as a decimal large number. The
+command `walrus convert-blob-id <BLOB_ID_DECIMAL>` may be used to convert it to a base64 URL safe
+encoding used by the command line tools and other APIs.
+
+The `walrus list-blobs` command lists all the non expired Sui blob object that the current account
+owns, including their blob ID, object ID, and metadata about expiry and deletable status.
+The option `--include-expired` also lists expired blob objects.
+
 ## Changing the default configuration
 
 Use the `--config` option to specify a custom path to the

diff --git a/docs/usage/setup.md b/docs/usage/setup.md
@@ -7,7 +7,7 @@ patterns (see [the next chapter](./interacting.md)). This chapter describes the
 of the Walrus client.
 
 ```admonish note
-Note that our Walrus Devnet uses Sui **Testnet** for coordination.
+Note that the Walrus Testnet uses Sui **Testnet** for coordination.
 ```
 
 ## Prerequisites
@@ -131,6 +131,36 @@ In addition to the latest version of the `walrus` binary, the GCS bucket also co
 versions. An overview in XML format is available at
 <https://storage.googleapis.com/mysten-walrus-binaries/>.
 
+## Testnet WAL faucet
+
+The Walrus Testnet uses Testnet WAL tokens to buy storage and stake. Testnet WAL tokens have no
+value and can be exchanged for some Testnet SUI tokens, which also have no value, thought the
+command:
+
+```
+walrus get-wal
+```
+
+You can check you have received Testnet WAL by checking the Sui balances:
+
+```
+sui client balance
+╭─────────────────────────────────────────╮
+│ Balance of coins owned by this address  │
+├─────────────────────────────────────────┤
+│ ╭─────────────────────────────────────╮ │
+│ │ coin  balance (raw)     balance     │ │
+│ ├─────────────────────────────────────┤ │
+│ │ Sui   8869252670        8.86 SUI    │ │
+│ │ WAL   500000000         0.50 WAL    │ │
+│ ╰─────────────────────────────────────╯ │
+╰─────────────────────────────────────────╯
+```
+
+By default 0.5 SUI are exchanged for 0.5 WAL, but a different amount of SUI may be exchanged
+using the `--amount` option, and a specific coin ID may be used through the `--exchange-id`.
+The `walrus get-wal --help` command provides more information about those.
+
 ## Configuration
 
 A single parameter is required to configure Walrus, namely the ID of the [system
@@ -153,6 +183,8 @@ you need to use the `--config` option when running the `walrus` binary.
 
 ### Advanced configuration (optional)
 
+<!-- TODO: Latest config -->
+
 The configuration file currently supports the following parameters:
 
 ```yaml

diff --git a/examples/move/walrus_dep/README.md b/examples/move/walrus_dep/README.md
@@ -1,3 +1,3 @@
 # Example Move package depending on Walrus
 
-A simple example of depending on the Walrus Devnet Move package.
+A simple example of depending on the Walrus Testnet Move package.