Document not found (404)
-This URL is invalid, sorry. Please use the navigation bar or search to continue.
- -
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/dev-guide/dev-guide.html b/pr-preview/pr-70/dev-guide/dev-guide.html
deleted file mode 100644
index 5c836ffb..00000000
--- a/pr-preview/pr-70/dev-guide/dev-guide.html
+++ /dev/null
@@ -1,249 +0,0 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- Components
-From a developer perspective, some Walrus components are objects and smart contracts on -Sui, and some components are Walrus-specific binaries and services. As a rule Sui is used to -manage blob and storage node metadata, while Walrus-specific services are used to store and -read blob contents, which can be very large.
-Walrus defines a number of objects and smart contracts on Sui:
--
-
- A shared system object, records and manages the current committee of storage nodes. -
- Storage resources, represent empty storage space that may be used to store blobs. -
- Blob resources, represent blobs being registered and certified as stored. -
- Changes to these objects emit Walrus-related events. -
The Walrus system object ID can be found in the Walrus client_config.yaml file (see
-Configuration). You may use any Sui explorer to look at its
-content, as well as explore the content of blob objects. There is more information about these in
-the quick reference to the Walrus Sui structures.
Walrus is also composed of a number of Walrus-specific services and binaries:
--
-
- A client (binary) can be executed locally and provides a -Command Line Interface (CLI), a JSON API -and an HTTP API to perform Walrus operations. -
- Aggregators services allow reading blobs via HTTP requests. -
- Publishers services are used store blobs to Walrus. -
- A set of storage nodes store encoded blobs. These nodes form the decentralized -storage infrastructure of Walrus. -
Aggregators, publishers and other services use the client APIs to interact with Walrus. End-users -of services using Walrus interact with the store via custom services, aggregators or publishers that -expose HTTP APIs to avoid the need to run locally a binary client.
- -
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/dev-guide/dev-operations.html b/pr-preview/pr-70/dev-guide/dev-operations.html
deleted file mode 100644
index bf2e6f0f..00000000
--- a/pr-preview/pr-70/dev-guide/dev-operations.html
+++ /dev/null
@@ -1,295 +0,0 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- Developer Guide
-This guide introduces all the concepts needed to build applications that use Walrus as a storage -or availability layer. The overview provides more background and explains -in more detail how Walrus operates internally.
-This developer guide describes:
--
-
- Components of Walrus of interest to developers that wish to use it for -storage or availability. -
- Operations supported through client binaries, APIs, or Sui operations. -
- The Sui structures Walrus uses to store metadata, and how they can be read -from Sui smart contracts, or through the Sui SDK. -
A glossary of terms used is also available for reference.
-Disclaimer about the Walrus developer preview
-This release of Walrus & Walrus Sites is a -developer preview, to showcase the technology and solicit feedback from builders. All storage nodes -and aggregators are operated by Mysten Labs, all transactions are executed on the Sui testnet, -and use testnet SUI which has no value. The state of the store can be, and will be wiped, at any -point and possibly with no warning. Do not rely on this developer preview for any production -purposes, it comes with no availability or persistence guarantees.
-Also see the Devnet terms of service under which this developer preview is made -available.
- -
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/dev-guide/sui-struct.html b/pr-preview/pr-70/dev-guide/sui-struct.html
deleted file mode 100644
index ad332517..00000000
--- a/pr-preview/pr-70/dev-guide/sui-struct.html
+++ /dev/null
@@ -1,360 +0,0 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- Operations
-Blob encoding and blob ID
-Walrus stores blobs across storage nodes in an encoded form, and refers -to blobs by their blob ID. The blob ID is deterministically derived from the content of a blob -and the Walrus configuration. The blob ID of two files with the same content will be the same.
-You can derive the blob ID of a file locally using the command: walrus blob-id <file path>
Store
-Walrus may be used to store a blob, via the native client APIs or a publisher.
---All blobs -stored in Walrus are public and discoverable by all. Therefore you must not use Walrus to store -anything that contains secrets or private data without additional measures to protect -confidentiality.
-
Under the hood a number of operations happen both on Sui as well as on storage nodes:
--
-
- The client or publisher encodes the blob and derives a blob ID that identifies the blob. This
-is a
u256often encoded as a URL safe base64 string.
- - A transaction is executed on Sui to purchase some storage from the system object, and then to -register the blob ID occupying this storage. Client APIs return the Sui blob object ID. The -transactions use SUI to purchase storage and pay for gas. -
- Encoded slivers of the blob are distributed to all storage nodes. They each sign a receipt. -
- Signed receipts are aggregated and submitted to the Sui blob object to certify the blob. -Certifying a blob emits a Sui event with the blob ID and the period of availability. -
A blob is considered available on Walrus once the corresponding Sui blob object has been -certified in the final step. The steps involved in a store operation can be executed by the binary -client, or a publisher that accepts and publishes blobs via HTTP.
-Walrus currently allows the storage of blobs up to a maximum size that may be determined
-through the walrus info CLI command. The
-maximum blob size is currently 957 MiB. You may store larger blobs by splitting them into
-smaller chunks.
Blobs are stored for a certain number of epochs, as specified at the time they were stored. Walrus -storage nodes ensure that within these epochs a read succeeds. The Walrus devnet only uses a single -epoch today, and blobs uploaded will be available in that single epoch (until the devnet is wiped). -Future devnets may span across multiple epochs.
-Read
-Walrus can also be used to read a blob after it is stored by providing its blob ID. -A read is executed by performing the following steps:
--
-
- The system object on Sui is read to determine the Walrus storage node committee. -
- A number of storage nodes are queried for blob metadata and the slivers they store. -
- The blob is reconstructed from the recovered slivers and checked against the blob ID. -
The steps involved in the read operation are performed by the binary client, or the aggregator -service that exposes an HTTP interface to read blobs. Reads are extremely resilient and will -succeed in recovering the blob stored even if up to one-third of storage nodes are -unavailable in all cases. Eventually, after synchronization is complete, even if two-thirds -of storage nodes are down reads will succeed.
-Certify Availability
-Walrus can be used to certify the availability of a blob using Sui. Checking that this happened -may currently be done in 3 different ways:
--
-
- A Sui SDK read can be
-used to authenticate the certified blob event emitted when the blob ID was certified on Sui. The
-client
walrus blob-statuscommand may be used to identify the event ID that needs to be checked.
- - A Sui SDK read may be -used to authenticate the Sui blob object corresponding to the blob ID, and check it is certified. -
- A Sui smart contract can read the blob object on Sui (or a reference to it) to check -is is certified. -
The underlying protocol of the -Sui light client -returns digitally signed evidence for emitted events -or objects, and can be used by off-line or non-interactive applications as a proof of availability -for the blob ID for a certain number of epochs.
-Once a blob is certified, Walrus will ensure that sufficient slivers will always be -available on storage nodes to recover it within the specified epochs.
- -
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/elasticlunr.min.js b/pr-preview/pr-70/elasticlunr.min.js
deleted file mode 100644
index 94b20dd2..00000000
--- a/pr-preview/pr-70/elasticlunr.min.js
+++ /dev/null
@@ -1,10 +0,0 @@
-/**
- * elasticlunr - http://weixsong.github.io
- * Lightweight full-text search engine in Javascript for browser search and offline search. - 0.9.5
- *
- * Copyright (C) 2017 Oliver Nightingale
- * Copyright (C) 2017 Wei Song
- * MIT Licensed
- * @license
- */
-!function(){function e(e){if(null===e||"object"!=typeof e)return e;var t=e.constructor();for(var n in e)e.hasOwnProperty(n)&&(t[n]=e[n]);return t}var t=function(e){var n=new t.Index;return n.pipeline.add(t.trimmer,t.stopWordFilter,t.stemmer),e&&e.call(n,n),n};t.version="0.9.5",lunr=t,t.utils={},t.utils.warn=function(e){return function(t){e.console&&console.warn&&console.warn(t)}}(this),t.utils.toString=function(e){return void 0===e||null===e?"":e.toString()},t.EventEmitter=function(){this.events={}},t.EventEmitter.prototype.addListener=function(){var e=Array.prototype.slice.call(arguments),t=e.pop(),n=e;if("function"!=typeof t)throw new TypeError("last argument must be a function");n.forEach(function(e){this.hasHandler(e)||(this.events[e]=[]),this.events[e].push(t)},this)},t.EventEmitter.prototype.removeListener=function(e,t){if(this.hasHandler(e)){var n=this.events[e].indexOf(t);-1!==n&&(this.events[e].splice(n,1),0==this.events[e].length&&delete this.events[e])}},t.EventEmitter.prototype.emit=function(e){if(this.hasHandler(e)){var t=Array.prototype.slice.call(arguments,1);this.events[e].forEach(function(e){e.apply(void 0,t)},this)}},t.EventEmitter.prototype.hasHandler=function(e){return e in this.events},t.tokenizer=function(e){if(!arguments.length||null===e||void 0===e)return[];if(Array.isArray(e)){var n=e.filter(function(e){return null===e||void 0===e?!1:!0});n=n.map(function(e){return t.utils.toString(e).toLowerCase()});var i=[];return n.forEach(function(e){var n=e.split(t.tokenizer.seperator);i=i.concat(n)},this),i}return e.toString().trim().toLowerCase().split(t.tokenizer.seperator)},t.tokenizer.defaultSeperator=/[\s\-]+/,t.tokenizer.seperator=t.tokenizer.defaultSeperator,t.tokenizer.setSeperator=function(e){null!==e&&void 0!==e&&"object"==typeof e&&(t.tokenizer.seperator=e)},t.tokenizer.resetSeperator=function(){t.tokenizer.seperator=t.tokenizer.defaultSeperator},t.tokenizer.getSeperator=function(){return t.tokenizer.seperator},t.Pipeline=function(){this._queue=[]},t.Pipeline.registeredFunctions={},t.Pipeline.registerFunction=function(e,n){n in t.Pipeline.registeredFunctions&&t.utils.warn("Overwriting existing registered function: "+n),e.label=n,t.Pipeline.registeredFunctions[n]=e},t.Pipeline.getRegisteredFunction=function(e){return e in t.Pipeline.registeredFunctions!=!0?null:t.Pipeline.registeredFunctions[e]},t.Pipeline.warnIfFunctionNotRegistered=function(e){var n=e.label&&e.label in this.registeredFunctions;n||t.utils.warn("Function is not registered with pipeline. This may cause problems when serialising the index.\n",e)},t.Pipeline.load=function(e){var n=new t.Pipeline;return e.forEach(function(e){var i=t.Pipeline.getRegisteredFunction(e);if(!i)throw new Error("Cannot load un-registered function: "+e);n.add(i)}),n},t.Pipeline.prototype.add=function(){var e=Array.prototype.slice.call(arguments);e.forEach(function(e){t.Pipeline.warnIfFunctionNotRegistered(e),this._queue.push(e)},this)},t.Pipeline.prototype.after=function(e,n){t.Pipeline.warnIfFunctionNotRegistered(n);var i=this._queue.indexOf(e);if(-1===i)throw new Error("Cannot find existingFn");this._queue.splice(i+1,0,n)},t.Pipeline.prototype.before=function(e,n){t.Pipeline.warnIfFunctionNotRegistered(n);var i=this._queue.indexOf(e);if(-1===i)throw new Error("Cannot find existingFn");this._queue.splice(i,0,n)},t.Pipeline.prototype.remove=function(e){var t=this._queue.indexOf(e);-1!==t&&this._queue.splice(t,1)},t.Pipeline.prototype.run=function(e){for(var t=[],n=e.length,i=this._queue.length,o=0;n>o;o++){for(var r=e[o],s=0;i>s&&(r=this._queue[s](r,o,e),void 0!==r&&null!==r);s++);void 0!==r&&null!==r&&t.push(r)}return t},t.Pipeline.prototype.reset=function(){this._queue=[]},t.Pipeline.prototype.get=function(){return this._queue},t.Pipeline.prototype.toJSON=function(){return this._queue.map(function(e){return t.Pipeline.warnIfFunctionNotRegistered(e),e.label})},t.Index=function(){this._fields=[],this._ref="id",this.pipeline=new t.Pipeline,this.documentStore=new t.DocumentStore,this.index={},this.eventEmitter=new t.EventEmitter,this._idfCache={},this.on("add","remove","update",function(){this._idfCache={}}.bind(this))},t.Index.prototype.on=function(){var e=Array.prototype.slice.call(arguments);return this.eventEmitter.addListener.apply(this.eventEmitter,e)},t.Index.prototype.off=function(e,t){return this.eventEmitter.removeListener(e,t)},t.Index.load=function(e){e.version!==t.version&&t.utils.warn("version mismatch: current "+t.version+" importing "+e.version);var n=new this;n._fields=e.fields,n._ref=e.ref,n.documentStore=t.DocumentStore.load(e.documentStore),n.pipeline=t.Pipeline.load(e.pipeline),n.index={};for(var i in e.index)n.index[i]=t.InvertedIndex.load(e.index[i]);return n},t.Index.prototype.addField=function(e){return this._fields.push(e),this.index[e]=new t.InvertedIndex,this},t.Index.prototype.setRef=function(e){return this._ref=e,this},t.Index.prototype.saveDocument=function(e){return this.documentStore=new t.DocumentStore(e),this},t.Index.prototype.addDoc=function(e,n){if(e){var n=void 0===n?!0:n,i=e[this._ref];this.documentStore.addDoc(i,e),this._fields.forEach(function(n){var o=this.pipeline.run(t.tokenizer(e[n]));this.documentStore.addFieldLength(i,n,o.length);var r={};o.forEach(function(e){e in r?r[e]+=1:r[e]=1},this);for(var s in r){var u=r[s];u=Math.sqrt(u),this.index[n].addToken(s,{ref:i,tf:u})}},this),n&&this.eventEmitter.emit("add",e,this)}},t.Index.prototype.removeDocByRef=function(e){if(e&&this.documentStore.isDocStored()!==!1&&this.documentStore.hasDoc(e)){var t=this.documentStore.getDoc(e);this.removeDoc(t,!1)}},t.Index.prototype.removeDoc=function(e,n){if(e){var n=void 0===n?!0:n,i=e[this._ref];this.documentStore.hasDoc(i)&&(this.documentStore.removeDoc(i),this._fields.forEach(function(n){var o=this.pipeline.run(t.tokenizer(e[n]));o.forEach(function(e){this.index[n].removeToken(e,i)},this)},this),n&&this.eventEmitter.emit("remove",e,this))}},t.Index.prototype.updateDoc=function(e,t){var t=void 0===t?!0:t;this.removeDocByRef(e[this._ref],!1),this.addDoc(e,!1),t&&this.eventEmitter.emit("update",e,this)},t.Index.prototype.idf=function(e,t){var n="@"+t+"/"+e;if(Object.prototype.hasOwnProperty.call(this._idfCache,n))return this._idfCache[n];var i=this.index[t].getDocFreq(e),o=1+Math.log(this.documentStore.length/(i+1));return this._idfCache[n]=o,o},t.Index.prototype.getFields=function(){return this._fields.slice()},t.Index.prototype.search=function(e,n){if(!e)return[];e="string"==typeof e?{any:e}:JSON.parse(JSON.stringify(e));var i=null;null!=n&&(i=JSON.stringify(n));for(var o=new t.Configuration(i,this.getFields()).get(),r={},s=Object.keys(e),u=0;u
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- Sui Structures
-This section is optional and enables advanced use cases.
-You can interact with Walrus purely -through the client CLI, and JSON or HTTP APIs provided, without querying or executing transactions -on Sui directly. However, Walrus uses Sui to manage its metadata and smart contract developers can -read information about the Walrus system, as well as stored blobs, on Sui.
-This section provides an overview of how you may use Walrus objects in your Sui smart contracts.
-Blob and Storage Objects
-Walrus blobs are represented as Sui objects of type Blob. A blob is first registered, indicating
-that the storage nodes should expect slivers from a Blob ID to be stored. Then a blob is certified
-indicating that a sufficient number of slivers have been stored to guarantee the blob's
-availability. When a blob is certified its certified_epoch field contains the epoch in which it
-was certified.
A Storage object is always associated with a Blob object, reserving enough space for
-a long enough period for the blob's storage. A certified blob is available for the period the
-underlying storage resource guarantees storage.
Concretely, Blob and Storage objects have the following fields, that can be read through the
-Sui SDKs:
/// The blob structure represents a blob that has been registered to with some
-/// storage, and then may eventually be certified as being available in the
-/// system.
-public struct Blob has key, store {
- id: UID,
- stored_epoch: u64,
- blob_id: u256,
- size: u64,
- erasure_code_type: u8,
- certified_epoch: option::Option<u64>, // The epoch first certified,
- // or None if not certified.
- storage: Storage,
-}
-
-/// Reservation for storage for a given period, which is inclusive start,
-/// exclusive end.
-public struct Storage has key, store {
- id: UID,
- start_epoch: u64,
- end_epoch: u64,
- storage_size: u64,
-}
-All fields of Blob and Storage objects can be read using the expected functions:
// Blob functions
-public fun stored_epoch(b: &Blob) : u64;
-public fun blob_id(b: &Blob) : u256;
-public fun size(b: &Blob) : u64;
-public fun erasure_code_type(b: &Blob) : u8;
-public fun certified_epoch(b: &Blob) : &Option<u64>;
-public fun storage(b: &Blob) : &Storage;
-
-// Storage functions
-public fun start_epoch(self: &Storage) : u64;
-public fun end_epoch(self: &Storage) : u64;
-public fun storage_size(self: &Storage) : u64;
-Events
-When a blob is first registered a BlobRegistered event is emitted that informs storage nodes
-that they should expect slivers associated with its Blob ID. Eventually when the blob is
-certified a BlobCertified is emitted containing information about the blob ID and the epoch
-after which the blob will be deleted. Before that epoch the blob is guaranteed to be available.
/// Signals a blob with metadata is registered.
-public struct BlobRegistered has copy, drop {
- epoch: u64,
- blob_id: u256,
- size: u64,
- erasure_code_type: u8,
- end_epoch: u64,
-}
-
-/// Signals a blob is certified.
-public struct BlobCertified has copy, drop {
- epoch: u64,
- blob_id: u256,
- end_epoch: u64,
-}
-The InvalidBlobID event is emitted when storage nodes detect an incorrectly encoded blob.
-Anyone attempting a read on such a blob is guaranteed to also detect it as invalid.
/// Signals that a BlobID is invalid.
-public struct InvalidBlobID has copy, drop {
- epoch: u64, // The epoch in which the blob ID is first registered as invalid
- blob_id: u256,
-}
-System information
-The Walrus system object contains metadata about the available and used storage, as well as the -price of storage per Kib of storage in MIST. The committee -structure within the system object can be used to read the current epoch number, as well as -information about the committee.
-const BYTES_PER_UNIT_SIZE : u64 = 1_024;
-
-public struct System<phantom WAL> has key, store {
-
- id: UID,
-
- /// The current committee, with the current epoch.
- /// The option is always Some, but need it for swap.
- current_committee: Option<Committee>,
-
- /// When we first enter the current epoch we SYNC,
- /// and then we are DONE after a cert from a quorum.
- epoch_status: u8,
-
- // Some accounting
- total_capacity_size : u64,
- used_capacity_size : u64,
-
- /// The price per unit size of storage.
- price_per_unit_size: u64,
-
- /// Tables about the future and the past.
- past_committees: Table<u64, Committee>,
- future_accounting: FutureAccountingRingBuffer<WAL>,
-}
-
-public struct Committee has store {
- epoch: u64,
- bls_committee : BlsCommittee,
-}
-A few public functions of the committee allow contracts to read Walrus metadata:
-/// Get epoch. Uses the committee to get the epoch.
-public fun epoch<WAL>(self: &System<WAL>) : u64;
-
-/// Accessor for total capacity size.
-public fun total_capacity_size<WAL>(self: &System<WAL>) : u64;
-
-/// Accessor for used capacity size.
-public fun used_capacity_size<WAL>(self: &System<WAL>) : u64;
-
-// The number of shards
-public fun n_shards<WAL>(self: &System<WAL>) : u16;
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/highlight.css b/pr-preview/pr-70/highlight.css
deleted file mode 100644
index ba57b82b..00000000
--- a/pr-preview/pr-70/highlight.css
+++ /dev/null
@@ -1,82 +0,0 @@
-/*
- * An increased contrast highlighting scheme loosely based on the
- * "Base16 Atelier Dune Light" theme by Bram de Haan
- * (http://atelierbram.github.io/syntax-highlighting/atelier-schemes/dune)
- * Original Base16 color scheme by Chris Kempson
- * (https://github.com/chriskempson/base16)
- */
-
-/* Comment */
-.hljs-comment,
-.hljs-quote {
- color: #575757;
-}
-
-/* Red */
-.hljs-variable,
-.hljs-template-variable,
-.hljs-attribute,
-.hljs-tag,
-.hljs-name,
-.hljs-regexp,
-.hljs-link,
-.hljs-name,
-.hljs-selector-id,
-.hljs-selector-class {
- color: #d70025;
-}
-
-/* Orange */
-.hljs-number,
-.hljs-meta,
-.hljs-built_in,
-.hljs-builtin-name,
-.hljs-literal,
-.hljs-type,
-.hljs-params {
- color: #b21e00;
-}
-
-/* Green */
-.hljs-string,
-.hljs-symbol,
-.hljs-bullet {
- color: #008200;
-}
-
-/* Blue */
-.hljs-title,
-.hljs-section {
- color: #0030f2;
-}
-
-/* Purple */
-.hljs-keyword,
-.hljs-selector-tag {
- color: #9d00ec;
-}
-
-.hljs {
- display: block;
- overflow-x: auto;
- background: #f6f7f6;
- color: #000;
-}
-
-.hljs-emphasis {
- font-style: italic;
-}
-
-.hljs-strong {
- font-weight: bold;
-}
-
-.hljs-addition {
- color: #22863a;
- background-color: #f0fff4;
-}
-
-.hljs-deletion {
- color: #b31d28;
- background-color: #ffeef0;
-}
diff --git a/pr-preview/pr-70/highlight.js b/pr-preview/pr-70/highlight.js
deleted file mode 100644
index 18d24345..00000000
--- a/pr-preview/pr-70/highlight.js
+++ /dev/null
@@ -1,54 +0,0 @@
-/*
- Highlight.js 10.1.1 (93fd0d73)
- License: BSD-3-Clause
- Copyright (c) 2006-2020, Ivan Sagalaev
-*/
-var hljs=function(){"use strict";function e(n){Object.freeze(n);var t="function"==typeof n;return Object.getOwnPropertyNames(n).forEach((function(r){!Object.hasOwnProperty.call(n,r)||null===n[r]||"object"!=typeof n[r]&&"function"!=typeof n[r]||t&&("caller"===r||"callee"===r||"arguments"===r)||Object.isFrozen(n[r])||e(n[r])})),n}class n{constructor(e){void 0===e.data&&(e.data={}),this.data=e.data}ignoreMatch(){this.ignore=!0}}function t(e){return e.replace(/&/g,"&").replace(//g,">").replace(/"/g,""").replace(/'/g,"'")}function r(e,...n){var t={};for(const n in e)t[n]=e[n];return n.forEach((function(e){for(const n in e)t[n]=e[n]})),t}function a(e){return e.nodeName.toLowerCase()}var i=Object.freeze({__proto__:null,escapeHTML:t,inherit:r,nodeStream:function(e){var n=[];return function e(t,r){for(var i=t.firstChild;i;i=i.nextSibling)3===i.nodeType?r+=i.nodeValue.length:1===i.nodeType&&(n.push({event:"start",offset:r,node:i}),r=e(i,r),a(i).match(/br|hr|img|input/)||n.push({event:"stop",offset:r,node:i}));return r}(e,0),n},mergeStreams:function(e,n,r){var i=0,s="",o=[];function l(){return e.length&&n.length?e[0].offset!==n[0].offset?e[0].offset
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- Walrus Glossary
-To make communication as clear and efficient as possible, we make sure to use a single term for -every Walrus entity/concept and do not use any synonyms. The following table lists various -concepts, their canonical name, and how they relate to or differ from other terms.
-Italicized terms in the description indicate other specific Walrus terms contained in the table.
-| Approved name | Description |
|---|---|
| storage node (SN) | entity storing data for Walrus; holds one or several shards |
| blob | single unstructured data object stored on Walrus |
| shard | (disjoint) subset of erasure-encoded data of all blobs; at every point in time, a shard is assigned to and stored on a single SN |
| sliver | erasure-encoded data of one shard corresponding to a single blob for one of the two encodings; this contains several erasure-encoded symbols of that blob but not the blob metadata |
| blob ID | cryptographic ID computed from a blob’s slivers |
| blob metadata | metadata of one blob; in particular, this contains a hash per shard to enable the authentication of slivers and recovery symbols |
| (end) user | any entity/person that wants to store or read blobs on/from Walrus; can act as a Walrus client itself or use the simple interface exposed by publishers and caches |
| publisher | service interacting with Sui and the SNs to store blobs on Walrus; offers a simple HTTP POST endpoint to end users |
| aggregator | service that reconstructs blobs by interacting with SNs and exposes a simple HTTP GET endpoint to end users |
| cache | an aggregator with additional caching capabilities |
| (Walrus) client | entity interacting directly with the SNs; this can be an aggregator/cache, a publisher, or an end user |
| (blob) reconstruction | decoding of the primary slivers to obtain the blob; includes re-encoding the blob and checking the Merkle proofs |
| (shard/sliver) recovery | process of an SN recovering a sliver or full shard by obtaining recovery symbols from other SNs |
| storage attestation | process where SNs exchange challenges and responses to demonstrate that they are storing their currently assigned shards |
| certificate of availability (CoA) | a blob ID with signatures of SNs holding at least $2f+1$ shards in a specific epoch |
| point of availability (PoA) | point in time when a CoA is submitted to Sui and the corresponding blob is guaranteed to be available until its expiration |
| inconsistency proof | set of several recovery symbols with their Merkle proofs such that the decoded sliver does not match the corresponding hash; this proves an incorrect/inconsistent encoding by the client |
| inconsistency certificate | an aggregated signature from 2/3 of SNs (weighted by their number of shards) that they have seen and stored an inconsistency proof for a blob ID |
| storage committee | the set of SNs for a storage epoch, including metadata about the shards they are responsible for and other metadata |
| member | an SN that is part of a committee at some epoch |
| storage epoch | the epoch for Walrus as distinct to the epoch for Sui |
| availability period | the period specified in storage epochs for which a blob is certified to be available on Walrus |
| expiry | the end epoch at which a blob is no longer available and can be deleted |
":e:f.tabReplace?e.replace(/\t/g,f.tabReplace):e):e}function E(e){let n=null;const t=function(e){var n=e.className+" ";n+=e.parentNode?e.parentNode.className:"";const t=f.languageDetectRe.exec(n);if(t){var r=T(t[1]);return r||(console.warn(g.replace("{}",t[1])),console.warn("Falling back to no-highlight mode for this block.",e)),r?t[1]:"no-highlight"}return n.split(/\s+/).find(e=>p(e)||T(e))}(e);if(p(t))return;S("before:highlightBlock",{block:e,language:t}),f.useBR?(n=document.createElement("div")).innerHTML=e.innerHTML.replace(/\n/g,"").replace(/
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/mark.min.js b/pr-preview/pr-70/mark.min.js
deleted file mode 100644
index 16362318..00000000
--- a/pr-preview/pr-70/mark.min.js
+++ /dev/null
@@ -1,7 +0,0 @@
-/*!***************************************************
-* mark.js v8.11.1
-* https://markjs.io/
-* Copyright (c) 2014–2018, Julian Kühnel
-* Released under the MIT license https://git.io/vwTVl
-*****************************************************/
-!function(e,t){"object"==typeof exports&&"undefined"!=typeof module?module.exports=t():"function"==typeof define&&define.amd?define(t):e.Mark=t()}(this,function(){"use strict";var e="function"==typeof Symbol&&"symbol"==typeof Symbol.iterator?function(e){return typeof e}:function(e){return e&&"function"==typeof Symbol&&e.constructor===Symbol&&e!==Symbol.prototype?"symbol":typeof e},t=function(e,t){if(!(e instanceof t))throw new TypeError("Cannot call a class as a function")},n=function(){function e(e,t){for(var n=0;n
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- Walrus
-Welcome to the developer documentation for Walrus, a decentralized storage and data availability -protocol designed specifically for large binary files, or "blobs". Walrus focuses on providing a -robust, but affordable solution for storing unstructured content on decentralized storage nodes -while ensuring high availability and reliability even in the presence of Byzantine faults.
-Fun fact: If you are viewing this site at https://docs.walrus.site, you are fetching this from -Walrus behind the scenes. See the Walrus Sites chapter for further -details on how this works.
-Features
--
-
-
-
Storage and retrieval: Walrus supports storage operations to write and read blobs. It also -allows anyone to prove that a blob has been stored and is available for retrieval at a later -time.
-
- -
-
Cost efficiency: By utilizing advanced error correction coding, Walrus maintains storage -costs at approximately five times the size of the stored blobs and encoded parts of each blob -are stored on each storage node. This is significantly more cost-effective compared to -traditional full replication methods and much more robust against failures compared to -protocols that only store each blob on a subset of storage nodes.
-
- -
-
Integration with Sui blockchain: Walrus leverages Sui -for coordination, attesting availability and payments. Storage space can be owned as a resource on -Sui, split, merged, and transferred. Blob storage is represented using storage objects on Sui, and -smart contracts can check whether a blob is available and for how long.
-
- -
-
Flexible access: Users can interact with Walrus through a command-line interface (CLI), -software development kits (SDKs), and web2 HTTP technologies. Walrus is designed to work well -with traditional caches and content distribution networks (CDNs), while ensuring all operations -can also be run using local tools to maximize decentralization.
-
-
Architecture and operations
-Walrus's architecture ensures that content remains accessible and retrievable even when many -storage nodes are unavailable or malicious. Under the hood it uses modern error correction -techniques based on fast linear fountain codes, augmented to ensure resilience against Byzantine -faults, and a dynamically changing set of storage nodes. The core of Walrus remains simple, and -storage node management and blob certification leverages Sui smart contracts.
-Organization
-This documentation is split into three parts:
--
-
- About Walrus describes the objectives, security properties, and architecture of Walrus. -
- Usage provides concrete information for developers. If you want to get started quickly, you can -jump directly to the setup chapter. -
- Walrus sites describes how you can use Walrus and Sui together to build truly decentralized -websites. -
Finally, we provide a glossary that explains the terminology used throughout the -documentation.
-Sources
-This documentation is built using mdBook from source files in -github.com/MystenLabs/walrus-docs/. Please report or -fix any errors you find in this documentation in that GitHub project.
- -
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/overview/encoding.html b/pr-preview/pr-70/overview/encoding.html
deleted file mode 100644
index 233d74f0..00000000
--- a/pr-preview/pr-70/overview/encoding.html
+++ /dev/null
@@ -1,284 +0,0 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- Basic architecture and security assumptions
-The key actors in the Walrus architecture are the following:
--
-
-
-
Users through clients want to store and read blobs identified by their blob ID.
-These actors are ready to pay for service -when it comes to writes and non-best-effort reads. Users also want to prove -the availability of a blob to third parties without the cost of sending or receiving the full -blob.
-Users might be malicious in various ways: they might not want to pay for services, prove the -availability of unavailable blobs, modify/delete blobs without authorization, try to -exhaust resources of storage nodes, and so on.
-
- -
-
Storage nodes hold one or many shards within a storage epoch.
-Each blob is erasure -encoded in many slivers. Slivers from each stored blob become part of all shards. A shard -at any storage epoch is associated with a storage node that actually stores all slivers of -the shard and is ready to serve them.
-A Sui smart contract controls the assignment of storage nodes to shards within -storage epochs and Walrus assumes that more than 2/3 of the -shards are managed by correct storage nodes within each storage epoch. This means that Walrus must -tolerate up to 1/3 of the shards managed by Byzantine storage nodes (approximately 1/3 of the -storage nodes being Byzantine) within each storage epoch and across storage epochs.
-
- -
-
All clients and storage nodes operate a blockchain client (specifically on Sui), and mediate -payments, resources (space), mapping of shards to storage nodes and metadata through blockchain -smart contracts. Users interact with the blockchain to acquire storage resources and upload -certificates for stored blobs. Storage nodes listen to the blockchain events to coordinate -their operations.
-
-
Walrus supports any additional number of optional infrastructure actors that can operate in a -permissionless way:
--
-
-
-
Caches are clients that store one or more full blobs and make them available to users -over traditional web2 technologies (such as HTTP).
-They are optional in that end users can also -operate a local cache, and perform Walrus reads over web2 technologies locally. However, cache -infrastructures can also act as CDNs, split the cost of blob reconstruction over many requests, -be better connected, and so on. A client can always verify that reads from such infrastructures -are correct.
-
- -
-
Publishers are clients that help end users store a blob using web2 technologies, -using less bandwidth and custom logic.
-In effect, they receive the blob to be published over -traditional web2 protocols (like HTTP) and run the Walrus store protocol on the end user's -behalf. This includes encoding the blob into slivers, distributing the slivers to shards, -collecting storage-node signatures and aggregating them into a certificate, as well as all -other on-chain actions.
-They are optional in that a user can directly interact with Sui and -the storage nodes to store blobs. An end user can always verify that a publisher -performed their duties correctly by checking that an event associated with the -Point of Availability for the blob exists on chain -and then either performing a read to see if Walrus returns the blob or encoding the blob -and comparing the result to the blob ID in the certificate.
-
-
Caches, publishers and end users are not considered trusted components of the system, and they might -deviate from the protocol arbitrarily. However, some of the security properties of Walrus only hold -for honest end users that use honest intermediaries (caches and publishers). Walrus provides a -means for end users to audit the correct operation of both caches and publishers.
- -
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/overview/future.html b/pr-preview/pr-70/overview/future.html
deleted file mode 100644
index 25dba198..00000000
--- a/pr-preview/pr-70/overview/future.html
+++ /dev/null
@@ -1,237 +0,0 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- Encoding, overheads and verification
-The following list summarizes the basic encoding and cryptographic techniques used in Walrus:
--
-
-
-
Storage nodes hold one or many shards in a storage epoch out of a larger total (1000, for -instance). Each shard contains one blob sliver for each blob past point of availability. Each -shard is assigned to a storage node in a storage epoch.
-
- -
-
An erasure code encode algorithm takes a blob -and encodes it as K symbols, such that any fraction p of symbols can be used to reconstruct -the blob. Each blob sliver contains a fixed number of such symbols.
-
- -
-
Walrus selects p < 1/3 so that a third of symbols and slivers can be used to reconstruct the blob -by the decode algorithm. The matrix used to produce the erasure code is fixed and is the same -for all blobs in the Walrus system, and encoders have no discretion about it.
-
- -
-
Storage nodes manage one or more shards, and corresponding slivers of each blob are distributed -to all the storage shards.
-As a result, the overhead of the distributed store is ~5x that of the blob itself, no matter how -many shards there are. The encoding is systematic, meaning that some storage nodes hold part of -the original blob, allowing for fast random access reads.
-
-
Each blob is also associated with some metadata including a blob ID to allow verification:
--
-
-
-
A blob ID is computed as an authenticator of the set of all shard data and metadata (byte -size, encoding, blob hash).
-Walrus hashes a sliver representation in each of the shards and adds the resulting -hashes into a Merkle tree. Then the root of the Merkle tree is the blob hash used to derive the -blob ID that identifies the blob in the system.
-
- -
-
Each storage node can use the blob ID to check if some shard data belongs to a blob using the -authenticated structure corresponding to the blob hash (Merkle tree). A successful check means -that the data is indeed as intended by the writer of the blob (who might be corrupt).
-
- -
-
When any party reconstructs a blob ID from shard slivers, or accepts any blob claiming -to be a specific blob ID, it must check that it encodes to the correct blob ID.
-This process involves re-coding the blob using the erasure correction code, and deriving the -blob ID again to check that the blob matches. This prevents a malformed blob (incorrectly -erasure coded) from ever being read as a valid blob at any correct recipient.
-
- -
-
A set of slivers equal to the reconstruction threshold belonging to a blob ID that are either -inconsistent or lead to the reconstruction of a different ID represent an incorrect encoding -(this happens only if the user that encoded the blob was malicious and encoded it incorrectly).
-Walrus can extract one symbol per sliver to form an inconsistency proof. -Storage nodes can delete slivers belonging to inconsistently encoded blobs, -and upon request return either the inconsistency proof or an inconsistency certificate posted -on chain.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/overview/objectives_use_cases.html b/pr-preview/pr-70/overview/objectives_use_cases.html
deleted file mode 100644
index 95b8df3e..00000000
--- a/pr-preview/pr-70/overview/objectives_use_cases.html
+++ /dev/null
@@ -1,312 +0,0 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- Future discussion
-In this document, we left out details of the following features:
--
-
- Shard transfer and recovery upon storage epoch change. The encoding scheme used has been designed -to allow this operation to be efficient. A storage node needs to only get data of the same -magnitude to the missing sliver data to reconstruct them. -
- Details of light clients that can be used to sample availability. Individual clients may sample -the certified blobs from Sui metadata, and sample the availability of some slivers that they -store. On-chain bounties may be used to retrieve these slivers for missing blobs. -
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/overview/operations-off-chain.html b/pr-preview/pr-70/overview/operations-off-chain.html
deleted file mode 100644
index 5df56db0..00000000
--- a/pr-preview/pr-70/overview/operations-off-chain.html
+++ /dev/null
@@ -1,364 +0,0 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- Objectives and use cases
-Walrus supports operations to store and retrieve blobs, and to prove and verify their availability. -It ensures content survives storage nodes suffering Byzantine faults and remains available and -retrievable. It provides APIs to access the stored content over a CLI, SDKs and over web2 HTTP -technologies, and supports content delivery infrastructures like caches and content distribution -networks (CDNs).
-Under the hood, storage cost is a small fixed multiple of the size of blobs (around 5x). -Advanced error correction coding keeps the cost low, in contrast to the full replication of data -traditional to blockchains, such as the >100x multiple for data stored in Sui objects. As a result, -storage of much bigger resources (up to several GiB) is possible on Walrus at substantially lower -cost compared to Sui or other blockchains. Because encoded blobs are stored on all storage nodes, -Walrus also provides superior robustness compared to designs with a small amount of replicas -storing the full blob.
-Walrus uses the Sui chain for coordination and payments. Available storage is represented as Sui -objects that can be acquired, owned, split, merged and transferred. Storage space can be tied to -a stored blob for a period of time, with the resulting Sui object used to prove -availability on chain in smart contracts, or off chain using light clients.
-The next chapter discusses the above operations relating to storage, -retrieval and availability in detail.
-In the future, we plan to include in Walrus some minimal governance to allow storage nodes to -change between storage epochs. Walrus is also compatible with periodic payments for continued -storage. We also plan to implement storage attestation based on challenges to get confidence -that blobs are stored or at least available. Walrus also allows light nodes that store small parts -of blobs to get rewards for proving availability and assisting recovery. We will cover these -topics in later documents. We also provide details of the encoding scheme in a separate document.
-Non-objectives
-There are a few things that Walrus explicitly is not:
--
-
-
-
Walrus does not reimplement a CDN that might be geo-replicated or have less than tens of -milliseconds of latency. Instead, it ensures that traditional CDNs are usable and compatible with -Walrus caches.
-
- -
-
Walrus does not re-implement a full smart contracts platform with consensus or execution. It -relies on Sui smart contracts when necessary, to manage Walrus resources and processes including -payments, storage epochs, and so on.
-
- -
-
Walrus supports storage of any blob, including encrypted blobs, however Walrus itself is not the -distributed key management infrastructure that manages and distributed encryption or decryption -keys to support a full private storage eco-system. It can, however, provide the storage layer for -such infrastructures.
-
-
Use cases
-App builders may use Walrus in conjunction with any L1 or L2 blockchains to build experiences that -require large amounts of data to be stored in a decentralized manner and possibly certified as -available:
--
-
-
-
Storage of media for NFT or dApps: Walrus can directly store and serve media such as images, -sounds, sprites, videos, other game assets, and so on. This is publicly available media that is -accessed using HTTP requests at caches to create multimedia dApps.
-
- -
-
AI related use cases: Walrus can store clean data sets of training data, datasets with a -known and verified provenance, models, weights and proofs of correct training for AI models. -It can also store and ensure the availability of an AI model output.
-
- -
-
Storage of long term archival of blockchain history: Walrus can act as a lower cost -decentralized store to store blockchain history. For Sui, this can include sequences of -checkpoints with all associated transaction and effects content, as well as historic snapshots -of the blockchain state, code or binaries.
-
- -
-
Support availability for L2s: Walrus allows parties to certify the availability of blobs, as -required by L2s that need data to be stored and be attested as available to all. This may also -include availability of extra audit data such as validity proofs, zero knowledge proofs of -correct execution or large fraud proofs.
-
- -
-
Support a fully decentralized web experience: Walrus can host fully decentralized web -experiences, including all resources (such as js, css, html, media). These can not only -provide content, but also host the UX of dApps to enable applications with fully decentralized -front end and back ends on chain. Walrus puts the full "web" into web3.
-
- -
-
Support subscription models for media: Creators can store encrypted media on Walrus and only -provide access via decryption keys to parties that have paid a subscription fee or have paid for -content. Walrus provides the storage, encryption and decryption needs to happen off -the system.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/overview/operations-sui.html b/pr-preview/pr-70/overview/operations-sui.html
deleted file mode 100644
index a56da67d..00000000
--- a/pr-preview/pr-70/overview/operations-sui.html
+++ /dev/null
@@ -1,294 +0,0 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- Off-chain operations
-While Walrus operations happen off Sui, they might interact with the blockchain flows defining the -resource life cycle.
-Write paths
-
Systems overview of writes, illustrated in the previous image:
--
-
-
-
A user acquires a storage resource of appropriate size and duration on chain, either by directly -buying it on the Walrus system object, or a secondary market. A user can split, merge and -transfer owned storage resources.
-
- -
-
When users want to write a blob, they first erasure code it using encode and compute the -blob ID. Then they can perform the following steps themselves, or use a publisher to perform steps -on their behalf.
-
- -
-
The user goes on chain (Sui) and updates a storage resource to register the blob ID with the -desired size and lifetime. This emits an event, received by storage nodes. After the -user receives they then continue the upload.
-
- -
-
The user sends each of the blob slivers and metadata to the storage nodes that currently -manages the corresponding shards.
-
- -
-
A storage node managing a shard receives a sliver and checks it against the blob ID. -It also checks that there is a blob resource with the blob ID that is authorized to store -a blob. If correct, the storage node then signs a statement that it holds the sliver for blob ID -(and metadata) and returns it to the user.
-
- -
-
The user puts together the signatures returned from storage nodes into an availability certificate -and sends it on chain. When the certificate is verified on chain, an availability event for the -blob ID is emitted, and all other storage nodes seek to download any missing shards for the blob -ID. This event emitted by Sui is the point of availability (PoA) for the blob -ID.
-
- -
-
After the PoA, and without user involvement, storage nodes sync and recover any missing slivers.
-
-
The user waits for 2/3 of shard signatures to return to create the certificate of -availability. The rate of the code is below 1/3, allowing for reconstruction even if only 1/3 of -shards return the sliver for a read. Because at most 1/3 of the storage nodes can fail, this ensures -reconstruction if a reader requests slivers from all storage nodes. The full process can -be mediated by a publisher that receives a blob and drives the process to completion.
-Refresh availability
-Because no content data is required to refresh the duration of storage, refresh is conducted fully -on chain within the protocol. To request an extension to the availability of a blob, a user provides -an appropriate storage resource. Upon success this emits an event that storage nodes receive to -extend the time for which each sliver is stored.
-Inconsistent resource flow
-When a correct storage node tries to reconstruct a shard it might fail if the encoding of a blob ID -past PoA was incorrect, but will instead extract an inconsistency proof for the -blob ID. It then uses the proof to create an inconsistency certificate and upload it on chain. -The flow is as follows:
--
-
-
-
A storage node fails to reconstruct a shard, and instead holds an inconsistency proof.
-
- -
-
The storage node sends the blob ID and inconsistency proof to all storage nodes of the Walrus -epoch. The storage node verifies the proof and signs it.
-
- -
-
The storage node aggregates the signatures into an inconsistency certificate and sends it to the -Walrus smart contract, which verifies it and emits a inconsistent resource event.
-
- -
-
Upon receiving an inconsistent resource event, correct storage nodes delete sliver data for the -blob ID and record in the metadata to return
-Nonefor the blob ID for the -availability period. No storage attestation challenges are issued for this -blob ID.
-
A blob ID that is inconsistent always resolves to None upon reading: this is because
-the read process re-encodes the received blob to check that the blob ID is correctly derived from a
-consistent encoding. This means that an inconsistency proof reveals only a true fact to storage
-nodes (that do not otherwise run decoding), and does not change the output of read in any case.
Note, however, that partial reads leveraging the systematic nature of the encoding might return -partial reads for inconsistently encoded files. Thus, if consistency and availability of reads is -important, dApps should do full reads rather than partial reads.
-Read paths
-A user can read stored blobs either directly or through a cache. The direct user journey is -discussed here because this is also how the cache operates in case of a cache miss. Assume that most -reads happen through caches for blobs that are hot, and do not result in requests to storage nodes.
--
-
-
-
The reader gets the metadata for the blob ID from any storage node, and authenticates it using -the blob ID.
-
- -
-
The reader then sends a request to the storage node for the shards corresponding to the blob ID, -and waits for f+1 to respond. Sufficient requests are sent in parallel to ensure low latency for -reads.
-
- -
-
The reader authenticates the slivers returned with the blob ID, reconstructs the blob, and decides -whether the contents are a valid blob or inconsistent.
-
- -
-
Optionally, for a cache, the result is cached and can be served without reconstruction until it is -evicted from the cache. Requests for the blob to the cache return the blob contents, or a proof -the blob is inconsistently encoded.
-
-
Challenge mechanism for storage attestation
-During an epoch, a correct storage node challenges all shards to provide blob slivers past PoA:
--
-
-
-
The list of available blobs for the epoch is determined by the sequence of Sui events up -to the past epoch. Inconsistent blobs are not challenged, and a record proving this status -can be returned instead.
-
- -
-
A challenge sequence is determined by providing a seed to the challenged shard. The sequence is -then computed based both on the seed AND the content of each challenged blob ID. This creates a -sequential read dependency.
-
- -
-
The response to the challenge provides the sequence of shard contents for the blob IDs in a -timely manner.
-
- -
-
The challenger node uses thresholds to determine whether the challenge was passed, and reports -the result on chain.
-
- -
-
The challenge/response communication is authenticated.
-
-
Challenges provide some reassurance that the storage node can actually recover shard data in a -probabilistic manner, avoiding storage nodes getting payment without any evidence they might -retrieve shard data. The sequential nature of the challenge and some reasonable timeout also ensures -that the process is timely.
- -
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/overview/operations.html b/pr-preview/pr-70/overview/operations.html
deleted file mode 100644
index 4cf6dbed..00000000
--- a/pr-preview/pr-70/overview/operations.html
+++ /dev/null
@@ -1,231 +0,0 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- Operations on Sui
-Walrus uses Sui smart contracts to coordinate storage operations as resources that have a lifetime, -and payments. Smart contracts also facilitate governance to determine the storage nodes holding each -storage shard. The following content outlines these operations and refers to them as part of the -read/write paths.
-Metadata is the only blob element ever exposed to Sui or its validators, as the content -of blobs is always stored off chain on Walrus storage nodes and caches. The storage nodes or caches -do not have to overlap with any Sui infrastructure components (such as validators), and the storage -epochs can be of different lengths and not have the same start/end times as Sui epochs.
-Storage resource life cycle on Sui
-A number of Sui smart contracts hold the metadata of the Walrus system and all its entities.
--
-
-
-
A Walrus system object holds the committee of storage nodes for the current storage epoch. The -system object also holds the total available space on Walrus and the price per unit of storage (1 -KiB).
-These values are determined by 2/3 agreement between the storage nodes for the storage -epoch. Users can pay to purchase storage space for some time duration. These space resources can -be split, merged and transferred. Later, they can be used to place a blob ID into Walrus.
-
- -
-
The storage fund holds funds for storing blobs over one storage epoch, multiple or -perpetually. When purchasing storage space from the system object, users pay into the storage fund -separated over multiple storage epochs. Payments are made each epoch to storage nodes -according to performance (details follow).
-
- -
-
A user acquires some storage through the contracts or transfer, and can assign to it a blob ID, -signifying they want to store this blob ID into it. This emits a Move resource event that -both caches and storage nodes listen for to expect and authorize off-chain storage operations.
-
- -
-
Eventually a user holds an off-chain availability certificate from storage nodes for a blob -ID. The user uploads the certificate on chain to signal that the blob ID is available for an -availability period. The certificate is checked against the latest Walrus committee, -and an availability event is emitted for the blob ID if correct. This is the point of -availability for the blob.
-
- -
-
At a later time, a certified blob's storage can be extended by adding a storage object to it -with a longer expiry period. This facility can be used by smart contracts to extend the -availability of blobs stored in perpetuity as long as funds exist to continue providing storage.
-
- -
-
In case a blob ID is not correctly encoded, an inconsistency proof certificate can be uploaded -on chain at a later time. This action emits an inconsistent blob event, signaling that the -blob ID read results always return
-None. This indicates that its slivers can be deleted by -storage nodes, except for an indicator to returnNone.
-
Users writing to Walrus, need to perform Sui transactions to acquire storage and certify blobs. -Users creating or consuming proofs for attestations of blob availability read the chain -only to prove or verify emission of events. Nodes read -the blockchain to get committee metadata only once per epoch, and then request slivers directly -from storage nodes by blob ID to perform reads on Walrus resources.
-Governance operations on Sui
-Each Walrus storage epoch is represented by the Walrus system object that contains a storage -committee and various metadata or storage nodes, like the mapping between shards and storage nodes, -available space and current costs.
-Users can go to the system object for the period and buy some -storage amount for one or more storage epochs. At each storage epoch there is a price for storage, -and the payment provided becomes part of a storage fund for all the storage epochs that span -the storage bought. There is a maximum number of storage epochs in the future for which storage can -be bought (~2 years). Storage is a resource that can be split, merged, and transferred.
-At the end of the storage epoch, part of the funds in the storage fund need to be allocated to -storage nodes. The idea here is for storage nodes to perform light audits of each other, -and suggest which nodes are to be paid based on the performance of these audits.
- -
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/overview/overview.html b/pr-preview/pr-70/overview/overview.html
deleted file mode 100644
index 9b40d824..00000000
--- a/pr-preview/pr-70/overview/overview.html
+++ /dev/null
@@ -1,231 +0,0 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- Operations
-Walrus operations can be separated in interactions with the Sui chain, which -is used by Walrus for coordination and governance, and off-chain -interactions between clients and storage nodes.
- -
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/overview/properties.html b/pr-preview/pr-70/overview/properties.html
deleted file mode 100644
index edbce766..00000000
--- a/pr-preview/pr-70/overview/properties.html
+++ /dev/null
@@ -1,260 +0,0 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- System overview
-This chapter provides an overview of the security properties, -architecture and encoding mechanisms of the Walrus system.
-Use the Glossary as a reference for bolded terms used in this documentation.
- -
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/print.html b/pr-preview/pr-70/print.html
deleted file mode 100644
index b07dd3c7..00000000
--- a/pr-preview/pr-70/print.html
+++ /dev/null
@@ -1,2070 +0,0 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- Walrus assurance and security properties
-The properties below hold true subject to the assumption that for all storage epochs 2/3 of shards -are operated by storage nodes that faithfully and correctly follow the Walrus protocol.
-Each blob is encoded using error correction into slivers and a blob ID is cryptographically -derived. For a given blob ID there is a point of availability (PoA) and an availability -period, observable through an event on the Sui chain.
-The following properties relate to the PoA:
--
-
- After the PoA, for a blob ID, any correct user that performs a read within the availability -period will eventually terminate and get a value V which is either the blob contents F or None. -
- After PoA if two correct users perform a read and get V and V’ then V = V’. -
- A correct user with an appropriate storage resource can always perform store for a blob F with a -blob ID and advance the protocol until the PoA. -
- A read after PoA for a blob F stored by a correct user, will result in F. -
Some assurance properties ensure the correct internal processes of Walrus storage nodes. -For the purposes of defining these, an inconsistency proof proves that a blob ID was -stored by a user that incorrectly encoded a blob.
--
-
- After PoA and for a blob ID stored by a correct user, a storage node is always able to recover -the correct sliver for its shards for this blob ID. -
- After PoA if a correct storage node cannot recover a sliver, it can produce an inconsistency proof -for the blob ID. -
- If a blob ID is stored by a correct user, an inconsistently proof cannot be derived for it. -
- A read by a correct user for a blob ID for which an inconsistency proof may exist returns None. -
Note that there is no delete operation and a blob ID past PoA will be available for the full -availability period.
-As a rule of thumb: before PoA it is the responsibility of a client to ensure the availability of -a blob and its upload to Walrus. After PoA it is the responsibility of Walrus as a system to -maintain the availability of the blob as part of its operation for the full availability period -remaining. Emission of the event corresponding to the PoA for a blob ID attests its -availability.
- -
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/searcher.js b/pr-preview/pr-70/searcher.js
deleted file mode 100644
index dc03e0a0..00000000
--- a/pr-preview/pr-70/searcher.js
+++ /dev/null
@@ -1,483 +0,0 @@
-"use strict";
-window.search = window.search || {};
-(function search(search) {
- // Search functionality
- //
- // You can use !hasFocus() to prevent keyhandling in your key
- // event handlers while the user is typing their search.
-
- if (!Mark || !elasticlunr) {
- return;
- }
-
- //IE 11 Compatibility from https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/startsWith
- if (!String.prototype.startsWith) {
- String.prototype.startsWith = function(search, pos) {
- return this.substr(!pos || pos < 0 ? 0 : +pos, search.length) === search;
- };
- }
-
- var search_wrap = document.getElementById('search-wrapper'),
- searchbar = document.getElementById('searchbar'),
- searchbar_outer = document.getElementById('searchbar-outer'),
- searchresults = document.getElementById('searchresults'),
- searchresults_outer = document.getElementById('searchresults-outer'),
- searchresults_header = document.getElementById('searchresults-header'),
- searchicon = document.getElementById('search-toggle'),
- content = document.getElementById('content'),
-
- searchindex = null,
- doc_urls = [],
- results_options = {
- teaser_word_count: 30,
- limit_results: 30,
- },
- search_options = {
- bool: "AND",
- expand: true,
- fields: {
- title: {boost: 1},
- body: {boost: 1},
- breadcrumbs: {boost: 0}
- }
- },
- mark_exclude = [],
- marker = new Mark(content),
- current_searchterm = "",
- URL_SEARCH_PARAM = 'search',
- URL_MARK_PARAM = 'highlight',
- teaser_count = 0,
-
- SEARCH_HOTKEY_KEYCODE = 83,
- ESCAPE_KEYCODE = 27,
- DOWN_KEYCODE = 40,
- UP_KEYCODE = 38,
- SELECT_KEYCODE = 13;
-
- function hasFocus() {
- return searchbar === document.activeElement;
- }
-
- function removeChildren(elem) {
- while (elem.firstChild) {
- elem.removeChild(elem.firstChild);
- }
- }
-
- // Helper to parse a url into its building blocks.
- function parseURL(url) {
- var a = document.createElement('a');
- a.href = url;
- return {
- source: url,
- protocol: a.protocol.replace(':',''),
- host: a.hostname,
- port: a.port,
- params: (function(){
- var ret = {};
- var seg = a.search.replace(/^\?/,'').split('&');
- var len = seg.length, i = 0, s;
- for (;i
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-Clone the repository and build the
-
-
-
-
-
- Walrus
-Welcome to the developer documentation for Walrus, a decentralized storage and data availability -protocol designed specifically for large binary files, or "blobs". Walrus focuses on providing a -robust, but affordable solution for storing unstructured content on decentralized storage nodes -while ensuring high availability and reliability even in the presence of Byzantine faults.
-Fun fact: If you are viewing this site at https://docs.walrus.site, you are fetching this from -Walrus behind the scenes. See the Walrus Sites chapter for further -details on how this works.
-Features
--
-
-
-
Storage and retrieval: Walrus supports storage operations to write and read blobs. It also -allows anyone to prove that a blob has been stored and is available for retrieval at a later -time.
-
- -
-
Cost efficiency: By utilizing advanced error correction coding, Walrus maintains storage -costs at approximately five times the size of the stored blobs and encoded parts of each blob -are stored on each storage node. This is significantly more cost-effective compared to -traditional full replication methods and much more robust against failures compared to -protocols that only store each blob on a subset of storage nodes.
-
- -
-
Integration with Sui blockchain: Walrus leverages Sui -for coordination, attesting availability and payments. Storage space can be owned as a resource on -Sui, split, merged, and transferred. Blob storage is represented using storage objects on Sui, and -smart contracts can check whether a blob is available and for how long.
-
- -
-
Flexible access: Users can interact with Walrus through a command-line interface (CLI), -software development kits (SDKs), and web2 HTTP technologies. Walrus is designed to work well -with traditional caches and content distribution networks (CDNs), while ensuring all operations -can also be run using local tools to maximize decentralization.
-
-
Architecture and operations
-Walrus's architecture ensures that content remains accessible and retrievable even when many -storage nodes are unavailable or malicious. Under the hood it uses modern error correction -techniques based on fast linear fountain codes, augmented to ensure resilience against Byzantine -faults, and a dynamically changing set of storage nodes. The core of Walrus remains simple, and -storage node management and blob certification leverages Sui smart contracts.
-Organization
-This documentation is split into three parts:
--
-
- About Walrus describes the objectives, security properties, and architecture of Walrus. -
- Usage provides concrete information for developers. If you want to get started quickly, you can -jump directly to the setup chapter. -
- Walrus sites describes how you can use Walrus and Sui together to build truly decentralized -websites. -
Finally, we provide a glossary that explains the terminology used throughout the -documentation.
-Sources
-This documentation is built using mdBook from source files in -github.com/MystenLabs/walrus-docs/. Please report or -fix any errors you find in this documentation in that GitHub project.
-The Walrus Dev Blog
-We will use this section of the Walrus documentation to publish news and updates about Walrus' -development!
-Announcing Walrus & Walrus Sites: A Decentralized Store and Web DApp Platform
-Walrus is an innovative decentralized storage network for blockchain apps and autonomous agents. The -Walrus storage system is released today as a developer preview to Sui developers to gather -feedback. We expect a general rollout to other web3 communities very soon!
-Leveraging innovations in erasure coding, Walrus enables fast and robust encoding of unstructured -data blobs into smaller slivers distributed and stored over a network of storage nodes. A subset of -slivers can be used to rapidly reconstruct the original blob reliably, even when up to two thirds of -the slivers are missing. This is possible while keeping the replication factor down to a minimal -4x-5x, similar to existing cloud-based services, but with the additional benefits of -decentralization and resilience to more widespread faults.
-The Replication Challenge
-Sui is the most advanced blockchain system in relation to storage on validators, with innovations -such as a storage fund that future proofs -the cost of storing on-chain data. Nevertheless, Sui still requires complete data replication among -all validators, resulting in a replication factor of 100x or more in today's Sui mainnet. This is -necessary for replicated computing and smart contracts acting on the state of the blockchain, but is -inefficient when it comes to simply storing unstructured data blobs, such as music, video, -blockchain history, etc.
-Introducing Walrus: Efficient and Robust Decentralized Storage
-To tackle the challenge of high replication costs, Mysten Labs has developed Walrus, a decentralized -storage network offering exceptional data availability and robustness with a minimal replication -factor of 4x-5x. Walrus provides two key benefits:
--
-
-
-
Cost-Effective Blob Storage: Walrus allows for the upload of gigabytes of data at a time with -minimal cost, making it an ideal solution for storing large volumes of data.\ Walrus can do this -because the data blob is transmitted only once over the network, and storage nodes only spend a -fraction of resources compared to the blob size. As a result, the more storage nodes the system -has, the fewer resources each storage node uses per blob.
-
- -
-
High Availability and Robustness: Data stored on Walrus enjoys enhanced reliability and -availability under fault conditions. Data recovery is still possible even if two-thirds of the -storage nodes crash or come under adversarial control. Further, availability may be certified -efficiently without downloading the full blob.
-
-
Decentralized storage can take multiple forms in modern ecosystems. For instance, it offers better -guarantees for digital assets traded as NFTs. Unlike current designs that store data off-chain, -decentralized storage ensures users own the actual resource, not just metadata, mitigating risks of -data being taken down or misrepresented.
-Additionally, decentralized storage is not only useful for storing data such as pictures or files -with high availability; it can also double as a low-cost data availability layer for rollups. Here, -sequencers can upload transactions on Walrus, and the rollup executor only needs to temporarily -reconstruct them for execution.
-We also believe Walrus will accompany existing disaster recovery strategies for millions of -enterprise companies. Not only is Walrus low-cost, but it also provides unmatched layers of data -availability, integrity, transparency, and resilience that centralized solutions by design cannot -offer.
-Walrus is powered by the Sui Network and scales horizontally to hundreds of networked decentralized -storage nodes. This should enable Walrus to offer Exabytes of storage at costs competitive with -current centralized offerings, given the higher assurance and decentralization.
-The Future of Walrus
-By releasing this developer preview we hope to share some of the design decisions with the -decentralized app developer community, and gather feedback on the approach and the APIs for storing, -retrieving, and certifying blobs. In this developer preview, all storage nodes are operated by -Mysten Labs to help us understand use-cases, fix bugs, and improve the performance of the software.
-Future updates to Walrus will allow for dynamically changing the set of decentralized storage nodes, -as well as changing the mapping of what slivers are managed by each storage node. The available -operations and tools will also be expanded to cover more storage related use-cases. Many of these -functions will be designed with the feedback we gather in mind.
-Stay tuned for more updates on how Walrus will revolutionize data storage in the web3 ecosystem.
-What can developers build?
-As part of this developer preview we provide a binary client (currently macos, ubuntu) that can be -operated from the command line interface, a JSON -API, and an HTTP -API. We also offer the community an aggregator and -publisher service, and a Devnet deployment of 10 storage nodes operated by Mysten Labs.
-We hope Sui developers experiment with building applications that leverage the Walrus Decentralized -Store in a variety of ways. As examples we hope to see the community build:
--
-
-
-
Storage of media for NFT or dapps: Walrus can directly store and serve media such as images, -sounds, sprites, videos, other game assets, etc. This is publicly available media that can be -accessed using HTTP requests at caches to create multimedia dapps.
-
- -
-
AI related use cases: Walrus can store clean data sets of training data, datasets with a known -and verified provenance, models weights, and proofs of correct training for AI models. Or it may -be used to store and ensure the availability and authenticity of an AI model output.
-
- -
-
Storage of long term archival of blockchain history: Walrus can be used as a lower cost -decentralized store to store blockchain history. For Sui this can include sequences of checkpoints -with all associated transaction and effects content. As well as historic snapshots of the -blockchain state, code or binaries.
-
- -
-
Support availability for L2s: Walrus allows parties to certify the availability of blobs, as -required by L2s that need data to be stored and be attested as available to all. This may also -include availability of extra audit data such as validity proofs, zero knowledge proofs of correct -execution or large fraud proofs.
-
- -
-
Support a full decentralized web experience: Walrus can host full decentralized web -experiences including all resources (such as js, css, html, media). These can provide content but -also host the UX of dapps, enabling fully decentralized front end and back ends on chain. It -brings the full "web" back in "web3".
-
- -
-
Support subscription models for media: Creators can store encrypted media on Walrus and only -provide access via decryption keys to parties that have paid a subscription fee or have paid for -contents. (Note that Walrus provides the storage, encryption and decryption must be done off -Walrus).
-
-
We are excited to see what else the web3 developer community can imagine!
-Getting Started
-For this developer preview the public Walrus Devnet is openly available to all developers. Developer -documentation is available at https://docs.walrus.site.
-SUI Testnet token is the main currency for interacting with Walrus. Developers pay for Walrus -Devnet storage using SUI Testnet tokens which can be acquired at the Sui Testnet Discord -faucet.
-One more thing
-The Walrus Sites website, the Walrus docs, and -this very blog are hosted on Walrus. To learn more about Walrus Sites -and how you can deploy your own, click -here.
-Objectives and use cases
-Walrus supports operations to store and retrieve blobs, and to prove and verify their availability. -It ensures content survives storage nodes suffering Byzantine faults and remains available and -retrievable. It provides APIs to access the stored content over a CLI, SDKs and over web2 HTTP -technologies, and supports content delivery infrastructures like caches and content distribution -networks (CDNs).
-Under the hood, storage cost is a small fixed multiple of the size of blobs (around 5x). -Advanced error correction coding keeps the cost low, in contrast to the full replication of data -traditional to blockchains, such as the >100x multiple for data stored in Sui objects. As a result, -storage of much bigger resources (up to several GiB) is possible on Walrus at substantially lower -cost compared to Sui or other blockchains. Because encoded blobs are stored on all storage nodes, -Walrus also provides superior robustness compared to designs with a small amount of replicas -storing the full blob.
-Walrus uses the Sui chain for coordination and payments. Available storage is represented as Sui -objects that can be acquired, owned, split, merged and transferred. Storage space can be tied to -a stored blob for a period of time, with the resulting Sui object used to prove -availability on chain in smart contracts, or off chain using light clients.
-The next chapter discusses the above operations relating to storage, -retrieval and availability in detail.
-In the future, we plan to include in Walrus some minimal governance to allow storage nodes to -change between storage epochs. Walrus is also compatible with periodic payments for continued -storage. We also plan to implement storage attestation based on challenges to get confidence -that blobs are stored or at least available. Walrus also allows light nodes that store small parts -of blobs to get rewards for proving availability and assisting recovery. We will cover these -topics in later documents. We also provide details of the encoding scheme in a separate document.
-Non-objectives
-There are a few things that Walrus explicitly is not:
--
-
-
-
Walrus does not reimplement a CDN that might be geo-replicated or have less than tens of -milliseconds of latency. Instead, it ensures that traditional CDNs are usable and compatible with -Walrus caches.
-
- -
-
Walrus does not re-implement a full smart contracts platform with consensus or execution. It -relies on Sui smart contracts when necessary, to manage Walrus resources and processes including -payments, storage epochs, and so on.
-
- -
-
Walrus supports storage of any blob, including encrypted blobs, however Walrus itself is not the -distributed key management infrastructure that manages and distributed encryption or decryption -keys to support a full private storage eco-system. It can, however, provide the storage layer for -such infrastructures.
-
-
Use cases
-App builders may use Walrus in conjunction with any L1 or L2 blockchains to build experiences that -require large amounts of data to be stored in a decentralized manner and possibly certified as -available:
--
-
-
-
Storage of media for NFT or dApps: Walrus can directly store and serve media such as images, -sounds, sprites, videos, other game assets, and so on. This is publicly available media that is -accessed using HTTP requests at caches to create multimedia dApps.
-
- -
-
AI related use cases: Walrus can store clean data sets of training data, datasets with a -known and verified provenance, models, weights and proofs of correct training for AI models. -It can also store and ensure the availability of an AI model output.
-
- -
-
Storage of long term archival of blockchain history: Walrus can act as a lower cost -decentralized store to store blockchain history. For Sui, this can include sequences of -checkpoints with all associated transaction and effects content, as well as historic snapshots -of the blockchain state, code or binaries.
-
- -
-
Support availability for L2s: Walrus allows parties to certify the availability of blobs, as -required by L2s that need data to be stored and be attested as available to all. This may also -include availability of extra audit data such as validity proofs, zero knowledge proofs of -correct execution or large fraud proofs.
-
- -
-
Support a fully decentralized web experience: Walrus can host fully decentralized web -experiences, including all resources (such as js, css, html, media). These can not only -provide content, but also host the UX of dApps to enable applications with fully decentralized -front end and back ends on chain. Walrus puts the full "web" into web3.
-
- -
-
Support subscription models for media: Creators can store encrypted media on Walrus and only -provide access via decryption keys to parties that have paid a subscription fee or have paid for -content. Walrus provides the storage, encryption and decryption needs to happen off -the system.
-
-
System overview
-This chapter provides an overview of the security properties, -architecture and encoding mechanisms of the Walrus system.
-Use the Glossary as a reference for bolded terms used in this documentation.
-Basic architecture and security assumptions
-The key actors in the Walrus architecture are the following:
--
-
-
-
Users through clients want to store and read blobs identified by their blob ID.
-These actors are ready to pay for service -when it comes to writes and non-best-effort reads. Users also want to prove -the availability of a blob to third parties without the cost of sending or receiving the full -blob.
-Users might be malicious in various ways: they might not want to pay for services, prove the -availability of unavailable blobs, modify/delete blobs without authorization, try to -exhaust resources of storage nodes, and so on.
-
- -
-
Storage nodes hold one or many shards within a storage epoch.
-Each blob is erasure -encoded in many slivers. Slivers from each stored blob become part of all shards. A shard -at any storage epoch is associated with a storage node that actually stores all slivers of -the shard and is ready to serve them.
-A Sui smart contract controls the assignment of storage nodes to shards within -storage epochs and Walrus assumes that more than 2/3 of the -shards are managed by correct storage nodes within each storage epoch. This means that Walrus must -tolerate up to 1/3 of the shards managed by Byzantine storage nodes (approximately 1/3 of the -storage nodes being Byzantine) within each storage epoch and across storage epochs.
-
- -
-
All clients and storage nodes operate a blockchain client (specifically on Sui), and mediate -payments, resources (space), mapping of shards to storage nodes and metadata through blockchain -smart contracts. Users interact with the blockchain to acquire storage resources and upload -certificates for stored blobs. Storage nodes listen to the blockchain events to coordinate -their operations.
-
-
Walrus supports any additional number of optional infrastructure actors that can operate in a -permissionless way:
--
-
-
-
Caches are clients that store one or more full blobs and make them available to users -over traditional web2 technologies (such as HTTP).
-They are optional in that end users can also -operate a local cache, and perform Walrus reads over web2 technologies locally. However, cache -infrastructures can also act as CDNs, split the cost of blob reconstruction over many requests, -be better connected, and so on. A client can always verify that reads from such infrastructures -are correct.
-
- -
-
Publishers are clients that help end users store a blob using web2 technologies, -using less bandwidth and custom logic.
-In effect, they receive the blob to be published over -traditional web2 protocols (like HTTP) and run the Walrus store protocol on the end user's -behalf. This includes encoding the blob into slivers, distributing the slivers to shards, -collecting storage-node signatures and aggregating them into a certificate, as well as all -other on-chain actions.
-They are optional in that a user can directly interact with Sui and -the storage nodes to store blobs. An end user can always verify that a publisher -performed their duties correctly by checking that an event associated with the -Point of Availability for the blob exists on chain -and then either performing a read to see if Walrus returns the blob or encoding the blob -and comparing the result to the blob ID in the certificate.
-
-
Caches, publishers and end users are not considered trusted components of the system, and they might -deviate from the protocol arbitrarily. However, some of the security properties of Walrus only hold -for honest end users that use honest intermediaries (caches and publishers). Walrus provides a -means for end users to audit the correct operation of both caches and publishers.
-Encoding, overheads and verification
-The following list summarizes the basic encoding and cryptographic techniques used in Walrus:
--
-
-
-
Storage nodes hold one or many shards in a storage epoch out of a larger total (1000, for -instance). Each shard contains one blob sliver for each blob past point of availability. Each -shard is assigned to a storage node in a storage epoch.
-
- -
-
An erasure code encode algorithm takes a blob -and encodes it as K symbols, such that any fraction p of symbols can be used to reconstruct -the blob. Each blob sliver contains a fixed number of such symbols.
-
- -
-
Walrus selects p < 1/3 so that a third of symbols and slivers can be used to reconstruct the blob -by the decode algorithm. The matrix used to produce the erasure code is fixed and is the same -for all blobs in the Walrus system, and encoders have no discretion about it.
-
- -
-
Storage nodes manage one or more shards, and corresponding slivers of each blob are distributed -to all the storage shards.
-As a result, the overhead of the distributed store is ~5x that of the blob itself, no matter how -many shards there are. The encoding is systematic, meaning that some storage nodes hold part of -the original blob, allowing for fast random access reads.
-
-
Each blob is also associated with some metadata including a blob ID to allow verification:
--
-
-
-
A blob ID is computed as an authenticator of the set of all shard data and metadata (byte -size, encoding, blob hash).
-Walrus hashes a sliver representation in each of the shards and adds the resulting -hashes into a Merkle tree. Then the root of the Merkle tree is the blob hash used to derive the -blob ID that identifies the blob in the system.
-
- -
-
Each storage node can use the blob ID to check if some shard data belongs to a blob using the -authenticated structure corresponding to the blob hash (Merkle tree). A successful check means -that the data is indeed as intended by the writer of the blob (who might be corrupt).
-
- -
-
When any party reconstructs a blob ID from shard slivers, or accepts any blob claiming -to be a specific blob ID, it must check that it encodes to the correct blob ID.
-This process involves re-coding the blob using the erasure correction code, and deriving the -blob ID again to check that the blob matches. This prevents a malformed blob (incorrectly -erasure coded) from ever being read as a valid blob at any correct recipient.
-
- -
-
A set of slivers equal to the reconstruction threshold belonging to a blob ID that are either -inconsistent or lead to the reconstruction of a different ID represent an incorrect encoding -(this happens only if the user that encoded the blob was malicious and encoded it incorrectly).
-Walrus can extract one symbol per sliver to form an inconsistency proof. -Storage nodes can delete slivers belonging to inconsistently encoded blobs, -and upon request return either the inconsistency proof or an inconsistency certificate posted -on chain.
-
-
Operations
-Walrus operations can be separated in interactions with the Sui chain, which -is used by Walrus for coordination and governance, and off-chain -interactions between clients and storage nodes.
-Operations on Sui
-Walrus uses Sui smart contracts to coordinate storage operations as resources that have a lifetime, -and payments. Smart contracts also facilitate governance to determine the storage nodes holding each -storage shard. The following content outlines these operations and refers to them as part of the -read/write paths.
-Metadata is the only blob element ever exposed to Sui or its validators, as the content -of blobs is always stored off chain on Walrus storage nodes and caches. The storage nodes or caches -do not have to overlap with any Sui infrastructure components (such as validators), and the storage -epochs can be of different lengths and not have the same start/end times as Sui epochs.
-Storage resource life cycle on Sui
-A number of Sui smart contracts hold the metadata of the Walrus system and all its entities.
--
-
-
-
A Walrus system object holds the committee of storage nodes for the current storage epoch. The -system object also holds the total available space on Walrus and the price per unit of storage (1 -KiB).
-These values are determined by 2/3 agreement between the storage nodes for the storage -epoch. Users can pay to purchase storage space for some time duration. These space resources can -be split, merged and transferred. Later, they can be used to place a blob ID into Walrus.
-
- -
-
The storage fund holds funds for storing blobs over one storage epoch, multiple or -perpetually. When purchasing storage space from the system object, users pay into the storage fund -separated over multiple storage epochs. Payments are made each epoch to storage nodes -according to performance (details follow).
-
- -
-
A user acquires some storage through the contracts or transfer, and can assign to it a blob ID, -signifying they want to store this blob ID into it. This emits a Move resource event that -both caches and storage nodes listen for to expect and authorize off-chain storage operations.
-
- -
-
Eventually a user holds an off-chain availability certificate from storage nodes for a blob -ID. The user uploads the certificate on chain to signal that the blob ID is available for an -availability period. The certificate is checked against the latest Walrus committee, -and an availability event is emitted for the blob ID if correct. This is the point of -availability for the blob.
-
- -
-
At a later time, a certified blob's storage can be extended by adding a storage object to it -with a longer expiry period. This facility can be used by smart contracts to extend the -availability of blobs stored in perpetuity as long as funds exist to continue providing storage.
-
- -
-
In case a blob ID is not correctly encoded, an inconsistency proof certificate can be uploaded -on chain at a later time. This action emits an inconsistent blob event, signaling that the -blob ID read results always return
-None. This indicates that its slivers can be deleted by -storage nodes, except for an indicator to returnNone.
-
Users writing to Walrus, need to perform Sui transactions to acquire storage and certify blobs. -Users creating or consuming proofs for attestations of blob availability read the chain -only to prove or verify emission of events. Nodes read -the blockchain to get committee metadata only once per epoch, and then request slivers directly -from storage nodes by blob ID to perform reads on Walrus resources.
-Governance operations on Sui
-Each Walrus storage epoch is represented by the Walrus system object that contains a storage -committee and various metadata or storage nodes, like the mapping between shards and storage nodes, -available space and current costs.
-Users can go to the system object for the period and buy some -storage amount for one or more storage epochs. At each storage epoch there is a price for storage, -and the payment provided becomes part of a storage fund for all the storage epochs that span -the storage bought. There is a maximum number of storage epochs in the future for which storage can -be bought (~2 years). Storage is a resource that can be split, merged, and transferred.
-At the end of the storage epoch, part of the funds in the storage fund need to be allocated to -storage nodes. The idea here is for storage nodes to perform light audits of each other, -and suggest which nodes are to be paid based on the performance of these audits.
-Off-chain operations
-While Walrus operations happen off Sui, they might interact with the blockchain flows defining the -resource life cycle.
-Write paths
-
Systems overview of writes, illustrated in the previous image:
--
-
-
-
A user acquires a storage resource of appropriate size and duration on chain, either by directly -buying it on the Walrus system object, or a secondary market. A user can split, merge and -transfer owned storage resources.
-
- -
-
When users want to write a blob, they first erasure code it using encode and compute the -blob ID. Then they can perform the following steps themselves, or use a publisher to perform steps -on their behalf.
-
- -
-
The user goes on chain (Sui) and updates a storage resource to register the blob ID with the -desired size and lifetime. This emits an event, received by storage nodes. After the -user receives they then continue the upload.
-
- -
-
The user sends each of the blob slivers and metadata to the storage nodes that currently -manages the corresponding shards.
-
- -
-
A storage node managing a shard receives a sliver and checks it against the blob ID. -It also checks that there is a blob resource with the blob ID that is authorized to store -a blob. If correct, the storage node then signs a statement that it holds the sliver for blob ID -(and metadata) and returns it to the user.
-
- -
-
The user puts together the signatures returned from storage nodes into an availability certificate -and sends it on chain. When the certificate is verified on chain, an availability event for the -blob ID is emitted, and all other storage nodes seek to download any missing shards for the blob -ID. This event emitted by Sui is the point of availability (PoA) for the blob -ID.
-
- -
-
After the PoA, and without user involvement, storage nodes sync and recover any missing slivers.
-
-
The user waits for 2/3 of shard signatures to return to create the certificate of -availability. The rate of the code is below 1/3, allowing for reconstruction even if only 1/3 of -shards return the sliver for a read. Because at most 1/3 of the storage nodes can fail, this ensures -reconstruction if a reader requests slivers from all storage nodes. The full process can -be mediated by a publisher that receives a blob and drives the process to completion.
-Refresh availability
-Because no content data is required to refresh the duration of storage, refresh is conducted fully -on chain within the protocol. To request an extension to the availability of a blob, a user provides -an appropriate storage resource. Upon success this emits an event that storage nodes receive to -extend the time for which each sliver is stored.
-Inconsistent resource flow
-When a correct storage node tries to reconstruct a shard it might fail if the encoding of a blob ID -past PoA was incorrect, but will instead extract an inconsistency proof for the -blob ID. It then uses the proof to create an inconsistency certificate and upload it on chain. -The flow is as follows:
--
-
-
-
A storage node fails to reconstruct a shard, and instead holds an inconsistency proof.
-
- -
-
The storage node sends the blob ID and inconsistency proof to all storage nodes of the Walrus -epoch. The storage node verifies the proof and signs it.
-
- -
-
The storage node aggregates the signatures into an inconsistency certificate and sends it to the -Walrus smart contract, which verifies it and emits a inconsistent resource event.
-
- -
-
Upon receiving an inconsistent resource event, correct storage nodes delete sliver data for the -blob ID and record in the metadata to return
-Nonefor the blob ID for the -availability period. No storage attestation challenges are issued for this -blob ID.
-
A blob ID that is inconsistent always resolves to None upon reading: this is because
-the read process re-encodes the received blob to check that the blob ID is correctly derived from a
-consistent encoding. This means that an inconsistency proof reveals only a true fact to storage
-nodes (that do not otherwise run decoding), and does not change the output of read in any case.
Note, however, that partial reads leveraging the systematic nature of the encoding might return -partial reads for inconsistently encoded files. Thus, if consistency and availability of reads is -important, dApps should do full reads rather than partial reads.
-Read paths
-A user can read stored blobs either directly or through a cache. The direct user journey is -discussed here because this is also how the cache operates in case of a cache miss. Assume that most -reads happen through caches for blobs that are hot, and do not result in requests to storage nodes.
--
-
-
-
The reader gets the metadata for the blob ID from any storage node, and authenticates it using -the blob ID.
-
- -
-
The reader then sends a request to the storage node for the shards corresponding to the blob ID, -and waits for f+1 to respond. Sufficient requests are sent in parallel to ensure low latency for -reads.
-
- -
-
The reader authenticates the slivers returned with the blob ID, reconstructs the blob, and decides -whether the contents are a valid blob or inconsistent.
-
- -
-
Optionally, for a cache, the result is cached and can be served without reconstruction until it is -evicted from the cache. Requests for the blob to the cache return the blob contents, or a proof -the blob is inconsistently encoded.
-
-
Challenge mechanism for storage attestation
-During an epoch, a correct storage node challenges all shards to provide blob slivers past PoA:
--
-
-
-
The list of available blobs for the epoch is determined by the sequence of Sui events up -to the past epoch. Inconsistent blobs are not challenged, and a record proving this status -can be returned instead.
-
- -
-
A challenge sequence is determined by providing a seed to the challenged shard. The sequence is -then computed based both on the seed AND the content of each challenged blob ID. This creates a -sequential read dependency.
-
- -
-
The response to the challenge provides the sequence of shard contents for the blob IDs in a -timely manner.
-
- -
-
The challenger node uses thresholds to determine whether the challenge was passed, and reports -the result on chain.
-
- -
-
The challenge/response communication is authenticated.
-
-
Challenges provide some reassurance that the storage node can actually recover shard data in a -probabilistic manner, avoiding storage nodes getting payment without any evidence they might -retrieve shard data. The sequential nature of the challenge and some reasonable timeout also ensures -that the process is timely.
-Walrus assurance and security properties
-The properties below hold true subject to the assumption that for all storage epochs 2/3 of shards -are operated by storage nodes that faithfully and correctly follow the Walrus protocol.
-Each blob is encoded using error correction into slivers and a blob ID is cryptographically -derived. For a given blob ID there is a point of availability (PoA) and an availability -period, observable through an event on the Sui chain.
-The following properties relate to the PoA:
--
-
- After the PoA, for a blob ID, any correct user that performs a read within the availability -period will eventually terminate and get a value V which is either the blob contents F or None. -
- After PoA if two correct users perform a read and get V and V’ then V = V’. -
- A correct user with an appropriate storage resource can always perform store for a blob F with a -blob ID and advance the protocol until the PoA. -
- A read after PoA for a blob F stored by a correct user, will result in F. -
Some assurance properties ensure the correct internal processes of Walrus storage nodes. -For the purposes of defining these, an inconsistency proof proves that a blob ID was -stored by a user that incorrectly encoded a blob.
--
-
- After PoA and for a blob ID stored by a correct user, a storage node is always able to recover -the correct sliver for its shards for this blob ID. -
- After PoA if a correct storage node cannot recover a sliver, it can produce an inconsistency proof -for the blob ID. -
- If a blob ID is stored by a correct user, an inconsistently proof cannot be derived for it. -
- A read by a correct user for a blob ID for which an inconsistency proof may exist returns None. -
Note that there is no delete operation and a blob ID past PoA will be available for the full -availability period.
-As a rule of thumb: before PoA it is the responsibility of a client to ensure the availability of -a blob and its upload to Walrus. After PoA it is the responsibility of Walrus as a system to -maintain the availability of the blob as part of its operation for the full availability period -remaining. Emission of the event corresponding to the PoA for a blob ID attests its -availability.
-Future discussion
-In this document, we left out details of the following features:
--
-
- Shard transfer and recovery upon storage epoch change. The encoding scheme used has been designed -to allow this operation to be efficient. A storage node needs to only get data of the same -magnitude to the missing sliver data to reconstruct them. -
- Details of light clients that can be used to sample availability. Individual clients may sample -the certified blobs from Sui metadata, and sample the availability of some slivers that they -store. On-chain bounties may be used to retrieve these slivers for missing blobs. -
Developer Guide
-This guide introduces all the concepts needed to build applications that use Walrus as a storage -or availability layer. The overview provides more background and explains -in more detail how Walrus operates internally.
-This developer guide describes:
--
-
- Components of Walrus of interest to developers that wish to use it for -storage or availability. -
- Operations supported through client binaries, APIs, or Sui operations. -
- The Sui structures Walrus uses to store metadata, and how they can be read -from Sui smart contracts, or through the Sui SDK. -
A glossary of terms used is also available for reference.
-Disclaimer about the Walrus developer preview
-This release of Walrus & Walrus Sites is a -developer preview, to showcase the technology and solicit feedback from builders. All storage nodes -and aggregators are operated by Mysten Labs, all transactions are executed on the Sui testnet, -and use testnet SUI which has no value. The state of the store can be, and will be wiped, at any -point and possibly with no warning. Do not rely on this developer preview for any production -purposes, it comes with no availability or persistence guarantees.
-Also see the Devnet terms of service under which this developer preview is made -available.
-Components
-From a developer perspective, some Walrus components are objects and smart contracts on -Sui, and some components are Walrus-specific binaries and services. As a rule Sui is used to -manage blob and storage node metadata, while Walrus-specific services are used to store and -read blob contents, which can be very large.
-Walrus defines a number of objects and smart contracts on Sui:
--
-
- A shared system object, records and manages the current committee of storage nodes. -
- Storage resources, represent empty storage space that may be used to store blobs. -
- Blob resources, represent blobs being registered and certified as stored. -
- Changes to these objects emit Walrus-related events. -
The Walrus system object ID can be found in the Walrus client_config.yaml file (see
-Configuration). You may use any Sui explorer to look at its
-content, as well as explore the content of blob objects. There is more information about these in
-the quick reference to the Walrus Sui structures.
Walrus is also composed of a number of Walrus-specific services and binaries:
--
-
- A client (binary) can be executed locally and provides a -Command Line Interface (CLI), a JSON API -and an HTTP API to perform Walrus operations. -
- Aggregators services allow reading blobs via HTTP requests. -
- Publishers services are used store blobs to Walrus. -
- A set of storage nodes store encoded blobs. These nodes form the decentralized -storage infrastructure of Walrus. -
Aggregators, publishers and other services use the client APIs to interact with Walrus. End-users -of services using Walrus interact with the store via custom services, aggregators or publishers that -expose HTTP APIs to avoid the need to run locally a binary client.
-Operations
-Blob encoding and blob ID
-Walrus stores blobs across storage nodes in an encoded form, and refers -to blobs by their blob ID. The blob ID is deterministically derived from the content of a blob -and the Walrus configuration. The blob ID of two files with the same content will be the same.
-You can derive the blob ID of a file locally using the command: walrus blob-id <file path>
Store
-Walrus may be used to store a blob, via the native client APIs or a publisher.
---All blobs -stored in Walrus are public and discoverable by all. Therefore you must not use Walrus to store -anything that contains secrets or private data without additional measures to protect -confidentiality.
-
Under the hood a number of operations happen both on Sui as well as on storage nodes:
--
-
- The client or publisher encodes the blob and derives a blob ID that identifies the blob. This
-is a
u256often encoded as a URL safe base64 string.
- - A transaction is executed on Sui to purchase some storage from the system object, and then to -register the blob ID occupying this storage. Client APIs return the Sui blob object ID. The -transactions use SUI to purchase storage and pay for gas. -
- Encoded slivers of the blob are distributed to all storage nodes. They each sign a receipt. -
- Signed receipts are aggregated and submitted to the Sui blob object to certify the blob. -Certifying a blob emits a Sui event with the blob ID and the period of availability. -
A blob is considered available on Walrus once the corresponding Sui blob object has been -certified in the final step. The steps involved in a store operation can be executed by the binary -client, or a publisher that accepts and publishes blobs via HTTP.
-Walrus currently allows the storage of blobs up to a maximum size that may be determined
-through the walrus info CLI command. The
-maximum blob size is currently 957 MiB. You may store larger blobs by splitting them into
-smaller chunks.
Blobs are stored for a certain number of epochs, as specified at the time they were stored. Walrus -storage nodes ensure that within these epochs a read succeeds. The Walrus devnet only uses a single -epoch today, and blobs uploaded will be available in that single epoch (until the devnet is wiped). -Future devnets may span across multiple epochs.
-Read
-Walrus can also be used to read a blob after it is stored by providing its blob ID. -A read is executed by performing the following steps:
--
-
- The system object on Sui is read to determine the Walrus storage node committee. -
- A number of storage nodes are queried for blob metadata and the slivers they store. -
- The blob is reconstructed from the recovered slivers and checked against the blob ID. -
The steps involved in the read operation are performed by the binary client, or the aggregator -service that exposes an HTTP interface to read blobs. Reads are extremely resilient and will -succeed in recovering the blob stored even if up to one-third of storage nodes are -unavailable in all cases. Eventually, after synchronization is complete, even if two-thirds -of storage nodes are down reads will succeed.
-Certify Availability
-Walrus can be used to certify the availability of a blob using Sui. Checking that this happened -may currently be done in 3 different ways:
--
-
- A Sui SDK read can be
-used to authenticate the certified blob event emitted when the blob ID was certified on Sui. The
-client
walrus blob-statuscommand may be used to identify the event ID that needs to be checked.
- - A Sui SDK read may be -used to authenticate the Sui blob object corresponding to the blob ID, and check it is certified. -
- A Sui smart contract can read the blob object on Sui (or a reference to it) to check -is is certified. -
The underlying protocol of the -Sui light client -returns digitally signed evidence for emitted events -or objects, and can be used by off-line or non-interactive applications as a proof of availability -for the blob ID for a certain number of epochs.
-Once a blob is certified, Walrus will ensure that sufficient slivers will always be -available on storage nodes to recover it within the specified epochs.
-Sui Structures
-This section is optional and enables advanced use cases.
-You can interact with Walrus purely -through the client CLI, and JSON or HTTP APIs provided, without querying or executing transactions -on Sui directly. However, Walrus uses Sui to manage its metadata and smart contract developers can -read information about the Walrus system, as well as stored blobs, on Sui.
-This section provides an overview of how you may use Walrus objects in your Sui smart contracts.
-Blob and Storage Objects
-Walrus blobs are represented as Sui objects of type Blob. A blob is first registered, indicating
-that the storage nodes should expect slivers from a Blob ID to be stored. Then a blob is certified
-indicating that a sufficient number of slivers have been stored to guarantee the blob's
-availability. When a blob is certified its certified_epoch field contains the epoch in which it
-was certified.
A Storage object is always associated with a Blob object, reserving enough space for
-a long enough period for the blob's storage. A certified blob is available for the period the
-underlying storage resource guarantees storage.
Concretely, Blob and Storage objects have the following fields, that can be read through the
-Sui SDKs:
/// The blob structure represents a blob that has been registered to with some
-/// storage, and then may eventually be certified as being available in the
-/// system.
-public struct Blob has key, store {
- id: UID,
- stored_epoch: u64,
- blob_id: u256,
- size: u64,
- erasure_code_type: u8,
- certified_epoch: option::Option<u64>, // The epoch first certified,
- // or None if not certified.
- storage: Storage,
-}
-
-/// Reservation for storage for a given period, which is inclusive start,
-/// exclusive end.
-public struct Storage has key, store {
- id: UID,
- start_epoch: u64,
- end_epoch: u64,
- storage_size: u64,
-}
-All fields of Blob and Storage objects can be read using the expected functions:
// Blob functions
-public fun stored_epoch(b: &Blob) : u64;
-public fun blob_id(b: &Blob) : u256;
-public fun size(b: &Blob) : u64;
-public fun erasure_code_type(b: &Blob) : u8;
-public fun certified_epoch(b: &Blob) : &Option<u64>;
-public fun storage(b: &Blob) : &Storage;
-
-// Storage functions
-public fun start_epoch(self: &Storage) : u64;
-public fun end_epoch(self: &Storage) : u64;
-public fun storage_size(self: &Storage) : u64;
-Events
-When a blob is first registered a BlobRegistered event is emitted that informs storage nodes
-that they should expect slivers associated with its Blob ID. Eventually when the blob is
-certified a BlobCertified is emitted containing information about the blob ID and the epoch
-after which the blob will be deleted. Before that epoch the blob is guaranteed to be available.
/// Signals a blob with metadata is registered.
-public struct BlobRegistered has copy, drop {
- epoch: u64,
- blob_id: u256,
- size: u64,
- erasure_code_type: u8,
- end_epoch: u64,
-}
-
-/// Signals a blob is certified.
-public struct BlobCertified has copy, drop {
- epoch: u64,
- blob_id: u256,
- end_epoch: u64,
-}
-The InvalidBlobID event is emitted when storage nodes detect an incorrectly encoded blob.
-Anyone attempting a read on such a blob is guaranteed to also detect it as invalid.
/// Signals that a BlobID is invalid.
-public struct InvalidBlobID has copy, drop {
- epoch: u64, // The epoch in which the blob ID is first registered as invalid
- blob_id: u256,
-}
-System information
-The Walrus system object contains metadata about the available and used storage, as well as the -price of storage per Kib of storage in MIST. The committee -structure within the system object can be used to read the current epoch number, as well as -information about the committee.
-const BYTES_PER_UNIT_SIZE : u64 = 1_024;
-
-public struct System<phantom WAL> has key, store {
-
- id: UID,
-
- /// The current committee, with the current epoch.
- /// The option is always Some, but need it for swap.
- current_committee: Option<Committee>,
-
- /// When we first enter the current epoch we SYNC,
- /// and then we are DONE after a cert from a quorum.
- epoch_status: u8,
-
- // Some accounting
- total_capacity_size : u64,
- used_capacity_size : u64,
-
- /// The price per unit size of storage.
- price_per_unit_size: u64,
-
- /// Tables about the future and the past.
- past_committees: Table<u64, Committee>,
- future_accounting: FutureAccountingRingBuffer<WAL>,
-}
-
-public struct Committee has store {
- epoch: u64,
- bls_committee : BlsCommittee,
-}
-A few public functions of the committee allow contracts to read Walrus metadata:
-/// Get epoch. Uses the committee to get the epoch.
-public fun epoch<WAL>(self: &System<WAL>) : u64;
-
-/// Accessor for total capacity size.
-public fun total_capacity_size<WAL>(self: &System<WAL>) : u64;
-
-/// Accessor for used capacity size.
-public fun used_capacity_size<WAL>(self: &System<WAL>) : u64;
-
-// The number of shards
-public fun n_shards<WAL>(self: &System<WAL>) : u16;
-Setup
-At this stage of the project, our Walrus code is not yet public. Instead, we provide a pre-compiled
-walrus client binary for macOS (Intel and Apple CPUs) and Ubuntu, which supports different usage
-patterns (see the next chapter). This chapter describes the
-prerequisites, installation, and configuration
-of the Walrus client.
Note that our Walrus devnet uses Sui testnet for coordination.
-Prerequisites
-Interacting with Walrus requires a valid Sui testnet wallet with some amount of SUI tokens. The -easiest way to set this up is via the Sui CLI; see the installation -instructions in the Sui -documentation.
-After installing the Sui CLI, you need to set up a testnet wallet by running sui client, which
-prompts you to set up a new configuration. Make sure to point it to Sui testnet, you can use the
-full node at https://fullnode.testnet.sui.io:443 for this. See
-here for further details.
If you already have a Sui wallet configured, you can directly set up the testnet environment (if you -don't have it yet),
-sui client new-env --alias testnet --rpc https://fullnode.testnet.sui.io:443
-and switch the active environment to it:
-sui client switch --env testnet
-After this, you should get something like this (everything besides the "testnet" line is optional):
-$ sui client envs
-╭──────────┬─────────────────────────────────────┬────────╮
-│ alias │ url │ active │
-├──────────┼─────────────────────────────────────┼────────┤
-│ devnet │ https://fullnode.devnet.sui.io:443 │ │
-│ localnet │ http://127.0.0.1:9000 │ │
-│ testnet │ https://fullnode.testnet.sui.io:443 │ * │
-│ mainnet │ https://fullnode.mainnet.sui.io:443 │ │
-╰──────────┴─────────────────────────────────────┴────────╯
-Finally, make sure you have at least 2 separate gas coins, with at least 1 SUI each. You can obtain -these coins from the testnet faucet:
-sui client faucet && sui client faucet
-After some seconds, you should see your new SUI coins:
-$ sui client gas
-╭─────────────────┬────────────────────┬──────────────────╮
-│ gasCoinId │ mistBalance (MIST) │ suiBalance (SUI) │
-├─────────────────┼────────────────────┼──────────────────┤
-│ 0x65dca966dc... │ 1000000000 │ 1.00 │
-│ 0xb07a091c1f... │ 1000000000 │ 1.00 │
-╰─────────────────┴────────────────────┴──────────────────╯
-The system-wide wallet will be used by Walrus if no other path is specified. If you want to use a -different Sui wallet, you can specify this in the Walrus configuration file or -when running the CLI.
-Installation
-We currently provide the walrus client binary for macOS (Intel and Apple CPUs) and Ubuntu:
| OS | CPU | Architecture |
|---|---|---|
| MacOS | Apple Silicon | macos-arm64 |
| MacOS | Intel 64bit | macos-x86_64 |
| Ubuntu | Intel 64bit | ubuntu-x86_64 |
You can download the latest build from our Google Cloud Storage (GCS) bucket (correctly setting the
-$SYSTEM variable) and move it to a directory included in your $PATH:
SYSTEM=ubuntu-x86_64 # or macos-x86_64 or macos-arm64
-curl https://storage.googleapis.com/mysten-walrus-binaries/latest/walrus-latest-$SYSTEM -o walrus
-chmod +x walrus
-mv walrus ~/.local/bin
-Once this is done, you should be able to simply type walrus in your terminal. For example you can
-get usage instructions (see the next chapter for further details):
$ walrus --help
-Walrus client
-
-Usage: walrus [OPTIONS] <COMMAND>
-
-Commands:
-⋮
-Custom path (optional)
-Instead of ~/.local/bin, you can place the binary in any other directory you like. You need to
-either make sure to add that directory to your $PATH or always call the binary as
-/full/path/to/walrus.
Previous versions (optional)
-In addition to the latest version of the walrus binary, the GCS bucket also contains previous
-versions. An overview in XML format is available at
-https://storage.googleapis.com/mysten-walrus-binaries/.
Configuration
-A single parameter is required to configure Walrus, namely the ID of the system -object on Sui. You can create your client -configuration as follows:
- -mkdir ~/.walrus
-curl https://storage.googleapis.com/mysten-walrus-binaries/walrus-configs/client_config.yaml \
- -o ~/.walrus/client_config.yaml
-Custom path (optional)
-By default, the Walrus client will look for the client_config.yaml configuration file in the
-current directory or in ~/.walrus/, but you can place the file anywhere and name it anything you
-like; in this case you need to use the --config option when running the walrus binary.
Advanced configuration (optional)
-The configuration file currently supports the following parameters:
-# This is the only mandatory field. The system object is specific for a particular Walrus
-# deployment.
-#
-# NOTE: THE VALUE INCLUDED HERE IS AN EXAMPLE VALUE.
-# You can get the object ID for the current Walrus devnet deployment as described above.
-system_object: 0x3243....
-
-# You can define a custom path to your Sui wallet configuration here. If this is unset or `null`,
-# the wallet is configured from `./sui_config.yaml` (relative to your current working directory), or
-# the system-wide wallet at `~/.sui/sui_config/client.yaml` in this order.
-wallet_config: null
-
-# The following parameters can be used to tune the networking behavior of the client. There is no
-# risk in playing around with these values. In the worst case, you may not be able to store/read
-# blob due to timeouts or other networking errors.
-communication_config:
- max_concurrent_writes: null
- max_concurrent_sliver_reads: null
- max_concurrent_metadata_reads: 3
- max_concurrent_status_reads: null
- reqwest_config:
- total_timeout:
- secs: 180
- nanos: 0
- pool_idle_timeout: null
- http2_keep_alive_timeout:
- secs: 5
- nanos: 0
- http2_keep_alive_interval:
- secs: 30
- nanos: 0
- http2_keep_alive_while_idle: true
- request_rate_config:
- max_node_connections: 10
- max_retries: 5
- min_backoff:
- secs: 2
- nanos: 0
- max_backoff:
- secs: 60
- nanos: 0
-Important: If you specify a wallet path, make sure your wallet is set up for Sui testnet.
-Interacting with Walrus
-We provide 3 ways to interact directly with the Walrus storage system:
--
-
- Through the Walrus client command line interface (CLI). -
- Through a JSON API of the Walrus CLI. -
- Through an HTTP API exposed by a public or local Walrus client daemon. -
Using the Walrus client
-The walrus binary can be used to interact with Walrus as a client. See the setup
-chapter for prerequisites, installation, and configuration.
Detailed usage information is available through
-walrus --help
-Each sub-command of walrus can also be called with --help to print its specific
-arguments and their meaning.
Walrus system information
-Information about the Walrus system is available through the walrus info command. For example,
$ walrus info
-
-Walrus system information
-
-Storage nodes
-Number of nodes: 10
-Number of shards: 270
-
-Blob size
-Maximum blob size: 957 MiB (1,003,471,920 B)
-
-Approximate storage prices per epoch
-Price per encoded storage unit: 50 MIST/KiB
-Price to store metadata: 850 MIST
-Marginal price per additional 1 MiB (w/o metadata): 239,250 MIST
-Total price per max blob (957 MiB): 0.227 SUI
-gives an overview of the number of storage nodes and shards in the system, the maximum blob size, -and the current cost in (testnet) Sui for storing blobs.
-Additional information such as encoding parameters and sizes, BFT system information, and
-information on the storage nodes and their shard distribution can be viewed with the --dev
-argument: walrus info --dev.
Storing, querying status and reading blobs
-Storing blobs on Walrus can be achieved through the following commands:
-walrus store <some file>
-The store command takes a CLI argument --epochs <EPOCHS> (or -e) indicating the number of
-epochs the blob should be stored for. This defaults to 1 epoch, namely the current one. If the blob
-is already stored on Walrus for a sufficient number of epochs the command does not store it again.
However, this behavior can be overwritten with the --force (or -f) CLI option, which stores
-the blob again and creates a fresh blob object on Sui belonging to the wallet address.
The status of a blob by blob ID can be queried using the command:
-walrus blob-status --blob-id <BLOB_ID>
-This returns whether the blob is stored and its availability period. You can also query the -status of a blob by using a file that stores it, as:
-walrus blob-status --file <FILE>
-This command re-encodes the content of the file, derives the blob ID and returns whether it -is stored on Walrus and its availability period.
-When the blob is available the blob-status command also returns the BlobCertified Sui event ID,
-which consists of a transaction ID and a sequence number in the events emitted by the transaction.
-The existence of this event certifies the availability of the blob.
Reading blobs from Walrus can be achieved through the following command:
-walrus read <some blob ID>
-By default the blob data is written to the standard output. The --out <OUT> CLI option (or -o)
-can be used to specify and output file name. The --rpc-url <URL> (or -r) may be used to specify
-a Sui RPC node to use instead of the one set in the wallet configuration or the default one.
Changing the default configuration
-Use the --config option to specify a custom path to the
-configuration location.
The
---wallet <WALLET> argument may be used to specify a non standard Sui wallet configuration file.
-And a --gas-budget <GAS_BUDGET> argument may be used to change the maximum amount of Sui (in MIST)
-that the command is allowed to use.
Troubleshooting
-If you get an error like "the specified Walrus system object does not exist", make sure your wallet -is set up for Sui testnet and you use the latest configuration.
-JSON mode
-All Walrus client commands (except, currently, the info command) are also available in JSON mode.
-In this mode, all the command line flags of the original CLI command can be specified in JSON
-format. The JSON mode therefore simplifies programmatic access to the CLI.
For example, to store a blob, run:
-walrus json \
- '{
- "config": "path/to/client_config.yaml",
- "command": {
- "store": {
- "file": "README.md"
- }
- }
- }'
-Or, to read a blob knowing the blob ID:
-walrus json \
- '{
- "config": "path/to/client_config.yaml",
- "command": {
- "read": {
- "blobId": "4BKcDC0Ih5RJ8R0tFMz3MZVNZV8b2goT6_JiEEwNHQo"
- }
- }
- }'
-All options, default values, and commands are equal to those of the "standard" CLI mode, except that -they are written in "camelCase" instead of "kebab-case".
-The json command also accepts input from stdin.
The output of a json command will itself be JSON-formatted, again to simplify parsing the results
-in a programmatic way. For example, the JSON output can be piped to the jq command for parsing and
-manually extracting relevant fields.
Client Daemon mode & HTTP API
-In addition to the CLI and JSON modes, the Walrus client offers a daemon mode. In this mode, it -runs a simple web server offering HTTP interfaces to store and read blobs in an aggregator and -publisher role respectively. We also offer -public aggregator and publisher services to try the Walrus HTTP APIs without -the need to run a local client.
-Starting the daemon locally
-You can run the daemon with the following command, to offer both an aggregator and publisher on -the same address and port:
-ADDRESS="127.0.0.1:31415" # bind the daemon to localhost and port 31415 for both
-walrus daemon -b $ADDRESS # run a daemon combining an aggregator and a publisher
-Or you may run two separate processes. One for the aggregator:
-AGG_ADDRESS="127.0.0.1:31415" # Aggregator only port
-walrus aggregator -b $AGG_ADDRESS # run an aggregator to read blobs
-And a different one for the publisher:
-PUB_ADDRESS="127.0.0.1:31416" # Note different port for publisher
-walrus publisher -b $PUB_ADDRESS # run a publisher to store blobs
-The aggregator provides all read APIs, the publisher all the store APIs, and the daemon provides -both. Note that the aggregator does not perform Sui on-chain actions, and therefore consumes no gas. -However, the publisher does perform actions on-chain and will consume gas. It is therefore important -to ensure only authorized parties may access it, or other measures to manage gas costs.
-Using a public aggregator or publisher
-For some use cases (e.g., a public website), or to just try out the HTTP API, a publicly accessible -aggregator and/or publisher is required. For your convenience, we provide these at the following -hosts:
--
-
- Aggregator:
https://aggregator-devnet.walrus.space
- - Publisher:
https://publisher-devnet.walrus.space
-
Our publisher is currently limiting requests to 10 MiB. If you want to upload larger files, you need -to run your own publisher or use the CLI.
-Note that the publisher consumes (testnet) Sui on the service side, and a mainnet deployment would -likely not be able to provide uncontrolled public access to publishing without requiring some -authentication and compensation for the Sui used.
-HTTP API Usage
-Store
-You can interact with the daemon through simple HTTP PUT requests. For example, with -cURL, you can store blobs using a publisher or daemon as follows:
-ADDRESS="127.0.0.1:31415"
-curl -X PUT "http://$ADDRESS/v1/store" -d "some string" # store the string `some string` for 1 storage epoch
-curl -X PUT "http://$ADDRESS/v1/store?epochs=5" --upload-file "some/file" # store file `some/file` for 5 storage epochs
-The store HTTP API end points return information about the blob stored in JSON format. When a blob
-is stored for the first time, a newlyCreated field contains information about the
-new blob:
$ curl -X PUT "http://ord-dnt-sto-00.devnet.sui.io:9000/v1/store" -d "some other string"
-{
- "newlyCreated":{
- "id":"0xa74d376bb6923c4b8d73825fce3b798524e2bda34d02dae64ab5865556c54000",
- "storedEpoch":0,
- "blobId":"gsYzDXsK326Wihzt3X5evZCPFgaNZrb84zCjodLJaVs",
- "size":36,
- "erasureCodeType":"RedStuff",
- "certified":0,
- "storage":{
- "id":"0x2479b3096efa58fa9bfbebe805268746c235110ee0713e5a123d8f1754c2ccc3",
- "startEpoch":0,
- "endEpoch":1,
- "storageSize":4747680
- }
- }
-}
-The information returned is the content of the Sui blob object.
-When the aggregator finds a certified blob with the same blob ID and a sufficient validity period,
-it returns a alreadyCertified JSON structure:
$ curl -X PUT "http://ord-dnt-sto-00.devnet.sui.io:9000/v1/store" -d "some other string"
-{
- "alreadyCertified":{
- "blobId":"gsYzDXsK326Wihzt3X5evZCPFgaNZrb84zCjodLJaVs",
- "event":{
- "txDigest":"2oMC1dTaMGpApphFWKzARSsTM9837ox4zN8rY2RqLf6c",
- "eventSeq":"0"
- },
- "endEpoch":1
- }
-}
-The field event returns the Sui event ID that can be used to
-find the transaction that created the Sui Blob object on the Sui explorer or using a Sui SDK.
Read
-Blobs may be read from an aggregator or daemon using HTTP GET. For example the following cURL -command reads a blob and writes it to an output file:
-ADDRESS="127.0.0.1:31415"
-curl "http://$ADDRESS/v1/<some blob ID> -o <some file name>"
-Alternatively you may print the contents of a blob in the terminal with the cURL command:
-ADDRESS="127.0.0.1:31415"
-curl "http://$ADDRESS/v1/<some blob ID>
-Modern browsers will attempt to sniff the content type for such resources, and will generally do a -good job of inferring content types for media. However, the aggregator on purpose prevents such -sniffing from inferring dangerous executable types such as javascript or style sheet types.
-Examples
-As inspiration, we provide several simple examples in different programming languages to interact -with Walrus through the various interfaces. They are located at -https://github.com/MystenLabs/walrus-docs/tree/main/examples and described below.
-In addition, we have built actual applications on top of Walrus. The prime example is Walrus -Sites, with code available in the -github.com/MystenLabs/walrus-sites repository.
-And for an example of how to build a static website and store it as a Walrus Site with GitHub -actions, just look at the CI -workflow we use -to publish this very site.
-Python
-The Python examples folder -contains a number of examples:
--
-
- How to use the HTTP -API -to store and read a blob. -
- How to use the JSON -API -to store, read, and check the availability of a blob. Checking the certification of a blob -illustrates reading the Blob Sui object that certifies (see the Walrus Sui -reference). -
- How to read information from the Walrus system -object. -
- How to track Walrus related -Events. -
JavaScript
-A JavaScript example is -provided showing how to upload and download a blob through a web form using the HTTP API.
-Introduction to Walrus Sites
-Walrus Sites are "web"-sites that use Sui and Walrus as their underlying technology. They are a -prime example of how Walrus can be used to build new and exciting decentralized applications. Anyone -can build and deploy a Walrus Site and make it accessible to the World! Funnily, this documentation -is itself available as a Walrus site at https://docs.walrus.site/walrus-sites/intro.html (if you -aren't there already).
-At a high level, the most exciting features include:
--
-
- Publishing a site does not require managing servers or complex configurations; just provide the -source files (produced by your favorite web framework), publish them to Walrus Sites using the -site builder tool, and you are done! -
- Sites can be linked to from ordinary Sui objects. This feature enables, for example, creating an -NFT collection in which every single NFT has a personalized website dedicated to it. -
- Walrus Sites are owned by addresses on Sui and can be exchanged, shared, and updated thanks to -Sui's flexible programming model. This means, among other things, that Walrus Sites can leverage -the SuiNS naming system to have human readable names. No more messing around -with DNS! -
- Thanks to Walrus' decentralization and extremely high data availability, there is no risk of -having your site wiped for no reason. -
- Since they live on Walrus, these sites cannot have a backend in the traditional sense, and can be -therefore considered "static" sites. However, the developer can harness Sui's programmability -to add backend functionality to Walrus Sites! -
Show me
-To give you a very high level intuition of how Walrus Sites work, let's look at an example: A simple -NFT collection on Sui that has a frontend dApp to mint the NFTs hosted on Walrus Sites, and in -which each NFT has a specific, personalized Walrus Site.
-You can check out the mint page at https://flatland.walrus.site/. This site is served to your
-browser through the Walrus Site Portal https://walrus.site. While the Portal's operation is
-explained in a later section, consider for now that there can be many Portals (hosted
-by whoever wants to have their own, and even on localhost). Further, the only function of the
-Portal is to provide the browser with some code (specifically, a service worker) that allows it to
-fetch the Walrus Site from Sui and Walrus.
If you have a Sui wallet with some Testnet SUI, you can try and "mint a new Flatlander" from the -site. This creates an NFT from the collection, and shows you two links, one to the explorer, and one -to the "Flatlander site". This latter site is a special Walrus Site that exists only for that NFT, -and has special characteristics (the background color, the image...) that are based on the contents -of the NFT.
-The URL to this per-NFT site look something like this
-https://4egmmrw9izzjn0dm2lkd3k0l8phk386z60ub1tpdc1jswbb5dr.walrus.site/. You'll notice that the
-domain remains walrus.site, but the subdomain is a long and random-looking string. This string is
-actually the Base36 encoding of the object ID of the NFT,
-which is
-0xb09b312b....
In summary:
--
-
- Walrus Sites are served through a Portal; in this case,
https://walrus.site. There can be many -Portals, and anyone can host one.
- - The subdomain on the URL points to a specific object on Sui, that allows the browser to fetch and
-render the site resources. This pointer can be:
-
-
-
- A SuiNS name, such as
flatlandinhttps://flatland.walrus.site; or
- - the Base36 encoding of a the Sui object ID, such as
0xb09b312b...in the example above.
-
- - A SuiNS name, such as
Curious to know how this magic is possible? Read the technical -overview! If you just want to get started trying Walrus Sites out, check the -tutorial.
-Your first Walrus Site
-This tutorial walks you through the steps necessary to publish a Walrus Site. We also provide the -instructions on how to add a SuiNS name to it for convenient browsing.
-Installing the site builder
-We describe here the steps necessary to setup the Walrus Sites' site-builder tool, and prepare
-your environment for development.
Prerequisites
-Before you start, make sure you:
--
-
- have a recent version of Rust installed; -
- have
gitinstalled; and
- - followed all Walrus setup instructions. -
Then, follow these additional setup steps.
-Clone the repository and build the site-builder tool
-First clone and enter the Walrus Sites repo from https://github.com/MystenLabs/walrus-sites.
-git clone git@github.com:MystenLabs/walrus-sites.git
-cd walrus-sites
-cd site-builder
-Then, build the release version of the site builder:
-cargo build --release
-After the build process completes, you are ready to run the site builder:
-$ ./target/release/site-builder
-Usage: site-builder [OPTIONS] <COMMAND>
-
-Commands:
- publish Publish a new site on Sui
- update Update an existing site
- convert Convert an object ID in hex format to the equivalent Base36
- format
- sitemap Show the pages composing the Walrus site at the given object ID
- help Print this message or the help of the given subcommand(s)
-
-⋮
-Publishing a Walrus Site
-Now that everything is installed and configured, you should be able to start publishing -your first Walrus Site!
-Select the source material for the site
-The site-builder works by uploading a directory of files produced by any web framework to Walrus,
-and adding the relevant metadata to Sui. This directory should have a file called index.html in
-its root, which will be the entry point to the Walrus Site.
For the rest of the tutorial, we will use as an example the simple site contained in
-./examples/snake.
Publish the site
-Since we have placed the walrus binary and configuration in their default locations, publishing
-the ./examples/snake site is simple:
-
-
-
-
Ensure that you are in the
-site-builderdirectory;
- -
-
Run the publishing command:
-
-./target/release/site-builder --config assets/builder-example.yaml publish ../examples/snake -
-
The output should look like the following:
-Operations performed:
-- created resource /Oi-Regular.ttf with blob ID 2YLU3Usb-WoJAgoNSZUNAFnmyo8cfV8hJYt2YdHL2Hs
-- created resource /file.png with blob ID R584P82qm4Dn8LoQMlzkGZS9IAkU0lNZTVlruOsUyOs
-- created resource /index.html with blob ID SSzbpPfO2Tqk6xNyF1i-NG9I9CjUjuWnhUATVSs5nic
-- created resource /walrus.png with blob ID SGrrw5NQyFWtqtxzLAQ1tLpcChGc0VNbtFRhfsQPuiM
-
-Created new site: test site
-New site object ID: 0x5ac988828a0c9842d91e6d5bdd9552ec9fcdddf11c56bf82dff6d5566685a31e
-
-Browse the resulting site at: https://29gjzk8yjl1v7zm2etee1siyzaqfj9jaru5ufs6yyh1yqsgun2.walrus.site
-This output tells you that, for each file in the folder, a new Walrus blob was created, and the -respective blob ID. Further it prints the object ID of the Walrus Site object on Sui (so you can -have a look in the explorer, and use it to set the SuiNS name), and, finally, the URL at which you -can browse the site.
-Note here that we are passing the example config assets/builder-example.yaml as the config for the
-site builder. The configuration file is necessary to ensure that the site-builder knows the
-correct Sui package for the Walrus Sites logic.
More details on the configuration of the site-builder can be found under the advanced
-configuration section.
Update the site
-Let's say now you want to update the content of the site, for example by changing the title from -"eat all the blobs!" to "Glob all the Blobs!".
-First, make this edit on in the ../examples/snake/index.html file.
Then, you can update the existing site by running the update command, and providing the directory
-where to find the updated files (still ../example/snake), and the object ID of the existing site
-(0x5ac988...):
./target/release/site-builder --config assets/builder-example.yaml update ../examples/snake 0x5ac9888...
-The output this time should be:
-Operations performed:
- - deleted resource /index.html with blob ID SSzbpPfO2Tqk6xNyF1i-NG9I9CjUjuWnhUATVSs5nic
- - created resource /index.html with blob ID LXtY0VdY5kM-3Ph7gLvj8URdz5yiRa5DUy3ZxYqDView
-
-Updated site at object ID: 0x5ac988828a0c9842d91e6d5bdd9552ec9fcdddf11c56bf82dff6d5566685a31e
-
-Browse the resulting site at: https://29gjzk8yjl1v7zm2etee1siyzaqfj9jaru5ufs6yyh1yqsgun2.walrus.site
-Compared to the publish action, we can see that now the only actions performed were to delete the
-old index.html, and update it with the newer one.
Browsing to the provided URL should reflect the change. You've updated the site!
-Additional commands
-The site-builder tool provides two additional utilities:
-
-
- the
convertcommand, which converts an object ID in hex format to the equivalent Base36 -format. This command is useful if you have the Sui object ID of a Walrus Site, and want to know -the subdomain where you can browse it.
- - the
sitemapcommand, which shows the resources that compose the Walrus Site at the given object -ID.
-
In general, the --help flag is your friend!
Bonus: Set a SuiNS name
-Browsing a URL like https://29gjzk8yjl1v7zm2etee1siyzaqfj9jaru5ufs6yyh1yqsgun2.walrus.site is not
-particularly nice. Therefore, Walrus Sites allows to use SuiNS names (this is like DNS for Sui) to
-give human-readable names to site. To do so, you simply have to get a SuiNS name you like, and point
-it to the object ID of the Walrus Site (as provided by the publish or update commands).
Let's do this step by step.
-Get a SuiNS name
--
-
- Navigate to https://testnet.suins.io/, and buy a domain name with
-your testnet wallet. For example,
walrusgame(NOTE: this is already taken, choose another you -like!). NOTE: At the moment, you can only select names that are composed of lettersa-zand -numbers0-9, but no special characters (e.g.,-).
- - In the page listing the domains you own, you should -see the newly-bought name. -
- Click the three-dots menu on the top-right corner of the name you want to assign. Choose "View all
-info", and copy the
ObjectID. In our case, this is -0x6412c4cfbe50e219c2d4d30108d7321d064e15bf64e752307100bff5eb91da38.
-
Send the SuiNS registration object to the address you use with the Sui CLI
-The steps that follow require that the SuiNS registration object is owned by the address you are -using on the Sui CLI. Therefore, we need to send this registration object from the address you use -in your browser wallet, to the address of your Sui CLI.
-To find the Sui CLI address, execute:
-sui client active-address
-Then, from your browser wallet, select the "Assets" tab, and look for the NFT of the SuiNS -registration, which should look as follows:
-
Click on it, scroll down to "Send NFT", and send it to the address discovered with the command -above. Now, your Sui CLI address owns the registration NFT, and you can proceed to the next step.
-Map the SuiNS name to the Walrus Site
-This step associates the name walrusgame to the object ID of our Walrus Site. There are possibly
-many ways to achieve this, and as the SuiNS UI improves this could be done from the webapp as well.
Here, we issue a transaction using the Sui CLI that creates this mapping:
-SUINS_UTILS_PACKAGE=0x7954ae683314ec7e156acbf0c0fc964ce035fd7f456fe7576848226502cfde1b
-SUINS_CORE_OBJECT=0x300369e8909b9a6464da265b9a5a9ab6fe2158a040e84e808628cde7a07ee5a3
-MY_SUINS_REGISTRATION_OBJECT=0x6412... # adjust this to your own SuiNS object
-MY_WALRUS_SITE_OBJECT=0x5ac9... # adjust this to your Walrus Site object
-sui client call \
- --package $SUINS_UTILS_PACKAGE \
- --module direct_setup \
- --function set_target_address \
- --gas-budget 500000000 \
- --args $SUINS_CORE_OBJECT \
- --args $MY_SUINS_REGISTRATION_OBJECT \
- --args "[$MY_WALRUS_SITE_OBJECT]" \
- --args 0x6
-Note that the SuiNS package and object on testnet may change. You can
-find the latest ones by looking at the TESTNET_CONFIG in the SuiNS
-contract.
If all succeeds, we can now browse https://walrusgame.walrus.site!
-Configuring the site builder
-Configuring the site-builder tool is straightforward, but care is required to ensure that
-everything works correctly.
The site-builder tool requires a configuration file to know which package to use on Sui, which
-wallet to use, the gas budget, and other operational details. Most of these are abstracted away
-through sensible defaults, so you should not need to touch them. Yet, for completeness, we provide
-here the details for all the configuration options.
Minimal configuration
-The config file is expected to be in ./builder.yaml, and it is possible to point elsewhere with
-the --config flag. For your first run, it should be sufficient to call the site-builder with
---config assets/builder-example.yaml, which is already configured appropriately.
If, for any reason, you didn't add walrus to $PATH, make sure to configure a pointer to the
-binary, see below.
Advanced configuration
-If you want to have more control over the behavior of the site builder, you can customize the -following variables in the config file:
--
-
package: the object ID of the Walrus Sites package on Sui. This must always be specified in the -config, and is already appropriately configured inassets/example-config.yaml.
-portal: the name of the Portal through which the site will be viewed; this only affects the -output of the CLI, and nothing else (default:walrus.site). -All Walrus Sites are accessible through any Portal independent of this setting.
-general: these are general options, that can be configured both through the CLI and the config: --
-
rpc_url: The URL of the Sui RPC node to use. If not set, thesite-builderwill infer it from -the wallet.
-wallet: Pointer to the Sui wallet to be used. By default, it uses the system-wide wallet (the -one fromsui client addresses).
-walrus_binary: Pointer to thewalrusbinary. By default, this is expected to be run from -$PATH.
-walrus_config: The configuration for thewalrusclient binary, see the relevant -chapter.
-gas_budget: The maximum amount of gas to be spent for transactions (default: 500M MIST).
-
-
Technical Overview
-In the following sections, we delve deeper in the technical specification of Walrus Sites.
-High-level picture
-Walrus Sites are enabled by Sui and Walrus. The sites' resources (html, css, js, images,
-etc.) are stored on Walrus, while the main entry points to the sites are objects stored on Sui,
-which contain the metadata for the site and point to the Walrus blob IDs.
The Walrus Sites objects on Sui
-A Walrus Site is represented on Sui as a very simple object:
struct Site has key, store {
- id: UID,
- name: String,
-}
-The resources associated with this site are then added to this object as dynamic
-fields of type Resource:
struct Resource has store, drop {
- path: String,
- content_type: String,
- content_encoding: String,
- // The walrus blob id containing the bytes for this resource
- blob_id: u256,
-}
-Each resource contains:
--
-
- The
pathof the resource, for example/index.html(all the paths are always represented as -starting from root/);
- - the
content_typeof the resource, for exampletext/html;
- - the
content_encodingof the resource. At the moment the only available value isplaintext; and
- - the
blob_id, which is the Walrus blob ID where the resource can be found.
-
These Resource dynamic fields are keyed with a struct of type ResourcePath
struct ResourcePath has copy, store, drop {
- path: String,
-}
-This struct just holds the string of the path (/index.html); having a separate type ensures that
-we will not have namespace collisions with other dynamic fields, possibly added by other packages.
To see this in action, look at a Walrus Site in the -explorer, -and check its dynamic fields.
-The site rendering path
-Given the Sui object ID of a Walrus Site, then, it is easy to look up the resources that compose it
-by looking at the dynamic fields, and then fetch these resources from Walrus using the blob ID
-contained in the Resource struct.
The only outstanding question is, therefore, how to perform these lookups on the client. A few -approaches are possible:
--
-
- Having a server that accepts requests for a Sui object ID and possibly a path, and performs this -resolution on behalf of the client, serving back the resource as a standard HTML Response. -
- Using a custom application on the client, that has both a web browser and knowledge of how Walrus -Sites work, and can locally perform this resolution. -
- A hybrid approach based on service workers, where a service worker that is able to perform the -resolution is installed in the browser from a Portal. -
All of these approaches are viable (the first has been used for similar applications in IPFS -gateways, for example), and have trade offs. As an initial step, we have chosen to use the -service-worker based approach, as it is light weight and ensures that the Portal does not have to -process all the traffic from clients. In the following, therefore, we present the details of this -Portal-based approach.
-Browsing and domain isolation
-We must ensure that, when browsing multiple sites through a Portal, for example the one hosted at -https://walrus.site, these sites are isolated. Isolation is necessary for security, and to ensure -that the wallet connection in the browser works as expected.
-To do so, we give each Walrus Site a specific subdomain of the Portal's domain. For example, the
-Flatland mint dApp is hosted at https://flatland.walrus.site, where the subdomain flatland is
-uniquely associated to the object ID of the Walrus Site through SuiNS.
Walrus Sites also work without SuiNS: a site can always be browsed by using as subdomain the -Base36 encoding of the Sui object ID of the site. For the Flatland dApp, this URL is: -https://44terjw9uzwbmtful0387e2bx3k3ro64s0it82hw9x9sz4ttm.blocksite.net .
-Base36 was chosen for two reasons, forced by the subdomain standards:
--
-
- A subdomain can have at most 63, while a Hex-encoded Sui object ID requires 64. -
- A subdomain is case insensitive, ruling out other popular encodings, e.g., Base64 or Base58. -
The end-to-end resolution of a Walrus Site
-We now show in greater detail how a Walrus Site is rendered in a client's browser with the service -worker approach. The steps below all reference the following figure:
-
-
-
- Site publishing (step 0) The site developer publishes the Walrus Site using the
-
site-builder, or making use of a publisher. Assume the developer uses the -SuiNS namedapp.suito point to the object ID of the created Walrus Site.
- - Browsing starts (step 1) A client browses
dapp.walrus.site/index.htmlin their browser.
- - Service worker installation (steps 2-6) The browser connects to the Portal hosted at
-
walrus.site, which responds with a page that installs the service worker for -dapp.walrus.site. The page is refreshed to activate the service worker.
- - Site resolution (steps 7-10) The service worker, which is now installed, interprets its
-origin
dapp.walrus.site, and makes a SuiNS resolution fordapp.sui, obtaining the relative -object ID. Using the object ID, it then fetches the dynamic fields of the object (also checking -redirects). From the dynamic fields, it selects the one for/index.html, and -extracts its Walrus blob ID and content type.
- - Blob fetch (steps 11-14) Given the blob ID, the service worker queries a Walrus aggregator for -the blob. -
- Returning the response (steps 15-16) Now that the service worker has the bytes for
-
/index.html, and itscontent_type, it can craft a response that is then rendered by the -browser.
-
These steps are executed for all resources the browser may query thereafter (for example, if
-/index.html points to assets/cat.png).
Linking from and to Walrus Sites
-Links in Walrus Sites work almost as you would expect in a regular website. We specify here a few -of the details.
-Linking to resources within the same site
-Relative and absolute links (href="/path/to/resource.html") work as usual.
Linking to resources on the Web
-Linking to a resource on the web (href="https://some.cdn.example.com/stylesheet.css") also work as
-usual.
Linking to resources in other Walrus Sites
-Here is the part that is a bit different. Assume there is some image that you can browse at
-https://gallery.walrus.site/walrus_arctic.webp, and you want to link it from your own Walrus Site.
Recall that, however, https://walrus.site is just one of the possibly many Portals. I.e., the same
-resource is browsable from a local portal (http://gallery.localhost:8080/walrus_arctic.webp), or
-from any other portal (e.g., https://gallery.myotherportal.com/walrus_arctic.webp). Therefore, how
-can you link the resource in a Portal independent way? This is important for interoperability,
-availability, and respecting the user's choice of portal.
The solution: Walrus Sites links
-We solve this problem by having the Portals interpret special links, that are normally invalid on -the Web, and redirect to the corresponding Walrus Sites resource in the portal itself.
-Consider the example above, where the resource /walrus_arctic.webp is browsed from the Walrus Site
-with SuiNS name gallery, which points to the object ID abcd123… (in Base36 encoding). Then,
-the Portal-independent link is: https://gallery.suiobj/walrus_arctic.webp. To fix the object ID
-instead of the SuiNS name, you can use https://abcd123….suiobj/walrus_arctic.webp.
Another possibility is to directly point to the Walrus blob ID of the resource, and have the
-browser "sniff" the content type. This works for images, for example, but not for script or
-stylesheets. For example to point to the blob ID (e.g., containing an image) qwer5678…, use the
-URL https://blobid.walrus/qwer5678….
With such a link, the Portal will extract the blob ID and redirect the request to the aggregator it -is using to fetch blobs.
-Redirecting objects to Walrus Sites
-We have seen in the overview how a Walrus Site object on Sui looks like. We will -discuss now how you can create ensure that a set of arbitrary objects can all be tied to a -specific, and possibly unique, Walrus site.
-The goal
-Consider a collection of NFTs, such as the one published by https://flatland.walrus.site. As we -show there, each minted NFT has its own Walrus Site, that can be personalized based on the contents -(e.g., the color) of the NFT itself. How can we achieve this?
-Redirect links
-The solution is simple: We add a "redirect" in the NFT's
-Display property. Each time an NFT's
-object ID is browsed through a Portal, the Portal will check the Display of the NFT and, if it
-encounters the walrus site address key, it will go fetch the Walrus Site that is at the
-corresponding object ID.
Redirects in Move
-Practically speaking, when creating the Display of the NFT, you can include the key-value pair
-that points to the Walrus site that is to beused.
...
-const VISUALIZATION_SITE: address = @0x901fb0...;
-display.add(b"walrus site address".to_string(), VISUALIZATION_SITE.to_string());
-...
-How to personalize based on the NFT?
-The code above will only open the specified Walrus Site when browsing the object ID of the NFT. How -do we ensure that the properties of the NFT can be used to personalize the site?
-This needs to be done in the VISUALIZATION_SITE: Since the subdomain is still pointing to the
-NFT's object ID, the Walrus Site that is loaded can check its origin in Javascript, and use the
-subdomain to determine the NFT, fetch it from chain, and use its internal fields to modify the
-displayed site.
The site builder
-To facilitate the creation of Walrus Sites, we provide the "site builder" tool. The site builder -takes care of creating Walrus Sites object on Sui, with the correct structure, and stores the site -resources to Walrus.
-Site builder commands
-The site builder tool exposes the following commands:
--
-
publish: Allows to publish a specific directory to Walrus. The directory must contain files that -can be served with HTTP, and have anindex.htmlfile. This command will return a Sui object ID -for the newly created site.
-update: After creating a site, you can update it with this command. It takes as input a -directory, as above, with the new or updated files, and the object ID of the site to update. Note -that the wallet you are using must be the owner of the Walrus Site object to be able to update -it. This command will remove and create resources as required, to ensure that the Walrus Sites -object on Sui matches the local directory. If run with--watch, this command re-updates the site -every time a file in the directory changes. This is useful during development, but pay attention -to costs!
-convert: A utility tool. Given a Sui object ID in Hex format, it converts it to Base36. This is -useful if you know the Sui object ID of a site, and want to find the URL.
-sitemap: A utility tool. For a give Walrus Sites Sui object ID, prints out all the resources -that compose the site and their object ID.
-
Check the commands' --help for more information.
The Walrus Sites Portal
-We use the term "Portal" to indicate any technology that is used to access an browse Walrus Sites. -As mentioned in the overview, we foresee three kinds of -Portals:
--
-
- Server-side Portals; -
- custom local apps; and -
- service-worker based Portals in the browser. -
Currently, only the service-worker based Portal is available.
-Running the Portal locally
-You can run a service worker Portal locally:
--
-
- To browse Walrus Sites without accessing external Portals; or -
- for development purposes. -
This requires having the pnpm tool installed. To start, clone the
-walrus-sites repo and enter the portal directory. Here, run:
pnpm install
-to install the dependencies, and
-pnpm serve
-to serve the Portal. Typically, you will find it served at localhost:8080 (but check the output of
-the serve command).
Configuring the Portal
-The most important configuration parameters for the Portal are in constants.ts.
-
-
NETWORK: The Sui network to be used for fetching the Walrus Sites objects. Currently, we -use Suitestnet.
-AGGREGATOR: The URL of the aggregator from which the service worker will -fetch the Walrus blobs.
-SITE_PACKAGE: "0x514cf7ce2df33b9e2ca69e75bc9645ef38aca67b6f2852992a34e35e9f907f58"
-MAX_REDIRECT_DEPTH: The number of redirects the service worker will follow -before stopping.
-SITE_NAMES: Hard codedname: objectIDmappings, to override the SuiNS names. For development -only.
-
Known restrictions
-Walrus Sites can be used to deploy almost any form of traditional static "Web 2" website build for -modern browsers. There are, however, a number of restrictions that a developer should keep in mind -when creating or porting a website to Walrus Sites.
-No secret values
-Walrus Sites are fully publicly accessible, as the metadata is stored on Sui and the site content is -stored on Walrus. Therefore, developers must not store secret values within the sites.
-We emphasize again that any such backend-specific operations (storing secret values, authentication, -etc.) are achievable by leveraging the integration with the Sui blockchain and a Sui-compatible -wallet.
-There is a maximum redirect depth
-The number of consecutive redirects a Walrus Site can perform is capped by the -Portal (see Portal configuration). This measure ensures that loading a Walrus Site -does not result in an infinite loading loop.
-Different Portals can set this limit as they desire. The limit for the Portal hosted at -walrus.site has a maximum redirect depth of 3.
-Service workers are not available
-WARNING: This limitation only applies to Portal based on service workers. A web Portal will not -have this limitation.
-Walrus Sites leverage service workers in the clients' browsers to perform these essential -operations:
--
-
- reading the site metadata from Sui; -
- fetching the page content from Walrus; and -
- serving the content to the browser. -
Therefore, a site deployed on Walrus Sites cannot use service workers. Installing a service worker -from within a Walrus Site will result in a dysfunctional site and a poor experience for the user.
-iOS Sui Mobile Wallets do not work with Walrus Sites
-WARNING: This limitation only applies to Portal based on service workers. A web Portal will not -have this limitation.
-Service workers cannot be loaded inside an in-app browser on iOS, because of a limitation of the -WebKit engine. As a consequence, Walrus Sites cannot be used within Sui-compatible wallet apps on -iOS. Therefore, Sui wallets cannot currently be used on a Walrus Site on iOS. Note, however, that -browsing a Walrus Site is still possible on iOS through any browser. Only the connection to the -wallet is impacted.
-The connection with the Sui Wallet apps works on Android devices.
-Walrus Glossary
-To make communication as clear and efficient as possible, we make sure to use a single term for -every Walrus entity/concept and do not use any synonyms. The following table lists various -concepts, their canonical name, and how they relate to or differ from other terms.
-Italicized terms in the description indicate other specific Walrus terms contained in the table.
-| Approved name | Description |
|---|---|
| storage node (SN) | entity storing data for Walrus; holds one or several shards |
| blob | single unstructured data object stored on Walrus |
| shard | (disjoint) subset of erasure-encoded data of all blobs; at every point in time, a shard is assigned to and stored on a single SN |
| sliver | erasure-encoded data of one shard corresponding to a single blob for one of the two encodings; this contains several erasure-encoded symbols of that blob but not the blob metadata |
| blob ID | cryptographic ID computed from a blob’s slivers |
| blob metadata | metadata of one blob; in particular, this contains a hash per shard to enable the authentication of slivers and recovery symbols |
| (end) user | any entity/person that wants to store or read blobs on/from Walrus; can act as a Walrus client itself or use the simple interface exposed by publishers and caches |
| publisher | service interacting with Sui and the SNs to store blobs on Walrus; offers a simple HTTP POST endpoint to end users |
| aggregator | service that reconstructs blobs by interacting with SNs and exposes a simple HTTP GET endpoint to end users |
| cache | an aggregator with additional caching capabilities |
| (Walrus) client | entity interacting directly with the SNs; this can be an aggregator/cache, a publisher, or an end user |
| (blob) reconstruction | decoding of the primary slivers to obtain the blob; includes re-encoding the blob and checking the Merkle proofs |
| (shard/sliver) recovery | process of an SN recovering a sliver or full shard by obtaining recovery symbols from other SNs |
| storage attestation | process where SNs exchange challenges and responses to demonstrate that they are storing their currently assigned shards |
| certificate of availability (CoA) | a blob ID with signatures of SNs holding at least $2f+1$ shards in a specific epoch |
| point of availability (PoA) | point in time when a CoA is submitted to Sui and the corresponding blob is guaranteed to be available until its expiration |
| inconsistency proof | set of several recovery symbols with their Merkle proofs such that the decoded sliver does not match the corresponding hash; this proves an incorrect/inconsistent encoding by the client |
| inconsistency certificate | an aggregated signature from 2/3 of SNs (weighted by their number of shards) that they have seen and stored an inconsistency proof for a blob ID |
| storage committee | the set of SNs for a storage epoch, including metadata about the shards they are responsible for and other metadata |
| member | an SN that is part of a committee at some epoch |
| storage epoch | the epoch for Walrus as distinct to the epoch for Sui |
| availability period | the period specified in storage epochs for which a blob is certified to be available on Walrus |
| expiry | the end epoch at which a blob is no longer available and can be deleted |
DEVNET TERMS OF SERVICE - WALRUS
-Last updated: June 13, 2024
-By using Mysten Labs Devnet software, technologies, tools, and other services (collectively -“Devnet”), you agree to the general Terms of Service and these additional Devnet Terms of Service -(together, the “Terms”). If you do not agree, do not participate in Devnet. If you are using Devnet -on behalf of an organization, you represent and warrant that you are an authorized representative of -that organization and have the authority to bind that business or entity to the Terms.
-Eligibility Criteria
-You may use Devnet only if you:
--
-
- Are 18 years or older and capable of forming a binding contract with us. -
- Are not otherwise barred from participating in Devnet under applicable law. -
We may, at our discretion, introduce new or change existing eligibility criteria or conditions we -deem appropriate. Devnet may operate in certain phases, and your participation in any one phase of -Devnet does not guarantee that you will be selected for any other phases of Devnet.
-Duration
-Devnet will commence on the date we prescribe and continue until terminated at our discretion. We -may change, discontinue, or wipe, temporarily or permanently, all or any part of Devnet, at any -time and without notice at our discretion, including, without limitation, the modification of the -presence, amounts, or any other conditions applicable to data you have stored within Devnet, without -any liability to you or other Devnet users.
-No Warranty
-Mysten Labs provides the Devnet platform solely as a developer preview. Devnet is provided "as is" -and "with all faults." We make no warranties, express or implied, regarding the reliability, -accuracy, performance, or fitness for a particular purpose of the service provided. You accept all -risks associated with the use of Devnet and agree that Mysten Labs, its affiliates, and its -employees shall not be liable for any damages, whether direct, indirect, incidental, special, -consequential, or punitive, arising out of the use or inability to use the service, including but -not limited to lost profits, loss of business, or data loss.
-No employee or representative of Mysten Labs is authorized to make any warranties or representations -beyond those stated in this agreement. Any statements made by employees or representatives of Mysten -Labs regarding the service shall not be construed as warranties or representations, and customers -agree to indemnify and hold harmless Mysten Labs from any such statements.
-Any deficiencies or errors in the Devnet platform shall not constitute a breach of this agreement, -and customers agree to waive any right to seek a refund or compensation based on such deficiencies -or errors. This "as is, no warranty" provision shall survive the termination or expiration of any -other agreements between you and Mysten Labs.
- -
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/usage/client-cli.html b/pr-preview/pr-70/usage/client-cli.html
deleted file mode 100644
index 48439303..00000000
--- a/pr-preview/pr-70/usage/client-cli.html
+++ /dev/null
@@ -1,296 +0,0 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- DEVNET TERMS OF SERVICE - WALRUS
-Last updated: June 13, 2024
-By using Mysten Labs Devnet software, technologies, tools, and other services (collectively -“Devnet”), you agree to the general Terms of Service and these additional Devnet Terms of Service -(together, the “Terms”). If you do not agree, do not participate in Devnet. If you are using Devnet -on behalf of an organization, you represent and warrant that you are an authorized representative of -that organization and have the authority to bind that business or entity to the Terms.
-Eligibility Criteria
-You may use Devnet only if you:
--
-
- Are 18 years or older and capable of forming a binding contract with us. -
- Are not otherwise barred from participating in Devnet under applicable law. -
We may, at our discretion, introduce new or change existing eligibility criteria or conditions we -deem appropriate. Devnet may operate in certain phases, and your participation in any one phase of -Devnet does not guarantee that you will be selected for any other phases of Devnet.
-Duration
-Devnet will commence on the date we prescribe and continue until terminated at our discretion. We -may change, discontinue, or wipe, temporarily or permanently, all or any part of Devnet, at any -time and without notice at our discretion, including, without limitation, the modification of the -presence, amounts, or any other conditions applicable to data you have stored within Devnet, without -any liability to you or other Devnet users.
-No Warranty
-Mysten Labs provides the Devnet platform solely as a developer preview. Devnet is provided "as is" -and "with all faults." We make no warranties, express or implied, regarding the reliability, -accuracy, performance, or fitness for a particular purpose of the service provided. You accept all -risks associated with the use of Devnet and agree that Mysten Labs, its affiliates, and its -employees shall not be liable for any damages, whether direct, indirect, incidental, special, -consequential, or punitive, arising out of the use or inability to use the service, including but -not limited to lost profits, loss of business, or data loss.
-No employee or representative of Mysten Labs is authorized to make any warranties or representations -beyond those stated in this agreement. Any statements made by employees or representatives of Mysten -Labs regarding the service shall not be construed as warranties or representations, and customers -agree to indemnify and hold harmless Mysten Labs from any such statements.
-Any deficiencies or errors in the Devnet platform shall not constitute a breach of this agreement, -and customers agree to waive any right to seek a refund or compensation based on such deficiencies -or errors. This "as is, no warranty" provision shall survive the termination or expiration of any -other agreements between you and Mysten Labs.
- -
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/usage/examples.html b/pr-preview/pr-70/usage/examples.html
deleted file mode 100644
index 34430202..00000000
--- a/pr-preview/pr-70/usage/examples.html
+++ /dev/null
@@ -1,258 +0,0 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- Using the Walrus client
-The walrus binary can be used to interact with Walrus as a client. See the setup
-chapter for prerequisites, installation, and configuration.
Detailed usage information is available through
-walrus --help
-Each sub-command of walrus can also be called with --help to print its specific
-arguments and their meaning.
Walrus system information
-Information about the Walrus system is available through the walrus info command. For example,
$ walrus info
-
-Walrus system information
-
-Storage nodes
-Number of nodes: 10
-Number of shards: 270
-
-Blob size
-Maximum blob size: 957 MiB (1,003,471,920 B)
-
-Approximate storage prices per epoch
-Price per encoded storage unit: 50 MIST/KiB
-Price to store metadata: 850 MIST
-Marginal price per additional 1 MiB (w/o metadata): 239,250 MIST
-Total price per max blob (957 MiB): 0.227 SUI
-gives an overview of the number of storage nodes and shards in the system, the maximum blob size, -and the current cost in (testnet) Sui for storing blobs.
-Additional information such as encoding parameters and sizes, BFT system information, and
-information on the storage nodes and their shard distribution can be viewed with the --dev
-argument: walrus info --dev.
Storing, querying status and reading blobs
-Storing blobs on Walrus can be achieved through the following commands:
-walrus store <some file>
-The store command takes a CLI argument --epochs <EPOCHS> (or -e) indicating the number of
-epochs the blob should be stored for. This defaults to 1 epoch, namely the current one. If the blob
-is already stored on Walrus for a sufficient number of epochs the command does not store it again.
However, this behavior can be overwritten with the --force (or -f) CLI option, which stores
-the blob again and creates a fresh blob object on Sui belonging to the wallet address.
The status of a blob by blob ID can be queried using the command:
-walrus blob-status --blob-id <BLOB_ID>
-This returns whether the blob is stored and its availability period. You can also query the -status of a blob by using a file that stores it, as:
-walrus blob-status --file <FILE>
-This command re-encodes the content of the file, derives the blob ID and returns whether it -is stored on Walrus and its availability period.
-When the blob is available the blob-status command also returns the BlobCertified Sui event ID,
-which consists of a transaction ID and a sequence number in the events emitted by the transaction.
-The existence of this event certifies the availability of the blob.
Reading blobs from Walrus can be achieved through the following command:
-walrus read <some blob ID>
-By default the blob data is written to the standard output. The --out <OUT> CLI option (or -o)
-can be used to specify and output file name. The --rpc-url <URL> (or -r) may be used to specify
-a Sui RPC node to use instead of the one set in the wallet configuration or the default one.
Changing the default configuration
-Use the --config option to specify a custom path to the
-configuration location.
The
---wallet <WALLET> argument may be used to specify a non standard Sui wallet configuration file.
-And a --gas-budget <GAS_BUDGET> argument may be used to change the maximum amount of Sui (in MIST)
-that the command is allowed to use.
Troubleshooting
-If you get an error like "the specified Walrus system object does not exist", make sure your wallet -is set up for Sui testnet and you use the latest configuration.
- -
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/usage/interacting.html b/pr-preview/pr-70/usage/interacting.html
deleted file mode 100644
index acafbb3e..00000000
--- a/pr-preview/pr-70/usage/interacting.html
+++ /dev/null
@@ -1,234 +0,0 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- Examples
-As inspiration, we provide several simple examples in different programming languages to interact -with Walrus through the various interfaces. They are located at -https://github.com/MystenLabs/walrus-docs/tree/main/examples and described below.
-In addition, we have built actual applications on top of Walrus. The prime example is Walrus -Sites, with code available in the -github.com/MystenLabs/walrus-sites repository.
-And for an example of how to build a static website and store it as a Walrus Site with GitHub -actions, just look at the CI -workflow we use -to publish this very site.
-Python
-The Python examples folder -contains a number of examples:
--
-
- How to use the HTTP -API -to store and read a blob. -
- How to use the JSON -API -to store, read, and check the availability of a blob. Checking the certification of a blob -illustrates reading the Blob Sui object that certifies (see the Walrus Sui -reference). -
- How to read information from the Walrus system -object. -
- How to track Walrus related -Events. -
JavaScript
-A JavaScript example is -provided showing how to upload and download a blob through a web form using the HTTP API.
- -
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/usage/json-api.html b/pr-preview/pr-70/usage/json-api.html
deleted file mode 100644
index e0f1b34a..00000000
--- a/pr-preview/pr-70/usage/json-api.html
+++ /dev/null
@@ -1,259 +0,0 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- Interacting with Walrus
-We provide 3 ways to interact directly with the Walrus storage system:
--
-
- Through the Walrus client command line interface (CLI). -
- Through a JSON API of the Walrus CLI. -
- Through an HTTP API exposed by a public or local Walrus client daemon. -
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/usage/setup.html b/pr-preview/pr-70/usage/setup.html
deleted file mode 100644
index 050c1835..00000000
--- a/pr-preview/pr-70/usage/setup.html
+++ /dev/null
@@ -1,368 +0,0 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- JSON mode
-All Walrus client commands (except, currently, the info command) are also available in JSON mode.
-In this mode, all the command line flags of the original CLI command can be specified in JSON
-format. The JSON mode therefore simplifies programmatic access to the CLI.
For example, to store a blob, run:
-walrus json \
- '{
- "config": "path/to/client_config.yaml",
- "command": {
- "store": {
- "file": "README.md"
- }
- }
- }'
-Or, to read a blob knowing the blob ID:
-walrus json \
- '{
- "config": "path/to/client_config.yaml",
- "command": {
- "read": {
- "blobId": "4BKcDC0Ih5RJ8R0tFMz3MZVNZV8b2goT6_JiEEwNHQo"
- }
- }
- }'
-All options, default values, and commands are equal to those of the "standard" CLI mode, except that -they are written in "camelCase" instead of "kebab-case".
-The json command also accepts input from stdin.
The output of a json command will itself be JSON-formatted, again to simplify parsing the results
-in a programmatic way. For example, the JSON output can be piped to the jq command for parsing and
-manually extracting relevant fields.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/usage/web-api.html b/pr-preview/pr-70/usage/web-api.html
deleted file mode 100644
index b3d827b2..00000000
--- a/pr-preview/pr-70/usage/web-api.html
+++ /dev/null
@@ -1,323 +0,0 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- Setup
-At this stage of the project, our Walrus code is not yet public. Instead, we provide a pre-compiled
-walrus client binary for macOS (Intel and Apple CPUs) and Ubuntu, which supports different usage
-patterns (see the next chapter). This chapter describes the
-prerequisites, installation, and configuration
-of the Walrus client.
Note that our Walrus devnet uses Sui testnet for coordination.
-Prerequisites
-Interacting with Walrus requires a valid Sui testnet wallet with some amount of SUI tokens. The -easiest way to set this up is via the Sui CLI; see the installation -instructions in the Sui -documentation.
-After installing the Sui CLI, you need to set up a testnet wallet by running sui client, which
-prompts you to set up a new configuration. Make sure to point it to Sui testnet, you can use the
-full node at https://fullnode.testnet.sui.io:443 for this. See
-here for further details.
If you already have a Sui wallet configured, you can directly set up the testnet environment (if you -don't have it yet),
-sui client new-env --alias testnet --rpc https://fullnode.testnet.sui.io:443
-and switch the active environment to it:
-sui client switch --env testnet
-After this, you should get something like this (everything besides the "testnet" line is optional):
-$ sui client envs
-╭──────────┬─────────────────────────────────────┬────────╮
-│ alias │ url │ active │
-├──────────┼─────────────────────────────────────┼────────┤
-│ devnet │ https://fullnode.devnet.sui.io:443 │ │
-│ localnet │ http://127.0.0.1:9000 │ │
-│ testnet │ https://fullnode.testnet.sui.io:443 │ * │
-│ mainnet │ https://fullnode.mainnet.sui.io:443 │ │
-╰──────────┴─────────────────────────────────────┴────────╯
-Finally, make sure you have at least 2 separate gas coins, with at least 1 SUI each. You can obtain -these coins from the testnet faucet:
-sui client faucet && sui client faucet
-After some seconds, you should see your new SUI coins:
-$ sui client gas
-╭─────────────────┬────────────────────┬──────────────────╮
-│ gasCoinId │ mistBalance (MIST) │ suiBalance (SUI) │
-├─────────────────┼────────────────────┼──────────────────┤
-│ 0x65dca966dc... │ 1000000000 │ 1.00 │
-│ 0xb07a091c1f... │ 1000000000 │ 1.00 │
-╰─────────────────┴────────────────────┴──────────────────╯
-The system-wide wallet will be used by Walrus if no other path is specified. If you want to use a -different Sui wallet, you can specify this in the Walrus configuration file or -when running the CLI.
-Installation
-We currently provide the walrus client binary for macOS (Intel and Apple CPUs) and Ubuntu:
| OS | CPU | Architecture |
|---|---|---|
| MacOS | Apple Silicon | macos-arm64 |
| MacOS | Intel 64bit | macos-x86_64 |
| Ubuntu | Intel 64bit | ubuntu-x86_64 |
You can download the latest build from our Google Cloud Storage (GCS) bucket (correctly setting the
-$SYSTEM variable) and move it to a directory included in your $PATH:
SYSTEM=ubuntu-x86_64 # or macos-x86_64 or macos-arm64
-curl https://storage.googleapis.com/mysten-walrus-binaries/latest/walrus-latest-$SYSTEM -o walrus
-chmod +x walrus
-mv walrus ~/.local/bin
-Once this is done, you should be able to simply type walrus in your terminal. For example you can
-get usage instructions (see the next chapter for further details):
$ walrus --help
-Walrus client
-
-Usage: walrus [OPTIONS] <COMMAND>
-
-Commands:
-⋮
-Custom path (optional)
-Instead of ~/.local/bin, you can place the binary in any other directory you like. You need to
-either make sure to add that directory to your $PATH or always call the binary as
-/full/path/to/walrus.
Previous versions (optional)
-In addition to the latest version of the walrus binary, the GCS bucket also contains previous
-versions. An overview in XML format is available at
-https://storage.googleapis.com/mysten-walrus-binaries/.
Configuration
-A single parameter is required to configure Walrus, namely the ID of the system -object on Sui. You can create your client -configuration as follows:
- -mkdir ~/.walrus
-curl https://storage.googleapis.com/mysten-walrus-binaries/walrus-configs/client_config.yaml \
- -o ~/.walrus/client_config.yaml
-Custom path (optional)
-By default, the Walrus client will look for the client_config.yaml configuration file in the
-current directory or in ~/.walrus/, but you can place the file anywhere and name it anything you
-like; in this case you need to use the --config option when running the walrus binary.
Advanced configuration (optional)
-The configuration file currently supports the following parameters:
-# This is the only mandatory field. The system object is specific for a particular Walrus
-# deployment.
-#
-# NOTE: THE VALUE INCLUDED HERE IS AN EXAMPLE VALUE.
-# You can get the object ID for the current Walrus devnet deployment as described above.
-system_object: 0x3243....
-
-# You can define a custom path to your Sui wallet configuration here. If this is unset or `null`,
-# the wallet is configured from `./sui_config.yaml` (relative to your current working directory), or
-# the system-wide wallet at `~/.sui/sui_config/client.yaml` in this order.
-wallet_config: null
-
-# The following parameters can be used to tune the networking behavior of the client. There is no
-# risk in playing around with these values. In the worst case, you may not be able to store/read
-# blob due to timeouts or other networking errors.
-communication_config:
- max_concurrent_writes: null
- max_concurrent_sliver_reads: null
- max_concurrent_metadata_reads: 3
- max_concurrent_status_reads: null
- reqwest_config:
- total_timeout:
- secs: 180
- nanos: 0
- pool_idle_timeout: null
- http2_keep_alive_timeout:
- secs: 5
- nanos: 0
- http2_keep_alive_interval:
- secs: 30
- nanos: 0
- http2_keep_alive_while_idle: true
- request_rate_config:
- max_node_connections: 10
- max_retries: 5
- min_backoff:
- secs: 2
- nanos: 0
- max_backoff:
- secs: 60
- nanos: 0
-Important: If you specify a wallet path, make sure your wallet is set up for Sui testnet.
- -
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/walrus-sites/intro.html b/pr-preview/pr-70/walrus-sites/intro.html
deleted file mode 100644
index 81e0e856..00000000
--- a/pr-preview/pr-70/walrus-sites/intro.html
+++ /dev/null
@@ -1,286 +0,0 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- Client Daemon mode & HTTP API
-In addition to the CLI and JSON modes, the Walrus client offers a daemon mode. In this mode, it -runs a simple web server offering HTTP interfaces to store and read blobs in an aggregator and -publisher role respectively. We also offer -public aggregator and publisher services to try the Walrus HTTP APIs without -the need to run a local client.
-Starting the daemon locally
-You can run the daemon with the following command, to offer both an aggregator and publisher on -the same address and port:
-ADDRESS="127.0.0.1:31415" # bind the daemon to localhost and port 31415 for both
-walrus daemon -b $ADDRESS # run a daemon combining an aggregator and a publisher
-Or you may run two separate processes. One for the aggregator:
-AGG_ADDRESS="127.0.0.1:31415" # Aggregator only port
-walrus aggregator -b $AGG_ADDRESS # run an aggregator to read blobs
-And a different one for the publisher:
-PUB_ADDRESS="127.0.0.1:31416" # Note different port for publisher
-walrus publisher -b $PUB_ADDRESS # run a publisher to store blobs
-The aggregator provides all read APIs, the publisher all the store APIs, and the daemon provides -both. Note that the aggregator does not perform Sui on-chain actions, and therefore consumes no gas. -However, the publisher does perform actions on-chain and will consume gas. It is therefore important -to ensure only authorized parties may access it, or other measures to manage gas costs.
-Using a public aggregator or publisher
-For some use cases (e.g., a public website), or to just try out the HTTP API, a publicly accessible -aggregator and/or publisher is required. For your convenience, we provide these at the following -hosts:
--
-
- Aggregator:
https://aggregator-devnet.walrus.space
- - Publisher:
https://publisher-devnet.walrus.space
-
Our publisher is currently limiting requests to 10 MiB. If you want to upload larger files, you need -to run your own publisher or use the CLI.
-Note that the publisher consumes (testnet) Sui on the service side, and a mainnet deployment would -likely not be able to provide uncontrolled public access to publishing without requiring some -authentication and compensation for the Sui used.
-HTTP API Usage
-Store
-You can interact with the daemon through simple HTTP PUT requests. For example, with -cURL, you can store blobs using a publisher or daemon as follows:
-ADDRESS="127.0.0.1:31415"
-curl -X PUT "http://$ADDRESS/v1/store" -d "some string" # store the string `some string` for 1 storage epoch
-curl -X PUT "http://$ADDRESS/v1/store?epochs=5" --upload-file "some/file" # store file `some/file` for 5 storage epochs
-The store HTTP API end points return information about the blob stored in JSON format. When a blob
-is stored for the first time, a newlyCreated field contains information about the
-new blob:
$ curl -X PUT "http://ord-dnt-sto-00.devnet.sui.io:9000/v1/store" -d "some other string"
-{
- "newlyCreated":{
- "id":"0xa74d376bb6923c4b8d73825fce3b798524e2bda34d02dae64ab5865556c54000",
- "storedEpoch":0,
- "blobId":"gsYzDXsK326Wihzt3X5evZCPFgaNZrb84zCjodLJaVs",
- "size":36,
- "erasureCodeType":"RedStuff",
- "certified":0,
- "storage":{
- "id":"0x2479b3096efa58fa9bfbebe805268746c235110ee0713e5a123d8f1754c2ccc3",
- "startEpoch":0,
- "endEpoch":1,
- "storageSize":4747680
- }
- }
-}
-The information returned is the content of the Sui blob object.
-When the aggregator finds a certified blob with the same blob ID and a sufficient validity period,
-it returns a alreadyCertified JSON structure:
$ curl -X PUT "http://ord-dnt-sto-00.devnet.sui.io:9000/v1/store" -d "some other string"
-{
- "alreadyCertified":{
- "blobId":"gsYzDXsK326Wihzt3X5evZCPFgaNZrb84zCjodLJaVs",
- "event":{
- "txDigest":"2oMC1dTaMGpApphFWKzARSsTM9837ox4zN8rY2RqLf6c",
- "eventSeq":"0"
- },
- "endEpoch":1
- }
-}
-The field event returns the Sui event ID that can be used to
-find the transaction that created the Sui Blob object on the Sui explorer or using a Sui SDK.
Read
-Blobs may be read from an aggregator or daemon using HTTP GET. For example the following cURL -command reads a blob and writes it to an output file:
-ADDRESS="127.0.0.1:31415"
-curl "http://$ADDRESS/v1/<some blob ID> -o <some file name>"
-Alternatively you may print the contents of a blob in the terminal with the cURL command:
-ADDRESS="127.0.0.1:31415"
-curl "http://$ADDRESS/v1/<some blob ID>
-Modern browsers will attempt to sniff the content type for such resources, and will generally do a -good job of inferring content types for media. However, the aggregator on purpose prevents such -sniffing from inferring dangerous executable types such as javascript or style sheet types.
- -
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/walrus-sites/linking.html b/pr-preview/pr-70/walrus-sites/linking.html
deleted file mode 100644
index 5e81fb03..00000000
--- a/pr-preview/pr-70/walrus-sites/linking.html
+++ /dev/null
@@ -1,256 +0,0 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- Introduction to Walrus Sites
-Walrus Sites are "web"-sites that use Sui and Walrus as their underlying technology. They are a -prime example of how Walrus can be used to build new and exciting decentralized applications. Anyone -can build and deploy a Walrus Site and make it accessible to the World! Funnily, this documentation -is itself available as a Walrus site at https://docs.walrus.site/walrus-sites/intro.html (if you -aren't there already).
-At a high level, the most exciting features include:
--
-
- Publishing a site does not require managing servers or complex configurations; just provide the -source files (produced by your favorite web framework), publish them to Walrus Sites using the -site builder tool, and you are done! -
- Sites can be linked to from ordinary Sui objects. This feature enables, for example, creating an -NFT collection in which every single NFT has a personalized website dedicated to it. -
- Walrus Sites are owned by addresses on Sui and can be exchanged, shared, and updated thanks to -Sui's flexible programming model. This means, among other things, that Walrus Sites can leverage -the SuiNS naming system to have human readable names. No more messing around -with DNS! -
- Thanks to Walrus' decentralization and extremely high data availability, there is no risk of -having your site wiped for no reason. -
- Since they live on Walrus, these sites cannot have a backend in the traditional sense, and can be -therefore considered "static" sites. However, the developer can harness Sui's programmability -to add backend functionality to Walrus Sites! -
Show me
-To give you a very high level intuition of how Walrus Sites work, let's look at an example: A simple -NFT collection on Sui that has a frontend dApp to mint the NFTs hosted on Walrus Sites, and in -which each NFT has a specific, personalized Walrus Site.
-You can check out the mint page at https://flatland.walrus.site/. This site is served to your
-browser through the Walrus Site Portal https://walrus.site. While the Portal's operation is
-explained in a later section, consider for now that there can be many Portals (hosted
-by whoever wants to have their own, and even on localhost). Further, the only function of the
-Portal is to provide the browser with some code (specifically, a service worker) that allows it to
-fetch the Walrus Site from Sui and Walrus.
If you have a Sui wallet with some Testnet SUI, you can try and "mint a new Flatlander" from the -site. This creates an NFT from the collection, and shows you two links, one to the explorer, and one -to the "Flatlander site". This latter site is a special Walrus Site that exists only for that NFT, -and has special characteristics (the background color, the image...) that are based on the contents -of the NFT.
-The URL to this per-NFT site look something like this
-https://4egmmrw9izzjn0dm2lkd3k0l8phk386z60ub1tpdc1jswbb5dr.walrus.site/. You'll notice that the
-domain remains walrus.site, but the subdomain is a long and random-looking string. This string is
-actually the Base36 encoding of the object ID of the NFT,
-which is
-0xb09b312b....
In summary:
--
-
- Walrus Sites are served through a Portal; in this case,
https://walrus.site. There can be many -Portals, and anyone can host one.
- - The subdomain on the URL points to a specific object on Sui, that allows the browser to fetch and
-render the site resources. This pointer can be:
-
-
-
- A SuiNS name, such as
flatlandinhttps://flatland.walrus.site; or
- - the Base36 encoding of a the Sui object ID, such as
0xb09b312b...in the example above.
-
- - A SuiNS name, such as
Curious to know how this magic is possible? Read the technical -overview! If you just want to get started trying Walrus Sites out, check the -tutorial.
- -
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/walrus-sites/overview.html b/pr-preview/pr-70/walrus-sites/overview.html
deleted file mode 100644
index 4b90e606..00000000
--- a/pr-preview/pr-70/walrus-sites/overview.html
+++ /dev/null
@@ -1,327 +0,0 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- Linking from and to Walrus Sites
-Links in Walrus Sites work almost as you would expect in a regular website. We specify here a few -of the details.
-Linking to resources within the same site
-Relative and absolute links (href="/path/to/resource.html") work as usual.
Linking to resources on the Web
-Linking to a resource on the web (href="https://some.cdn.example.com/stylesheet.css") also work as
-usual.
Linking to resources in other Walrus Sites
-Here is the part that is a bit different. Assume there is some image that you can browse at
-https://gallery.walrus.site/walrus_arctic.webp, and you want to link it from your own Walrus Site.
Recall that, however, https://walrus.site is just one of the possibly many Portals. I.e., the same
-resource is browsable from a local portal (http://gallery.localhost:8080/walrus_arctic.webp), or
-from any other portal (e.g., https://gallery.myotherportal.com/walrus_arctic.webp). Therefore, how
-can you link the resource in a Portal independent way? This is important for interoperability,
-availability, and respecting the user's choice of portal.
The solution: Walrus Sites links
-We solve this problem by having the Portals interpret special links, that are normally invalid on -the Web, and redirect to the corresponding Walrus Sites resource in the portal itself.
-Consider the example above, where the resource /walrus_arctic.webp is browsed from the Walrus Site
-with SuiNS name gallery, which points to the object ID abcd123… (in Base36 encoding). Then,
-the Portal-independent link is: https://gallery.suiobj/walrus_arctic.webp. To fix the object ID
-instead of the SuiNS name, you can use https://abcd123….suiobj/walrus_arctic.webp.
Another possibility is to directly point to the Walrus blob ID of the resource, and have the
-browser "sniff" the content type. This works for images, for example, but not for script or
-stylesheets. For example to point to the blob ID (e.g., containing an image) qwer5678…, use the
-URL https://blobid.walrus/qwer5678….
With such a link, the Portal will extract the blob ID and redirect the request to the aggregator it -is using to fetch blobs.
- -
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/walrus-sites/portal.html b/pr-preview/pr-70/walrus-sites/portal.html
deleted file mode 100644
index ce5b2653..00000000
--- a/pr-preview/pr-70/walrus-sites/portal.html
+++ /dev/null
@@ -1,265 +0,0 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- Technical Overview
-In the following sections, we delve deeper in the technical specification of Walrus Sites.
-High-level picture
-Walrus Sites are enabled by Sui and Walrus. The sites' resources (html, css, js, images,
-etc.) are stored on Walrus, while the main entry points to the sites are objects stored on Sui,
-which contain the metadata for the site and point to the Walrus blob IDs.
The Walrus Sites objects on Sui
-A Walrus Site is represented on Sui as a very simple object:
struct Site has key, store {
- id: UID,
- name: String,
-}
-The resources associated with this site are then added to this object as dynamic
-fields of type Resource:
struct Resource has store, drop {
- path: String,
- content_type: String,
- content_encoding: String,
- // The walrus blob id containing the bytes for this resource
- blob_id: u256,
-}
-Each resource contains:
--
-
- The
pathof the resource, for example/index.html(all the paths are always represented as -starting from root/);
- - the
content_typeof the resource, for exampletext/html;
- - the
content_encodingof the resource. At the moment the only available value isplaintext; and
- - the
blob_id, which is the Walrus blob ID where the resource can be found.
-
These Resource dynamic fields are keyed with a struct of type ResourcePath
struct ResourcePath has copy, store, drop {
- path: String,
-}
-This struct just holds the string of the path (/index.html); having a separate type ensures that
-we will not have namespace collisions with other dynamic fields, possibly added by other packages.
To see this in action, look at a Walrus Site in the -explorer, -and check its dynamic fields.
-The site rendering path
-Given the Sui object ID of a Walrus Site, then, it is easy to look up the resources that compose it
-by looking at the dynamic fields, and then fetch these resources from Walrus using the blob ID
-contained in the Resource struct.
The only outstanding question is, therefore, how to perform these lookups on the client. A few -approaches are possible:
--
-
- Having a server that accepts requests for a Sui object ID and possibly a path, and performs this -resolution on behalf of the client, serving back the resource as a standard HTML Response. -
- Using a custom application on the client, that has both a web browser and knowledge of how Walrus -Sites work, and can locally perform this resolution. -
- A hybrid approach based on service workers, where a service worker that is able to perform the -resolution is installed in the browser from a Portal. -
All of these approaches are viable (the first has been used for similar applications in IPFS -gateways, for example), and have trade offs. As an initial step, we have chosen to use the -service-worker based approach, as it is light weight and ensures that the Portal does not have to -process all the traffic from clients. In the following, therefore, we present the details of this -Portal-based approach.
-Browsing and domain isolation
-We must ensure that, when browsing multiple sites through a Portal, for example the one hosted at -https://walrus.site, these sites are isolated. Isolation is necessary for security, and to ensure -that the wallet connection in the browser works as expected.
-To do so, we give each Walrus Site a specific subdomain of the Portal's domain. For example, the
-Flatland mint dApp is hosted at https://flatland.walrus.site, where the subdomain flatland is
-uniquely associated to the object ID of the Walrus Site through SuiNS.
Walrus Sites also work without SuiNS: a site can always be browsed by using as subdomain the -Base36 encoding of the Sui object ID of the site. For the Flatland dApp, this URL is: -https://44terjw9uzwbmtful0387e2bx3k3ro64s0it82hw9x9sz4ttm.blocksite.net .
-Base36 was chosen for two reasons, forced by the subdomain standards:
--
-
- A subdomain can have at most 63, while a Hex-encoded Sui object ID requires 64. -
- A subdomain is case insensitive, ruling out other popular encodings, e.g., Base64 or Base58. -
The end-to-end resolution of a Walrus Site
-We now show in greater detail how a Walrus Site is rendered in a client's browser with the service -worker approach. The steps below all reference the following figure:
-
-
-
- Site publishing (step 0) The site developer publishes the Walrus Site using the
-
site-builder, or making use of a publisher. Assume the developer uses the -SuiNS namedapp.suito point to the object ID of the created Walrus Site.
- - Browsing starts (step 1) A client browses
dapp.walrus.site/index.htmlin their browser.
- - Service worker installation (steps 2-6) The browser connects to the Portal hosted at
-
walrus.site, which responds with a page that installs the service worker for -dapp.walrus.site. The page is refreshed to activate the service worker.
- - Site resolution (steps 7-10) The service worker, which is now installed, interprets its
-origin
dapp.walrus.site, and makes a SuiNS resolution fordapp.sui, obtaining the relative -object ID. Using the object ID, it then fetches the dynamic fields of the object (also checking -redirects). From the dynamic fields, it selects the one for/index.html, and -extracts its Walrus blob ID and content type.
- - Blob fetch (steps 11-14) Given the blob ID, the service worker queries a Walrus aggregator for -the blob. -
- Returning the response (steps 15-16) Now that the service worker has the bytes for
-
/index.html, and itscontent_type, it can craft a response that is then rendered by the -browser.
-
These steps are executed for all resources the browser may query thereafter (for example, if
-/index.html points to assets/cat.png).
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/walrus-sites/redirects.html b/pr-preview/pr-70/walrus-sites/redirects.html
deleted file mode 100644
index 550d5d3c..00000000
--- a/pr-preview/pr-70/walrus-sites/redirects.html
+++ /dev/null
@@ -1,256 +0,0 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- The Walrus Sites Portal
-We use the term "Portal" to indicate any technology that is used to access an browse Walrus Sites. -As mentioned in the overview, we foresee three kinds of -Portals:
--
-
- Server-side Portals; -
- custom local apps; and -
- service-worker based Portals in the browser. -
Currently, only the service-worker based Portal is available.
-Running the Portal locally
-You can run a service worker Portal locally:
--
-
- To browse Walrus Sites without accessing external Portals; or -
- for development purposes. -
This requires having the pnpm tool installed. To start, clone the
-walrus-sites repo and enter the portal directory. Here, run:
pnpm install
-to install the dependencies, and
-pnpm serve
-to serve the Portal. Typically, you will find it served at localhost:8080 (but check the output of
-the serve command).
Configuring the Portal
-The most important configuration parameters for the Portal are in constants.ts.
-
-
NETWORK: The Sui network to be used for fetching the Walrus Sites objects. Currently, we -use Suitestnet.
-AGGREGATOR: The URL of the aggregator from which the service worker will -fetch the Walrus blobs.
-SITE_PACKAGE: "0x514cf7ce2df33b9e2ca69e75bc9645ef38aca67b6f2852992a34e35e9f907f58"
-MAX_REDIRECT_DEPTH: The number of redirects the service worker will follow -before stopping.
-SITE_NAMES: Hard codedname: objectIDmappings, to override the SuiNS names. For development -only.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/walrus-sites/restrictions.html b/pr-preview/pr-70/walrus-sites/restrictions.html
deleted file mode 100644
index b5785f3a..00000000
--- a/pr-preview/pr-70/walrus-sites/restrictions.html
+++ /dev/null
@@ -1,264 +0,0 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- Redirecting objects to Walrus Sites
-We have seen in the overview how a Walrus Site object on Sui looks like. We will -discuss now how you can create ensure that a set of arbitrary objects can all be tied to a -specific, and possibly unique, Walrus site.
-The goal
-Consider a collection of NFTs, such as the one published by https://flatland.walrus.site. As we -show there, each minted NFT has its own Walrus Site, that can be personalized based on the contents -(e.g., the color) of the NFT itself. How can we achieve this?
-Redirect links
-The solution is simple: We add a "redirect" in the NFT's
-Display property. Each time an NFT's
-object ID is browsed through a Portal, the Portal will check the Display of the NFT and, if it
-encounters the walrus site address key, it will go fetch the Walrus Site that is at the
-corresponding object ID.
Redirects in Move
-Practically speaking, when creating the Display of the NFT, you can include the key-value pair
-that points to the Walrus site that is to beused.
...
-const VISUALIZATION_SITE: address = @0x901fb0...;
-display.add(b"walrus site address".to_string(), VISUALIZATION_SITE.to_string());
-...
-How to personalize based on the NFT?
-The code above will only open the specified Walrus Site when browsing the object ID of the NFT. How -do we ensure that the properties of the NFT can be used to personalize the site?
-This needs to be done in the VISUALIZATION_SITE: Since the subdomain is still pointing to the
-NFT's object ID, the Walrus Site that is loaded can check its origin in Javascript, and use the
-subdomain to determine the NFT, fetch it from chain, and use its internal fields to modify the
-displayed site.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/walrus-sites/site-builder.html b/pr-preview/pr-70/walrus-sites/site-builder.html
deleted file mode 100644
index 1d542495..00000000
--- a/pr-preview/pr-70/walrus-sites/site-builder.html
+++ /dev/null
@@ -1,250 +0,0 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- Known restrictions
-Walrus Sites can be used to deploy almost any form of traditional static "Web 2" website build for -modern browsers. There are, however, a number of restrictions that a developer should keep in mind -when creating or porting a website to Walrus Sites.
-No secret values
-Walrus Sites are fully publicly accessible, as the metadata is stored on Sui and the site content is -stored on Walrus. Therefore, developers must not store secret values within the sites.
-We emphasize again that any such backend-specific operations (storing secret values, authentication, -etc.) are achievable by leveraging the integration with the Sui blockchain and a Sui-compatible -wallet.
-There is a maximum redirect depth
-The number of consecutive redirects a Walrus Site can perform is capped by the -Portal (see Portal configuration). This measure ensures that loading a Walrus Site -does not result in an infinite loading loop.
-Different Portals can set this limit as they desire. The limit for the Portal hosted at -walrus.site has a maximum redirect depth of 3.
-Service workers are not available
-WARNING: This limitation only applies to Portal based on service workers. A web Portal will not -have this limitation.
-Walrus Sites leverage service workers in the clients' browsers to perform these essential -operations:
--
-
- reading the site metadata from Sui; -
- fetching the page content from Walrus; and -
- serving the content to the browser. -
Therefore, a site deployed on Walrus Sites cannot use service workers. Installing a service worker -from within a Walrus Site will result in a dysfunctional site and a poor experience for the user.
-iOS Sui Mobile Wallets do not work with Walrus Sites
-WARNING: This limitation only applies to Portal based on service workers. A web Portal will not -have this limitation.
-Service workers cannot be loaded inside an in-app browser on iOS, because of a limitation of the -WebKit engine. As a consequence, Walrus Sites cannot be used within Sui-compatible wallet apps on -iOS. Therefore, Sui wallets cannot currently be used on a Walrus Site on iOS. Note, however, that -browsing a Walrus Site is still possible on iOS through any browser. Only the connection to the -wallet is impacted.
-The connection with the Sui Wallet apps works on Android devices.
- -
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/walrus-sites/tutorial-config.html b/pr-preview/pr-70/walrus-sites/tutorial-config.html
deleted file mode 100644
index fb5a28a4..00000000
--- a/pr-preview/pr-70/walrus-sites/tutorial-config.html
+++ /dev/null
@@ -1,263 +0,0 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- The site builder
-To facilitate the creation of Walrus Sites, we provide the "site builder" tool. The site builder -takes care of creating Walrus Sites object on Sui, with the correct structure, and stores the site -resources to Walrus.
-Site builder commands
-The site builder tool exposes the following commands:
--
-
publish: Allows to publish a specific directory to Walrus. The directory must contain files that -can be served with HTTP, and have anindex.htmlfile. This command will return a Sui object ID -for the newly created site.
-update: After creating a site, you can update it with this command. It takes as input a -directory, as above, with the new or updated files, and the object ID of the site to update. Note -that the wallet you are using must be the owner of the Walrus Site object to be able to update -it. This command will remove and create resources as required, to ensure that the Walrus Sites -object on Sui matches the local directory. If run with--watch, this command re-updates the site -every time a file in the directory changes. This is useful during development, but pay attention -to costs!
-convert: A utility tool. Given a Sui object ID in Hex format, it converts it to Base36. This is -useful if you know the Sui object ID of a site, and want to find the URL.
-sitemap: A utility tool. For a give Walrus Sites Sui object ID, prints out all the resources -that compose the site and their object ID.
-
Check the commands' --help for more information.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/walrus-sites/tutorial-install.html b/pr-preview/pr-70/walrus-sites/tutorial-install.html
deleted file mode 100644
index 45acf67c..00000000
--- a/pr-preview/pr-70/walrus-sites/tutorial-install.html
+++ /dev/null
@@ -1,261 +0,0 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- Configuring the site builder
-Configuring the site-builder tool is straightforward, but care is required to ensure that
-everything works correctly.
The site-builder tool requires a configuration file to know which package to use on Sui, which
-wallet to use, the gas budget, and other operational details. Most of these are abstracted away
-through sensible defaults, so you should not need to touch them. Yet, for completeness, we provide
-here the details for all the configuration options.
Minimal configuration
-The config file is expected to be in ./builder.yaml, and it is possible to point elsewhere with
-the --config flag. For your first run, it should be sufficient to call the site-builder with
---config assets/builder-example.yaml, which is already configured appropriately.
If, for any reason, you didn't add walrus to $PATH, make sure to configure a pointer to the
-binary, see below.
Advanced configuration
-If you want to have more control over the behavior of the site builder, you can customize the -following variables in the config file:
--
-
package: the object ID of the Walrus Sites package on Sui. This must always be specified in the -config, and is already appropriately configured inassets/example-config.yaml.
-portal: the name of the Portal through which the site will be viewed; this only affects the -output of the CLI, and nothing else (default:walrus.site). -All Walrus Sites are accessible through any Portal independent of this setting.
-general: these are general options, that can be configured both through the CLI and the config: --
-
rpc_url: The URL of the Sui RPC node to use. If not set, thesite-builderwill infer it from -the wallet.
-wallet: Pointer to the Sui wallet to be used. By default, it uses the system-wide wallet (the -one fromsui client addresses).
-walrus_binary: Pointer to thewalrusbinary. By default, this is expected to be run from -$PATH.
-walrus_config: The configuration for thewalrusclient binary, see the relevant -chapter.
-gas_budget: The maximum amount of gas to be spent for transactions (default: 500M MIST).
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/walrus-sites/tutorial-publish.html b/pr-preview/pr-70/walrus-sites/tutorial-publish.html
deleted file mode 100644
index 5085857a..00000000
--- a/pr-preview/pr-70/walrus-sites/tutorial-publish.html
+++ /dev/null
@@ -1,301 +0,0 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- Clone the repository and build the
-
-
-
-
- Installing the site builder
-We describe here the steps necessary to setup the Walrus Sites' site-builder tool, and prepare
-your environment for development.
Prerequisites
-Before you start, make sure you:
--
-
- have a recent version of Rust installed; -
- have
gitinstalled; and
- - followed all Walrus setup instructions. -
Then, follow these additional setup steps.
-Clone the repository and build the site-builder tool
-First clone and enter the Walrus Sites repo from https://github.com/MystenLabs/walrus-sites.
-git clone git@github.com:MystenLabs/walrus-sites.git
-cd walrus-sites
-cd site-builder
-Then, build the release version of the site builder:
-cargo build --release
-After the build process completes, you are ready to run the site builder:
-$ ./target/release/site-builder
-Usage: site-builder [OPTIONS] <COMMAND>
-
-Commands:
- publish Publish a new site on Sui
- update Update an existing site
- convert Convert an object ID in hex format to the equivalent Base36
- format
- sitemap Show the pages composing the Walrus site at the given object ID
- help Print this message or the help of the given subcommand(s)
-
-⋮
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/walrus-sites/tutorial-suins.html b/pr-preview/pr-70/walrus-sites/tutorial-suins.html
deleted file mode 100644
index 735c809d..00000000
--- a/pr-preview/pr-70/walrus-sites/tutorial-suins.html
+++ /dev/null
@@ -1,279 +0,0 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- Publishing a Walrus Site
-Now that everything is installed and configured, you should be able to start publishing -your first Walrus Site!
-Select the source material for the site
-The site-builder works by uploading a directory of files produced by any web framework to Walrus,
-and adding the relevant metadata to Sui. This directory should have a file called index.html in
-its root, which will be the entry point to the Walrus Site.
For the rest of the tutorial, we will use as an example the simple site contained in
-./examples/snake.
Publish the site
-Since we have placed the walrus binary and configuration in their default locations, publishing
-the ./examples/snake site is simple:
-
-
-
-
Ensure that you are in the
-site-builderdirectory;
- -
-
Run the publishing command:
-
-./target/release/site-builder --config assets/builder-example.yaml publish ../examples/snake -
-
The output should look like the following:
-Operations performed:
-- created resource /Oi-Regular.ttf with blob ID 2YLU3Usb-WoJAgoNSZUNAFnmyo8cfV8hJYt2YdHL2Hs
-- created resource /file.png with blob ID R584P82qm4Dn8LoQMlzkGZS9IAkU0lNZTVlruOsUyOs
-- created resource /index.html with blob ID SSzbpPfO2Tqk6xNyF1i-NG9I9CjUjuWnhUATVSs5nic
-- created resource /walrus.png with blob ID SGrrw5NQyFWtqtxzLAQ1tLpcChGc0VNbtFRhfsQPuiM
-
-Created new site: test site
-New site object ID: 0x5ac988828a0c9842d91e6d5bdd9552ec9fcdddf11c56bf82dff6d5566685a31e
-
-Browse the resulting site at: https://29gjzk8yjl1v7zm2etee1siyzaqfj9jaru5ufs6yyh1yqsgun2.walrus.site
-This output tells you that, for each file in the folder, a new Walrus blob was created, and the -respective blob ID. Further it prints the object ID of the Walrus Site object on Sui (so you can -have a look in the explorer, and use it to set the SuiNS name), and, finally, the URL at which you -can browse the site.
-Note here that we are passing the example config assets/builder-example.yaml as the config for the
-site builder. The configuration file is necessary to ensure that the site-builder knows the
-correct Sui package for the Walrus Sites logic.
More details on the configuration of the site-builder can be found under the advanced
-configuration section.
Update the site
-Let's say now you want to update the content of the site, for example by changing the title from -"eat all the blobs!" to "Glob all the Blobs!".
-First, make this edit on in the ../examples/snake/index.html file.
Then, you can update the existing site by running the update command, and providing the directory
-where to find the updated files (still ../example/snake), and the object ID of the existing site
-(0x5ac988...):
./target/release/site-builder --config assets/builder-example.yaml update ../examples/snake 0x5ac9888...
-The output this time should be:
-Operations performed:
- - deleted resource /index.html with blob ID SSzbpPfO2Tqk6xNyF1i-NG9I9CjUjuWnhUATVSs5nic
- - created resource /index.html with blob ID LXtY0VdY5kM-3Ph7gLvj8URdz5yiRa5DUy3ZxYqDView
-
-Updated site at object ID: 0x5ac988828a0c9842d91e6d5bdd9552ec9fcdddf11c56bf82dff6d5566685a31e
-
-Browse the resulting site at: https://29gjzk8yjl1v7zm2etee1siyzaqfj9jaru5ufs6yyh1yqsgun2.walrus.site
-Compared to the publish action, we can see that now the only actions performed were to delete the
-old index.html, and update it with the newer one.
Browsing to the provided URL should reflect the change. You've updated the site!
-Additional commands
-The site-builder tool provides two additional utilities:
-
-
- the
convertcommand, which converts an object ID in hex format to the equivalent Base36 -format. This command is useful if you have the Sui object ID of a Walrus Site, and want to know -the subdomain where you can browse it.
- - the
sitemapcommand, which shows the resources that compose the Walrus Site at the given object -ID.
-
In general, the --help flag is your friend!
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
diff --git a/pr-preview/pr-70/walrus-sites/tutorial.html b/pr-preview/pr-70/walrus-sites/tutorial.html
deleted file mode 100644
index df34d12f..00000000
--- a/pr-preview/pr-70/walrus-sites/tutorial.html
+++ /dev/null
@@ -1,230 +0,0 @@
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- Bonus: Set a SuiNS name
-Browsing a URL like https://29gjzk8yjl1v7zm2etee1siyzaqfj9jaru5ufs6yyh1yqsgun2.walrus.site is not
-particularly nice. Therefore, Walrus Sites allows to use SuiNS names (this is like DNS for Sui) to
-give human-readable names to site. To do so, you simply have to get a SuiNS name you like, and point
-it to the object ID of the Walrus Site (as provided by the publish or update commands).
Let's do this step by step.
-Get a SuiNS name
--
-
- Navigate to https://testnet.suins.io/, and buy a domain name with
-your testnet wallet. For example,
walrusgame(NOTE: this is already taken, choose another you -like!). NOTE: At the moment, you can only select names that are composed of lettersa-zand -numbers0-9, but no special characters (e.g.,-).
- - In the page listing the domains you own, you should -see the newly-bought name. -
- Click the three-dots menu on the top-right corner of the name you want to assign. Choose "View all
-info", and copy the
ObjectID. In our case, this is -0x6412c4cfbe50e219c2d4d30108d7321d064e15bf64e752307100bff5eb91da38.
-
Send the SuiNS registration object to the address you use with the Sui CLI
-The steps that follow require that the SuiNS registration object is owned by the address you are -using on the Sui CLI. Therefore, we need to send this registration object from the address you use -in your browser wallet, to the address of your Sui CLI.
-To find the Sui CLI address, execute:
-sui client active-address
-Then, from your browser wallet, select the "Assets" tab, and look for the NFT of the SuiNS -registration, which should look as follows:
-
Click on it, scroll down to "Send NFT", and send it to the address discovered with the command -above. Now, your Sui CLI address owns the registration NFT, and you can proceed to the next step.
-Map the SuiNS name to the Walrus Site
-This step associates the name walrusgame to the object ID of our Walrus Site. There are possibly
-many ways to achieve this, and as the SuiNS UI improves this could be done from the webapp as well.
Here, we issue a transaction using the Sui CLI that creates this mapping:
-SUINS_UTILS_PACKAGE=0x7954ae683314ec7e156acbf0c0fc964ce035fd7f456fe7576848226502cfde1b
-SUINS_CORE_OBJECT=0x300369e8909b9a6464da265b9a5a9ab6fe2158a040e84e808628cde7a07ee5a3
-MY_SUINS_REGISTRATION_OBJECT=0x6412... # adjust this to your own SuiNS object
-MY_WALRUS_SITE_OBJECT=0x5ac9... # adjust this to your Walrus Site object
-sui client call \
- --package $SUINS_UTILS_PACKAGE \
- --module direct_setup \
- --function set_target_address \
- --gas-budget 500000000 \
- --args $SUINS_CORE_OBJECT \
- --args $MY_SUINS_REGISTRATION_OBJECT \
- --args "[$MY_WALRUS_SITE_OBJECT]" \
- --args 0x6
-Note that the SuiNS package and object on testnet may change. You can
-find the latest ones by looking at the TESTNET_CONFIG in the SuiNS
-contract.
If all succeeds, we can now browse https://walrusgame.walrus.site!
- -
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- Your first Walrus Site
-This tutorial walks you through the steps necessary to publish a Walrus Site. We also provide the -instructions on how to add a SuiNS name to it for convenient browsing.
- -