Skip to content

Commit

Permalink
feat: add dev blog (#62)
Browse files Browse the repository at this point in the history
Closes: #56

---------

Co-authored-by: Markus Legner <[email protected]>
  • Loading branch information
giac-mysten and mlegner authored Jun 17, 2024
1 parent 6419957 commit a3dd5f7
Show file tree
Hide file tree
Showing 3 changed files with 138 additions and 0 deletions.
5 changes: 5 additions & 0 deletions docs/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,11 @@

---

- [The Walrus Dev Blog](./blog/00_intro.md)
- [Announcing Walrus](./blog/01_announcing_walrus.md)

---

# Design

- [Objectives and use-cases](./overview/objectives_use_cases.md)
Expand Down
4 changes: 4 additions & 0 deletions docs/blog/00_intro.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# The Walrus Dev Blog

We will use this section of the Walrus documentation to publish news and updates about Walrus'
development!
129 changes: 129 additions & 0 deletions docs/blog/01_announcing_walrus.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
# Announcing Walrus & Walrus Sites: A Decentralized Store and Web DApp Platform

Walrus is an innovative decentralized storage network for blockchain apps and autonomous agents. The
Walrus storage system is released today as a developer preview to Sui developers to gather
feedback. We expect a general rollout to other web3 communities very soon!

Leveraging innovations in erasure coding, Walrus enables fast and robust encoding of unstructured
data blobs into smaller slivers distributed and stored over a network of storage nodes. A subset of
slivers can be used to rapidly reconstruct the original blob reliably, even when up to two thirds of
the slivers are missing. This is possible while keeping the replication factor down to a minimal
4x-5x, similar to existing cloud-based services, but with the additional benefits of
decentralization and resilience to more widespread faults.

## The Replication Challenge

Sui is the most advanced blockchain system in relation to storage on validators, with innovations
such as a [storage fund](https://docs.sui.io/concepts/tokenomics/storage-fund) that future proofs
the cost of storing on-chain data. Nevertheless, Sui still requires complete data replication among
all validators, resulting in a replication factor of 100x or more in today's Sui mainnet. This is
necessary for replicated computing and smart contracts acting on the state of the blockchain, but is
inefficient when it comes to simply storing unstructured data blobs, such as music, video,
blockchain history, etc.

## Introducing Walrus: Efficient and Robust Decentralized Storage

To tackle the challenge of high replication costs, Mysten Labs has developed Walrus, a decentralized
storage network offering exceptional data availability and robustness with a minimal replication
factor of 4x-5x. Walrus provides two key benefits:

1. **Cost-Effective Blob Storage**: Walrus allows for the upload of gigabytes of data at a time with
minimal cost, making it an ideal solution for storing large volumes of data.\ Walrus can do this
because the data blob is transmitted only once over the network, and storage nodes only spend a
fraction of resources compared to the blob size. As a result, the more storage nodes the system
has, the fewer resources each storage node uses per blob.

1. **High Availability and Robustness**: Data stored on Walrus enjoys enhanced reliability and
availability under fault conditions. Data recovery is still possible even if two-thirds of the
storage nodes crash or come under adversarial control. Further, availability may be certified
efficiently without downloading the full blob.

Decentralized storage can take multiple forms in modern ecosystems. For instance, it offers better
guarantees for digital assets traded as NFTs. Unlike current designs that store data off-chain,
decentralized storage ensures users own the actual resource, not just metadata, mitigating risks of
data being taken down or misrepresented.

Additionally, decentralized storage is not only useful for storing data such as pictures or files
with high availability; it can also double as a low-cost data availability layer for rollups. Here,
sequencers can upload transactions on Walrus, and the rollup executor only needs to temporarily
reconstruct them for execution.

We also believe Walrus will accompany existing disaster recovery strategies for millions of
enterprise companies. Not only is Walrus low-cost, but it also provides unmatched layers of data
availability, integrity, transparency, and resilience that centralized solutions by design cannot
offer.

Walrus is powered by the Sui Network and scales horizontally to hundreds of networked decentralized
storage nodes. This should enable Walrus to offer Exabytes of storage at costs competitive with
current centralized offerings, given the higher assurance and decentralization.

## The Future of Walrus

By releasing this developer preview we hope to share some of the design decisions with the
decentralized app developer community, and gather feedback on the approach and the APIs for storing,
retrieving, and certifying blobs. In this developer preview, all storage nodes are operated by
Mysten Labs to help us understand use-cases, fix bugs, and improve the performance of the software.

Future updates to Walrus will allow for dynamically changing the set of decentralized storage nodes,
as well as changing the mapping of what slivers are managed by each storage node. The available
operations and tools will also be expanded to cover more storage related use-cases. Many of these
functions will be designed with the feedback we gather in mind.

Stay tuned for more updates on how Walrus will revolutionize data storage in the web3 ecosystem.

## What can developers build?

As part of this developer preview we provide a binary client (currently macos, ubuntu) that can be
operated from the [command line interface](https://docs.walrus.site/usage/client-cli.html), a [JSON
API](https://docs.walrus.site/usage/json-api.html), and an [HTTP
API](https://docs.walrus.site/usage/web-api.html). We also offer the community an aggregator and
publisher service, and a Devnet deployment of 10 storage nodes operated by Mysten Labs.

We hope Sui developers experiment with building applications that leverage the Walrus Decentralized
Store in a variety of ways. As examples we hope to see the community build:

- **Storage of media for NFT or dapps:** Walrus can directly store and serve media such as images,
sounds, sprites, videos, other game assets, etc. This is publicly available media that can be
accessed using HTTP requests at caches to create multimedia dapps.

- **AI related use cases:** Walrus can store clean data sets of training data, datasets with a known
and verified provenance, models weights, and proofs of correct training for AI models. Or it may
be used to store and ensure the availability and authenticity of an AI model output.

- **Storage of long term archival of blockchain history:** Walrus can be used as a lower cost
decentralized store to store blockchain history. For Sui this can include sequences of checkpoints
with all associated transaction and effects content. As well as historic snapshots of the
blockchain state, code or binaries.

- **Support availability for L2s:** Walrus allows parties to certify the availability of blobs, as
required by L2s that need data to be stored and be attested as available to all. This may also
include availability of extra audit data such as validity proofs, zero knowledge proofs of correct
execution or large fraud proofs.

- **Support a full decentralized web experience:** Walrus can host full decentralized web
experiences including all resources (such as js, css, html, media). These can provide content but
also host the UX of dapps, enabling fully decentralized front end and back ends on chain. It
brings the full "web" back in "web3".

- **Support subscription models for media:** Creators can store encrypted media on Walrus and only
provide access via decryption keys to parties that have paid a subscription fee or have paid for
contents. (Note that Walrus provides the storage, encryption and decryption must be done off
Walrus).

We are excited to see what else the web3 developer community can imagine!

## Getting Started

For this developer preview the public Walrus Devnet is openly available to all developers. Developer
documentation is available at <https://docs.walrus.site>.

SUI Testnet token is the main currency for interacting with Walrus. Developers pay for Walrus
Devnet storage using SUI Testnet tokens which can be acquired at the [Sui Testnet Discord
faucet](https://discord.com/channels/916379725201563759/1037811694564560966).

## One more thing

The [Walrus Sites website](https://walrus.site/), the [Walrus docs](https://docs.walrus.site), and
[this very blog](https://blog.walrus.site) are hosted on Walrus. to learn more about Walrus Sites
and how you can deploy your own, [click
here](https://docs.walrus.site/walrus-sites/intro.html).

0 comments on commit a3dd5f7

Please sign in to comment.