Author: Alex Liu, Foresight News

The decentralized storage network Arweave has launched the computing layer AO, which has successfully led to the return of AR coin price, ecosystem, and popularity, making a remarkable comeback. As a general-purpose computing chain, Sui has introduced the decentralized storage network Walrus. What kind of waves will it create?

Background Introduction

Team

The development company behind Solana is called Solana Labs, the one behind Aptos is Aptos Labs, and the one behind Sui is Mysten Labs (they are quite unique). Most of the founders and employees of Mysten Labs come from the blockchain project Diem, which was dissolved by Facebook (now Meta).

Walrus is the latest product classified as "protocol, platform" by Mysten Labs, and it is a decentralized storage network. The name "Walrus" in English means "walrus," and its official website conveys the reliability and availability of the protocol as a storage system with slogans like "thriving like a walrus" and "adaptable like a walrus."

Connection with Sui

Walrus is built on Sui and utilizes Sui to coordinate the sale of storage space and metadata. However, using Walrus does not require building applications or products on Sui, and the new governance token WAL will serve as a utility token, rather than SUI.

Competitive Comparison

Decentralized storage protocols are generally divided into two categories. The first category is fully replicated systems, with Filecoin and Arweave being typical representatives in this race. The main advantage of this type is the availability of complete files on storage nodes, allowing easy access and migration of files even if a storage node goes offline. This setup creates a permissionless environment because storage nodes do not need to rely on each other to recover files.

The reliability of these systems depends on the robustness of the selected storage nodes. In the classic one-third static adversary model and the assumption of an infinite pool of candidate storage nodes, achieving a security level of "twelve nines" (i.e., the probability of losing file access is less than 10^-12) requires storing over 25 copies on the network, resulting in 25 times the storage overhead. Additionally, there is the potential issue of a Sybil attack, where malicious actors can pretend to store multiple copies of a file, undermining the system's integrity.

The second type of decentralized storage service uses Reed-Solomon (RS) coding. RS coding divides files into smaller parts called slices, with each slice representing a portion of the original file. However, RS coding has its drawbacks. The encoding and decoding process relies on field operations, polynomial evaluation, and interpolation, which incur significant computational costs. These operations are practical only when the size of the field and the number of slices are relatively small, limiting the size of encoded files and the number of participating storage nodes. Furthermore, when a storage node goes offline and needs to be replaced by another node, unlike fully replicated systems, data cannot simply be copied from one node to another. RS coding systems require all existing storage nodes to send their slices to the replacement node, which then reconstructs the lost slices. However, this process results in O(|blob|) data transmission over the network. Frequent recovery operations reduce the storage savings obtained by reducing replication.

Challenges in Storage

Regardless of the replication protocol used, all existing decentralized storage systems face two additional challenges:

• Continuous challenges are needed to ensure that storage nodes retain data without discarding it. In an open decentralized system that provides storage payments, this is crucial, but the current practice limits the scalability of the system because each file requires a separate challenge.
• Storage nodes need coordination: they need to know who is in the system, which files have paid storage fees, implement participation incentive mechanisms, manage challenges, and mitigate abuse. This is why each of the above systems has implemented a custom blockchain to execute transactions and introduced cryptocurrencies outside the storage protocol.

Core Innovation

Given these challenges, what innovations does Walrus bring to decentralized storage?

In simple terms:

By adopting innovative erasure coding technology, Walrus can quickly and robustly encode unstructured data blocks into smaller shards, which are distributed and stored in a network of storage nodes. Even if up to two-thirds of the shards are lost, the original data block can be quickly reconstructed using partial shards. This is made possible with a replication factor of only 4 to 5 times, comparable to existing cloud services, and with the advantages of decentralization and broader fault tolerance.

Specifically:

Walrus has introduced RedStuff, a new 2D coding algorithm designed for Byzantine Fault Tolerance. Based on fountain codes, RedStuff combines fast operations with high reliability.

RedStuff encodes data into primary and secondary shards through simple operations, mainly XOR operations. These shards are distributed among storage nodes, with each node holding a unique combination. For different dimensions of coding, RedStuff uses different thresholds. The primary dimension uses an f+1 recovery threshold, allowing asynchronous writing, as only 2f+1 signatures are needed to prove the availability of data blocks, resulting in a replication factor of 3 times.

The secondary dimension uses a 2f+1 recovery threshold, which achieves asynchronous storage proof for the first time, while introducing only 1.5 times additional replication, resulting in a total replication factor of less than 5 times. More importantly, lost shards can be recovered based on the amount of lost data, saving bandwidth, all thanks to 2D coding.

The advantages of RedStuff include faster encoding/decoding speeds due to the use of simple XOR operations compared to RS coding. With low storage overhead, the system can scale to hundreds of nodes, with high elasticity and fault tolerance, ensuring data recovery even in the event of Byzantine failures.

As a permissionless protocol, Walrus is equipped with an efficient committee reconfiguration protocol to address the natural attrition of storage nodes, ensuring the continuous availability of data. When a new committee replaces the current one between two epochs, the reconfiguration protocol ensures that all data blocks that have exceeded the availability point (PoA) remain available. The 2D coding of RedStuff makes state migration more efficient, allowing other nodes to recover lost shards even when some nodes are unavailable.

Node 1 and Node 3 help Node 4 recover shard data.

Walrus has introduced an asynchronous challenge protocol to verify whether nodes are storing data correctly. This protocol allows for efficient storage proofs, ensuring data availability without relying on network assumptions, with costs scaling logarithmically with the number of stored files.

Walrus's economic model is based on staking, combined with reward and penalty mechanisms. The innovative storage certification mechanism reduces the cost of proving file storage, scaling logarithmically with the number of stored files.

In summary, Walrus, with the RedStuff protocol at its core, provides a scalable, resilient, and economically viable decentralized storage solution, offering high authenticity, integrity, auditability, and availability at a reasonable cost.

And all of this is thanks to Sui as the control layer for Walrus. Having a scalable, programmable, and secure infrastructure as a coordination layer allows it to focus on the core issues of decentralized storage.

Potential Airdrop

Walrus will launch its independent token WAL, with utility for staking, governance, etc. How can one receive the WAL airdrop? Referring to the method of obtaining AO, holding SUI may be one way.

Walrus is expected to launch its testnet soon, with the mainnet launch date to be determined. For now, you can visit the official documentation to learn how to use Walrus to deploy your own website.

Sources:

Walrus Whitepaper:

https://docs.walrus.site/walrus.pdf

Walrus: Decentralized Storage and DA Protocol, can build L2 and large-scale storage based on Sui:

https://foresightnews.pro/article/detail/63040

Mysten Labs researcher X thread:

https://x.com/LefKok/status/1836868240666153293

免责声明：本文章仅代表作者个人观点，不代表本平台的立场和观点。本文章仅供信息分享，不构成对任何人的任何投资建议。用户与作者之间的任何争议，与本平台无关。如网页中刊载的文章或图片涉及侵权，请提供相关的权利证明和身份证明发送邮件到support@aicoin.com，本平台相关工作人员将会进行核查。