[Milestone] Data Availability (Verifiable Information Dispersal + ephemeral storage)

[Cofson]

Data Availability (Verifiable Information Dispersal + ephemeral storage)

As part of our Modular architecture design, we need to solve the problem of Data Availability. Note that as opposed to durable and provable storage as Codex provides, our requirements are different:

• It is not necessary to guarantee data availability beyond a certain threshold of time (still to be determined with precision). Our Data Availability protocol is concerned with guaranteeing only a fallback solution in case Zones deny data to users or other Zones. As such, the requirements vary from protocols like Codex or Filecoin, where guarantees of data archival increase the complexity of the protocol. Durable storage and data availability are different problems.
• NomosDA also differs from other Data Availability projects such as Polygon Avail or Celestia. Specifically, we do not shape our design by the need of quick and cheap reconstruction, since this is not expected as the main Zone-to-Zone communication channel (as it is the case in Celestia). Thus, we optimize our protocol with this in mind, and allow us to get rid of super nodes. In contrast, our DA protocol works in a fully decentralized way.
• In our current NomosDA v1, we do not incorporate the altruistic mechanism of DA dissemination, which helps increase the resiliency of the protocol by leveraging altruistic participation by Light Nodes. This is left for future work, and an example of why I chose the Subproject naming, since even after successful implementation of the basic functionality of NomosDA, we will continue working on the problem of data availability.

Estimated Delivery Date

July 2024

Resources Required

• Mathematical analysis of node attrition and data decay: Alexander Mozeika
• Specification of NomosDA (cryptography and protocol): Mehmet
• Implementation of cryptographic primitives: Daniel Sánchez + Mehmet
  • At this moment I do not identify the need for any new position to be filled. However, there are some potential projects that could be developed in parallel by Vac if they have the resources:
    • Trusted setup: Leveraging existing libraries is identified as a viable approach for implementing the trusted setup mechanism. This component is critical in showcasing the project's secure implementation, highlighting its importance among various tasks.
    • Implementation of storage node signing algorithm. Beyond the Trusted Setup, it's imperative for storage nodes to accurately generate and manage private keys for the signing algorithm, alongside verifying their public keys. The secure and precise execution of this key management and verification process is fundamental. A thorough definition and review of this process, including the specifics of the BLS signature algorithm.

Deliverables

• Preliminary studies and comparison of current DA protocols and cryptographic schemes.
  • Intended audience: research team, to set a clear direction.
• Mathematical analysis:
  • Node attrition in DA. Helps us understand the durability guarantees that NomosDA will provide.
  • Intended audience: research team, to understand the limitations of current designs and ways to improve it, as well as assessing the final proposal.
• Specification of full NomosDA protocol:
  • Cryptographic scheme
  • DA protocol
  • Intended audience: engineering team, for implementation.
• Rust implementation:
  • All things in specification
  • Intended audience: node validators running reference implementation.

Tracking Metrics

In progress:

• Notion specs
• Github commits (executable specs and node implementation)
• Weekly progress reports

Finalized:

• Code deployed in Testnet

Work Breakdown

• Encoding protocol
• Base Layer node verification
• Verifiable Dispersal proof

Perceived Risks

Implementing cryptography from scratch is always risky. In this regard, I encourage that in due time we hire an auditing firm to verify our cryptographic scheme and implementation. If we believe that we can do this internally, perhaps this verification can be performed by Vac.