Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Solana - Arweave Bridge #30

Open
cedriking opened this issue Jul 29, 2020 · 8 comments
Open

Solana - Arweave Bridge #30

cedriking opened this issue Jul 29, 2020 · 8 comments

Comments

@cedriking
Copy link
Member

Solana - Arweave Bridge

Interoperability Hack

Introduction

Arweave is a scalable and permanent data storage protocol. Solana is a high-performance blockchain boasting high throughput and sub-second block times using a network timestamp system called Proof-of-History. Assuming Solana performs at peak capacity consistently for a full year it’s expected to generate 4 petabytes of data. It’s essential that the entire ledger is permanently stored and available. This is why both the Solana and Arweave core teams are excited to launch a grant for an Arweave+Solana integration that can address this need.

Application requirements

The application shall facilitate a bridge to connect Solana ledger data to Arweave permanent storage and must fulfill the following criteria:

Solana

  1. Allow the storage of Solana ledger data in Arweave. Transactions should be efficiently batched into Arweave with the following tags

    • Signature: transaction signature
    • Pubkey0-N: pubkeys of the accounts in the transaction
    • Blockhash
    • Slot
    • Entry block index
    • Tx entry index

  2. If it is efficient to do so tag the following pubkey metadata as well

    • Fee Payer
    • Signer
    • Writable
    • Read-Only
    • Program
    • Leader

Application Design

  1. Standalone application that can use the Solana’s json RPC methods to retrieve the block data, (see docs.solana.com)

  2. Data should be packed to minimize costs in AR, within reason.

  3. Demo should include examples to query the Arweave chain for the stored transactions by tag
    Example, Pubkey=2ojv9BAiHUrvsm9gxDe7fJSzbNZSJcxZvf8dqmWGHG8S should paginate through the transactions for that key.

Nice to haves (all 3 add up to a 10% grant bonus)

  1. Application should automatically start and stop based on the user providing AR for funding.
  2. Application should be able to recover its position from a cold restart.
  3. Integration that can connect it to our explorer "fetch more data" button. https://explorer.solana.com/account/2ojv9BAiHUrvsm9gxDe7fJSzbNZSJcxZvf8dqmWGHG8S

Rules

Successful submissions will meet the following criteria:

  1. Submissions must be your own original work.
  2. Apps must be open source, with the full source code available on Github or another open code hosting repository. Feel free to use whatever OSS license you prefer.
  3. All submissions must include a brief description of the application and functionality in the GitHub repo
  4. All applicants must join the Solana Discord server and share your submission in the #arweave-bridge channel upon completion

Resources

Need to learn more about the Solana or Arweave technology? Use the resources below, or reach out to the relative support teams on the shared discord channel:

Solana Discord

Arweave Dev Discord

Solana

Access Solana's Web3 SDK

Polling for Blocks Guide

Developer Documentation

Arweave

Get an Arweave Wallet & Free Tokens

Arweave Web Extension Wallet

Developer Documentation & Getting Started Guides

Submission Deadline

The hackathon will take place from July 29th to August 28th. The submission deadline is August 28th @ 23:59 PM PST.
@gitcoinbot
Copy link

gitcoinbot commented Jul 29, 2020 •
edited
Loading

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

Work has been started.

These users each claimed they can complete the work by 1 week, 6 days ago.
Please review their action plans below:

1) developerfred has been approved to start work.

I am the right guy to build bridges, I will read all the documentation and already add this technology in tomato os.
2) zyfrank has applied to start work (Funders only: approve worker | reject worker).

I'm familiar with Arweave, and would like to take this task
3) theloneronin has been approved to start work.

Hi friends at Solana and Arweave.

I've been investigating the problem and have created a proposal spec at this repo: https://github.com/TheLoneRonin/SolarweaveBridge

I would love feedback on how my proposal spec is structured. Once approved to work on the task. I can work on it immediately and ideally have a prototype completed expediently that has unit and spec testing done in Travis CI.

~ The Lone Ronin
4) evgeniuz has applied to start work (Funders only: approve worker | reject worker).

Would like to work on this project, data structures and efficiency is really interesting to develop.

I'll write a tool that will collect Solana data, pre-process it and then efficiently store into Arweave. So far general plan is to use Go as a programming language (it's reasonably fast, already has integration library for Arweave, will produce static binary, which is easier for deployment), store ledger data by serializing it with protobufs (which are efficient and fast), compressing serialized data with gzip or similar. Those are just preliminary details, I'll run some tests on actual data to see if serialization/compression fit.
5) princesinha19 has applied to start work (Funders only: approve worker | reject worker).

We will create a bridge using Rust for persistent storage of Solana Transaction data to Arweave.

Our Steps will include:-

  1. Creating a Rust Client for Arweave
  2. Fetching every single tx of Solana.
  3. Extracting parameters such as (Slot, PubKeys, etc.) and, adding them as the Tag for Arweave tx.
  4. Storing as much as possible data in the buffer. (By considering TAG size of 2048 Bytes).
  5. Sending the Arweave Transaction and creating a new buffer to process next Solana Txs.

This process will be recursive so, for each Solana tx, the above process will be repeated.

Detail proposal is provided in this link:-
https://hackmd.io/@iQyKxnQ5TYCGcPHS21XW8Q/ryCSZQFZw
6) talhasch has applied to start work (Funders only: approve worker | reject worker).

I am planning to build a distributed task pipeline to sync solana ledger to arweave.
7) arasmar1983 has applied to start work (Funders only: approve worker | reject worker).

I am very interested
So be the best for future
8) cuba420 has applied to start work (Funders only: approve worker | reject worker).

Will get it done jjuvjbknm. N nbknnbv. Bjbjhvbjjb. Ub. Vb. B bbb bbbb. N bnnbbv k
9) rodgz1622 has applied to start work (Funders only: approve worker | reject worker).

More focust on work.. Must communicate each other for the action
Plan.. Always ask to the leader if their
Is a concern about. The project.. And must have a speed solution to the issue?...
10) tamino0710 has applied to start work (Funders only: approve worker | reject worker).

Ok no that tuyet voi . Toi rat thich
11) beringwaters has applied to start work (Funders only: approve worker | reject worker).

We are Bering Waters, OTC Desk specialised in unreleased tokens and blocks of liquid alts, and Ubik Group, a software development firm that builds tech projects with a purpose. We have created a strategic joint venture for the purpose of building the bridge between Solana and Arweave. We propose to implement a high-performance standalone application with JS6/NodeJS heavily utilizing multithreading for performance optimization. This would also save cost of time to deliver due to the already existing solana-web3 package. Additionally, the app itself could be easily turned into an npm package and thus easily distributed. The implementation will aim to optimize the storage size and efficiently batch transactions with a set of tags provided in the task. Please find attached a document: "Bering Waters & Ubik Group: Bridge Solana & Arweave
https://docsend.com/view/397q8htbpeuwch2k"
12) proy24 has applied to start work (Funders only: approve worker | reject worker).

I have successfully completed several Arweave bounties and am the right person for this project
13) zoek1 has applied to start work (Funders only: approve worker | reject worker).

I reviewed the docs to get ledger data, also i previously worked on arweave bounties. So i'll implement the bridge between Arweave and Solana using nodejs.
14) 1justinbarnett has applied to start work (Funders only: approve worker | reject worker).

I have reviewed the expectations for the Solana - Arweave. If approved we will write the service in GoLang to query the ArQL to insert the transaction tags

Learn more on the Gitcoin Issue Details page.

@gitcoinbot
Copy link

@developerfred Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

  • reminder (3 days)
  • escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

@developerfred
Copy link

It will be possible to track my progress in this repository AS-bridge

@cedriking
Copy link
Member Author

It will be possible to track my progress in this repository AS-bridge

Good morning @developerfred , we are investigating why the bot is asking you if you're working on this project. We have not yet selected the person to participate on the bounty.

Please don't dedicate time to it yet, as we first need to select the participant for the bounty.

@nanspro
Copy link

nanspro commented Aug 3, 2020 •
edited
Loading

@cedriking do we have to come up with a full proposal before applying. Curious about what the criteria is to select the appropriate candidate. IMO this bounty could be changed to 'co-operative' so that multiple people can collaborate but it's up to you ofc.
I will be willing to participate in it. The steps i'll take would be:

  • Create a library in typescript which interacts with solana and it's JSON RPC methods to fetch relevant data to be stored in arweave. Also will have inbuilt compression methods

  • A nodejs server running where you can upload data using API calls and also query arweave chain with custom parameters.

I have done protocol level work in ethereum so i won't have any issues understanding solana's architecture and RPC methods. Here's the library i created for eth2.0 network testing https://github.com/nanspro/Hobbits.py
Let me know if you are looking for more details in proposal and I can come up with one asap

@zoek1
Copy link

zoek1 commented Sep 2, 2020

@cedriking any update on this? Seems the bounty has expired :P

@cedriking
Copy link
Member Author

@jespern Got an update for this bounty please? 👍

@gitcoinbot
Copy link

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

Work for 15000.0 DAI (15000.00 USD @ $1.0/DAI) has been submitted by:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@zoek1 @nanspro @gitcoinbot @cedriking @developerfred and others