Nebula's contract search engine

[eloylp]

This is a research issue for studying the addition of a generic Soroban smart contract search engine to the nebula Wizard UI. This issue does not contemplate an studied plan, but to transmit an idea that aims to generate a discussion and research.

Motivation

We would like to provide a better smart contract search and categorisation tool, that will enable Nebula users to quickly find what they are looking for in our contract catalog. Also, reducing maintenance efforts when new contracts are added to the catalog.

The Soroban smart contract platform is growing fast, approaching mainnet at the time of this write. Currently, finding smart contracts like examples or production grade ones its only practically possible by knowing the projects in advance and looking into their repositories. We want to promote the knowledge transfer by increasing the visibility of the contracts, helping the community to quickly find references. So boosting overall ecosystem growth.

UI design

The initial idea is to be a tag based search bar that an be embedded and re-used:

Notes:

• By clicking in the "code" button the contract code should be loaded in the editor.
• The source column indicates from where the contract code was indexed.
• Pagination would be present.

Introducing search tags in our contract

The very first step would be to make the contracts visible to any form of search engine. For doing that, and as many other search engines work, we need to introduce the search metadata in our contracts.

Actually there is a Soroban macro we could try to use for this called contractmeta! that would allow us to add certain tags to contracts:

contractmeta!(key="author", val="Eiger");
contractmeta!(key="tags", val="governance,quadratic,audited");
contractmeta!(key="description", val="A governance contract");

// We could also follow how other search engines work and have policies:
contractmeta!(key="index", val="false");

We could study if such contract tags could be used for categorising Soroban contracts not only in our catalog, but also in other information channels, as we will see in next sections.

Gathering contracts in an VCS

To put an example, currently Github supports a really powerful search system. Question here would be, til what point we could make use of it ? Could we query Github periodically in search of smart contracts with a query similar to this one, that has into account contractmeta! ? How effective would that be ?

https://github.com/search?q=contractmeta%21&type=code

Gathering contracts in the blockchain

Looks like the metadata stored with the previously commented contractmeta! its being saved in the custom section of the wasm file. That could mean we can reach the wasm contracts directly from the blockchain and parse their custom section, nurturing the search engine database.

Another possible idea, would be to use Quasar (our indexer) for subscribing to contract upload and indexing their metadata as they are uploaded. If that's not too annoying.

The search engine backend

Depending how far we want to go, its possible the search engine to need backend. Specially, if we want it to be able to look for contracts outside the nebula repository.

Looking for all the commented information on the different information layers needs crawlers, data de-duplication systems and more to conform a sanitised model the Nebula UI can query in an efficient way.

It would be somehow an intersection of the data coming from the different channels.

Some initial questions to answer

In order of importance.

1. Would something like this be helpful for Nebula ?
2. What would be the impact of this community wide ?
3. What about the technical aspects ? Do we consider are they viable ideas ?
4. Is it there more information channels for gathering more contracts ?
5. What are the community plans or contractmeta! ?

Final conclusion

Waiting for feedback and research ⌛

[geofmureithi]

Is it there more information channels for gathering more contracts ?

Maybe the feature you are looking for is in quasar which allows indexing of contracts.
There is an issue here:
eigerco/quasar#42

Quasar does not have a UI, so maybe Nebula can call Quasar API?

[eloylp]

Is it there more information channels for gathering more contracts ?

Maybe the feature you are looking for is in quasar which allows indexing of contracts. There is an issue here: eigerco/quasar#42

Quasar does not have a UI, so maybe Nebula can call Quasar API?

It looks promising, thanks for linking the issue ! What kind of data do you think would be fair to serve from Quasar ? i.e maybe contract wasm hashes + their respective contractmeta! tags ?

[geofmureithi]

This is what we already have:

pub struct Contract {
    #[sea_orm(primary_key, auto_increment = false)]
    pub address: String,
    #[sea_orm(unique)]
    pub hash: String,
    pub key: String,
    pub r#type: String,
    pub last_modified: i32,
    pub created_at: DateTimeWithTimeZone,
}

We could just add a metadata field of type JSON.

[geofmureithi]

From docs:

Adds a serialized SCMetaEntry::SCMetaV0 to the WASM contracts custom section under the section name ‘contractmetav0’. Contract developers can use this to append metadata to their contract.

This should be straight forward in the ingestion process.

[eloylp]

Thanks for the information @geofmureithi , i have updated the description with an initial UI design mock based on the provided information.

[geofmureithi]

Is it there more information channels for gathering more contracts ?

I have been thinking about this and am curious if we are open to reversing the roles (make people bring contracts to us - think crates.io).
This would allow someone to submit their contract eg via PR which adds a markdown with something like frontmatter.

This would help us with several things:

• No limit on the contract channel source e it can be github, gitlab, oci, testnet, custom network etc.
• It would allow some quality control, we can signal high quality contracts.
• No extra logic to index or crawl.

I am just thinking out loud here, and I may be wrong. I am also not sure we have enough community engagement to go this direction.

[eloylp]

enough community engagement to go this direction.

IMO, you just 🎯 . What would be the motivation for a team to make a PR (and keep it up to date) to our repo ? There should be something here, like a free 🍬 offer.

The same happens, with current propsal (contractmeta! and tags addition). But this one seems probably simpler to implement for devs, and i think contractmeta! is already encouraged to use in contracts.