Skip to content
This repository has been archived by the owner on Nov 13, 2024. It is now read-only.

Commit

Permalink
Add onboarding docs and migrate old docs (#19)
Browse files Browse the repository at this point in the history
  • Loading branch information
nodiesBlade authored Jan 25, 2024
1 parent ef6dd3c commit 55affa4
Show file tree
Hide file tree
Showing 7 changed files with 378 additions and 34 deletions.
79 changes: 45 additions & 34 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,45 +2,31 @@
<img src="https://github.com/baaspoolsllc/os-gateway/assets/99137075/ced8035d-87da-4fd3-a51b-6c336fadc14c" width="500" alt="POKT Gateway Stack">
</div>

# POKT Gateway Stack
## What is POKT Gateway Server?

This project is currently in the alpha stage and actively under development. Before contributing, please reach out to [@nodiesBlade](https://github.com/nodiesBlade).
The POKT Gateway Server is a comprehensive solution designed to simplify the integration of applications with the POKT Network. Our goal is to reduce the complexities associated with directly interfacing with the protocol, making it accessible to a wide range of users, including application developers, existing centralized RPC platforms, and future gateway operators.

## What is POKT Gateway Stack?
Learn more about the vision and overall architecture [overview](docs%2Foverview.md)

The [POKT Gateway Stack](https://docs.nodies.app/pokt-integration-wip/nodies-gateway-stack) is a comprehensive solution designed to simplify the integration of applications with the POKT Network. Our goal is to reduce the complexities associated with directly interfacing with the protocol, making it accessible to a wide range of users, including application developers, existing centralized RPC platforms, and future gateway operators.
## New Gateway Operator Onboarding Path
1. [Overview](docs%2Foverview.md)
2. [pokt-primer.md](docs%2Fpokt-primer.md)
2. [Gateway API Endpoints](docs%2Fapi-endpoints.md)

Learn more about the vision and overall architecture [here](https://docs.nodies.app/pokt-integration-wip/nodies-gateway-stack).
## Additional Onboarding Resources
1. [Relay Specification](docs%2Fpokt-relay-specification.md)
3. [System Architecture](docs%2Fsystem-architecture.md)

## Project Structure

- **cmd:** Contains the entry point of the binaries
- **gateway_server:** HTTP Server for serving requests
- **internal:** Shared internal folder for all binaries
- **pkg:** Distributable dependencies

## Core Project Dependencies
- [FastHTTP](https://github.com/valyala/fasthttp) for both HTTP Client/Server
- [FastJSON](https://github.com/pquerna/ffjson) for performant JSON Serialization and Deserialization

## Lightweight Pocket Client

We have implemented our own lightweight Pocket client to enhance speed and efficiency. Leveraging the power of [FastHTTP](https://github.com/valyala/fasthttp) and [FastJSON](https://github.com/pquerna/ffjson), our custom client achieves remarkable performance gains. Additionally, it has the capability to properly parse node runner's POKT errors properly given that the network runs diverse POKT clients (geomesh, leanpokt, their own custom client).

### Why It's More Efficient/Faster
1. **FastHTTP:** This library is designed for high-performance scenarios, providing a faster alternative to standard HTTP clients. Its concurrency-focused design allows our Pocket client to handle multiple requests concurrently, improving overall responsiveness.
2. **FastJSON:** The use of FastJSON ensures swift and efficient JSON serialization and deserialization. This directly contributes to reduced processing times, making our Pocket client an excellent choice for high-scale web traffic.

## Local Development:
1. In order to operate the gateway server, build the project
```sh
go build cmd/gateway_server/main.go
```
2. Copy `.env.sample` over to `.env` and fill out the details
```sh
cp .env.sample .env
```
3. Run the binary `./main`
## Quick Getting Start:
1. In order to operate the gateway server, build the project
```sh
go build cmd/gateway_server/main.go
```
2. Copy `.env.sample` over to `.env` and fill out the details
```sh
cp .env.sample .env
```
3. Run the binary `./main`

## Running Tests
Before running any tests make sure to have the mock files in placed at `./mocks` folder.
Expand All @@ -56,6 +42,10 @@ Run this command to run some tests:
go test
```

## Docker Compose
There is an all-inclusive docker-compose file available for usage [docker-compose.yml](docker-compose.yml)


## Contributing Guidelines
1. Create a Github Issue on the feature/issue you're working on.
2. Fork the project
Expand All @@ -65,3 +55,24 @@ go test
4. Make sure your code lints with `go fmt ./...` (This will Lint and Prettify)
5. Commit code to your branch and issue a pull request and wait for at least one review.
- Always ensure changes are rebased on top of main branch.
---
## Project Structure
- **cmd:** Contains the entry point of the binaries
- **gateway_server:** HTTP Server for serving requests
- **internal:** Shared internal folder for all binaries
- **pkg:** Distributable dependencies
- **docs:** Project documentation and specifications
## Core Project Dependencies
- [FastHTTP](https://github.com/valyala/fasthttp) for both HTTP Client/Server
- [FastJSON](https://github.com/pquerna/ffjson) for performant JSON Serialization and Deserialization
- Lightweight Pocket Client
We have implemented our own lightweight Pocket client to enhance speed and efficiency. Leveraging the power of [FastHTTP](https://github.com/valyala/fasthttp) and [FastJSON](https://github.com/pquerna/ffjson), our custom client achieves remarkable performance gains. Additionally, it has the capability to properly parse node runner's POKT errors properly given that the network runs diverse POKT clients (geomesh, leanpokt, their own custom client).

### Why It's More Efficient/Faster
1. **FastHTTP:** This library is designed for high-performance scenarios, providing a faster alternative to standard HTTP clients. Its concurrency-focused design allows our Pocket client to handle multiple requests concurrently, improving overall responsiveness.
2. **FastJSON:** The use of FastJSON ensures swift and efficient JSON serialization and deserialization. This directly contributes to reduced processing times, making our Pocket client an excellent choice for high-scale web traffic.

15 changes: 15 additions & 0 deletions docs/api-endpoints.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# POKT Gateway Server API Endpoints

The Gateway Server currently exposes all its API endpoints in form of HTTP endpoints. Note, we can move this to Swagger in the future if our API endpoints become more complex.

`x-api-key` is an api key set by the gateway operator to transmit internal private data

| Endpoint | HTTP METHOD | Description | HEADERS | Request Parameters |
|-----------------------------|-------------|-----------------------------------------------------------------------|-------------|---------------------------------------------------------------------|
| `/relay/{chain_id}` | ANY | The main endpoint for your reverse proxy to send requests too | ANY | `{chain_id}` - Network identifier |
| `/metrics` | GET | Metadata on the gateway server performance for observability purposes | N/A | N/A |
| `/poktapp` | POST | Adds an existing app stake to the appstake database | `x-api-key` | `encrypted_private_key` - private key of app stake that's encrypted |
| `/poktapp` | DELETE | REmoves an existing app stake from the appstake database | `x-api-key` | `address` - address of the appstake that's encrypted |
| `/poktapp` | GET | A list of all the avaliable app stakes | `x-api-key` | N/A |
| `/poktapp/{app_id}` | GET | Return a specific app stake | `x-api-key` | `app_id` - internal app id |
| `/appstakes/{app_id}/chains` | POST | Edits the app stake chains | `x-api-key` | `chain_ids` - an string array of chain ids |
38 changes: 38 additions & 0 deletions docs/overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# POKT Gateway Server Overview

The Gateway server's goal is to reduce the complexity associated with directly interfacing with the protocol with an library that any developer can contribute to. The Gateway server kickstarts off as a light-weight process that enables developers of all kinds to be able to interact with the protocol and engage with 50+ blockchains without the need to store terabytes of data, require heavy computational power, or understand the POKT protocol specifications with a simple docker-compose file. The rhetorical question that we pose to future actors who want to maintain a blockchain node is: Why spin up an Ethereum node and maintain it yourself whenever you can just leverage POKT natively using the Gateway server? After all, using POKT would require a fraction of the required resources and technical staffing.

## Features
- Simple docker-compose file with minimal dependencies to spin up
- a single tenancy HTTP endpoint for each blockchain that POKT supports, abstracting POKT's Relay Specification. This endpoint's throughput should scale based on the number of app stakes the developer provides.
- QoS checks to allow for optimized latency and success rates
- Provides Prometheus metrics for success, error rates, and latency for sending a relay to the network
- Custom Pocket client and web server that allows for efficient computational resources and memory footprint
- FastHTTP for optimized webserver and client
- FastJSON for efficient JSON Deserialization
- Custom Integration leveraging the two for efficient resource management
- Functionality improvement such as allowing for proper decoding of POKT Node error messages such as max evidence sealed errors.

## What's not included in the Gateway Server
- Authentication
- Rate Limiting & Multi-tenancy endpoints
- SaaS-based UX
- Reverse Proxy / Load Balancing Mechanisms
- Any other Opinionated SaaS-like design decision.

The decision to exclude certain features such as Authentication, Rate Limiting and multi-tenancy endpoints, SaaS-based UX, and Reverse Proxy/Load Balancing Mechanisms is rooted in the project's philosophy. These aspects are often regarded as opinionated web2 functionalities, and there are already numerous resources available on how to build SaaS products with various authentication mechanisms, rate-limiting strategies, and user experience design patterns.

The Gateway server aims to simplify the POKT protocol, not reinventing the wheel. Each Gateway, being a distinct entity with its unique requirements and team dynamics, is better suited to decide on these aspects independently. For instance, the choice of authentication mechanisms can vary widely between teams, ranging from widely-used services like Auth0 and Amazon Cognito to in-house authentication solutions tailored to the specific language and skill set of the development team.

By not including these opinionated web2 functionalities, the Gateway server acknowledges the diversity of preferences and needs among developers and businesses. This approach allows teams to integrate their preferred solutions seamlessly, fostering flexibility and ensuring that the Gateway server remains lightweight and adaptable to a wide range of use cases.

As the project evolves, we anticipate that individual Gateways will incorporate their implementations of these features based on their unique requirements and preferences. This decentralized approach empowers developers to make decisions that align with their specific use cases, promoting a more customized and efficient integration with the Gateway server.

## Future
We envision that the server will be used as a foundation for the entirety of the ecosystem to continue to build on top of such as:
- Building their frontends and extending their backend to include POKT by using the gateway server for their own SaaS business
- Create Demo SaaS gateways that use the gateway server as the underlying foundation.
- Using POKT as a hyper scaler whenever they need more computational power or access to more blockchains (sticking the process into their LB rotation)
- Using POKT as a backend as a failover whenever their centralized nodes go down (sticking the process into their LB rotation)

Over time, as more gateways enter the network, there will be re-occurring patterns on what is needed on the foundational level and developers can create RFPs to have them included. For example, while rate limiting and multi-tenancy endpoints feel too opinionated right now, there is a future where we can create a service that distributes these endpoints natively in the GW server. The use cases are limitless and we expect that over time, community contributions into the gateway server will enable some of the aforementioned use cases natively.
31 changes: 31 additions & 0 deletions docs/pokt-primer.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# POKT Primer

## POKT Network: A Quick Overview
1. Apps (App developers): Individuals or entities that stake into the Pocket Network and obtain access to external blockchain nodes in return.
2. Node Runners: Individuals or entities that stake into the network and provide access to external blockchain nodes, such as Ethereum & Polygon in return for $POKT.

### The Challenge: Interacting with the Protocol
For application developers, directly engaging with the POKT Network can be intimidating due to its inherent complexities. The technical barriers and protocol nuances can dissuade many from integrating and adopting the network.

**Challenges faced by App Developers using the protocol:**
1. Managing Throughput: The network supports approximately 250 app stakes and around 10B relays. With each app stake being limited to roughly 20M requests per hour, developers who surpass this need to stake multiple applications and balance the load among these stakes.
2. Determining Quality of Service (QoS): The network doesn't currently enforce QoS standards. Apps are assigned a set of pseudo-randomly selected node runners, rotated over specified intervals, also known as sessions. It falls on the application developer to implement strategies, such as filtering and predictive analysis, to select node runners that align with their criteria for reliability, availability, and data integrity.
3. Protocol Interaction: Unlike the straightforward procedure of sending requests to an HTTP JSON-RPC server, interacting with the POKT Network requires far more complexities given its blockchain nature. (i.e. signing a request for a relay proof)

### The Solution: Gateway Operators
Gateway Operators act as a conduit between app developers and the POKT Network, streamlining the process by abstracting the network's complexities. Their operations on a high level can be seen as:
1. Managing Throughput: By staking and load-balancing app stakes, Gateway Operators ensure the required throughput for smooth network interactions.
2. Determining the Quality of Service: Gateway Operators filter malicious, out-of-sync, offline, or underperforming nodes.
3. Protocol Interaction: Gateway Operators offer a seamless HTTP JSON-RPC Server interface, making it simpler for developers to send and receive requests, akin to interactions with conventional servers. Under the hood, the web server will contain the necessary business logic in order to interact with the protocol.

### Conclusion
Engaging with the POKT Network's capabilities doesn't have to be an uphill task. Thanks to Gateway Operators, app developers can concentrate on their core competencies—developing remarkable applications using a familiar HTTP interface, like traditional RPC providers—all while reaping the benefits of a decentralized RPC platform.

---

###### Footnotes:

1. _As of 9/14/2023, the app stakes are permissioned and overseen by the Pocket Network Foundation for security considerations._
2. _The amount of POKT staked into an app doesn't carry significant implications as all gateway operators are charged a fee for every request sent through an app stake._
3. _Historically, Grove (formerly known as Pocket Network Inc.) has been the sole gateway operator. This will change by 2024 Q2 as more gateway operators join the network._
4. _Our research aims to invite more gateway operators to join the network in a sustainable fashion by documenting the protocol specifications and limitations and leveraging and providing open-source software and noncloud vendor-lock-in services._
Loading

0 comments on commit 55affa4

Please sign in to comment.