Example of using pgwire as a postgres proxy

[osawyerr]

Was looking at the examples and there isn't one that actually connects to a real postgres instance in the backend. For example to use pgwire as a proxy to a 'real' postgres server. i.e. maintaining different client sessions and forwarding them to a real postgres instance.

[sunng87]

hi @osawyerr , it's possible to write such a proxy but we need to define the purpose of it so we can choose at which layer we will proxy the traffic. For example, we can write a Layer-4 proxy by simply forward any tcp traffic to backend, or a Layer-7 one that understands postgresql's protocol or even sql statements.

If I'm creating a basic Layer-7 example, I will let the proxy to finish the startup and forward further query traffic to backend. I will find time for that.

[lucasyvas]

My vote is for Layer-7. Personally, I'd like to be able to fundamentally understand the request, execute the query, apply transforms to the result rows, then hand them back out.

[batmilkyway]

I'll 3rd the request for a l7 proxy example, @sunng87 . My use case is an auth n/z aware proxy that can make authorization decisions based on the query.
Even something as simple as how to connect to a backend postgres and forward/reply messages back and forth.

[sunng87]

Let me do this in weekend if everything goes well.

[batmilkyway]

Let me do this in weekend if everything goes well.

Thank you, looking forward to it!

[sunng87]

I have an unfinished proxy example that uses tokio_postgres in SimpleQueryHandler to forward requests to upstream. There is some residual work to convert data format from tokio_postgres to pgwire.

However, for a complete proxy solution, we will need to implement tokio layer for using pgwire data format as client for finer and better control over messages. That's beyond my time budget for this project. I welcome contributors to join the development if you are interested in this feature.

[adriangb]

I'll +1 this request.

My use case is to intercept every message and validate incoming queries.
I wrote a Python version of this that worked by starting two async tasks, one that listened to the server and pushed to the client and another that listened to the client and pushed to the server. Thus I didn't need any sort of state machine.

[FelixMalfait]

We also need this for https://github.com/twentyhq/twenty

I've never tried Replit bounties but thought it could be an occasion to give it a shot... Let's see if it works!
https://replit.com/bounties/@felix54/rust-postgres-proxy

[usocoder]

https://github.com/usocoder/felix/tree/main/twenty-postgres-proxy

Hey there! My name its Ritchie from New Zealand i have developed a program that acts like a security guard for a database I created a new folder called twenty postgres proxy to organize key files. The core of our solution is a Rust-based program that performs several crucial tasks. It listens for incoming database connection attempts, verifies if the user trying to connect has permission, and if authorized, restricts the user to only the sections of the database they’re allowed to access. It also forwards their queries to the actual database and returns the responses. I created a "config.json" file that allows easy customization of where the program should listen for connections, the location of the real database, and who is allowed to connect and what parts of the database they can access.
here is my email for instant contact
[email protected]

0x819ffeE0e30BBB4309282297C7F26D11a9cD3350
here is my Eth address if this is successful,
or email what payment methods you can do.

[FelixMalfait]

Thanks @usocoder - tbh I think the bounty was a mistake (apologies @sunng87 for the spam it just created), we got low quality contributions and I wouldn't do it this way again. @usocoder your code doesn't work on my side, it doesn't have any test / no readme and relies on an old version of pgwire (0.12 vs 0.25 today)

[sunng87]

@FelixMalfait no worry. I'm quite interested in your progress with pgwire on a postgres proxy. I've listed it in my projects for pgwire #204 . I just don't have enough bandwidth for building this project. But for anyone who is or will be working on this direction with pgwire, I will offer support.

[kernel-loophole]

@sunng87 would love to work on this .need some guide line and steps to do this.

[kernel-loophole]

@sunng87 can you describe a bit about StatelessMakeHandler in your example

pgwire/examples/proxy.rs

Line 10 in 902b25e

use pgwire::api::{ClientInfo, MakeHandler, StatelessMakeHandler, Type};

[sunng87]

@kernel-loophole Great to hear! This StatelessMakHandler is out of date. You can find a basic usage of the API in our sqlite example.

I shared my idea of building Postgres middleware in here. In the initial version, we can have StartupHandler, SimpleQueryHandler and ExtendedQueryHandler simply forward incoming messages to upstream. This requires us to:

1. implement some client logic to connect to upstream postgres, sending PgwireFrontendMessage and receiving PgwireBackendMessage
2. implement all those on_xxx handlers in the traits I mentioned, forward the message.

Once we finished 1 and 2, it should be a working layer-7 proxy. We can start to add:

• Upstream connection management
• API to monitoring or customize message forwarding

[kernel-loophole]

@sunng87 Thanks for sharing the details and the plan for building the Postgres middleware! I’ll take a look at the sqlite example for a basic usage reference. The approach you outlined for implementing the client logic and handlers to create a working layer-7 proxy makes sense. Once the initial setup is complete, adding connection management and monitoring APIs will definitely enhance the middleware. I'll start by exploring the traits you mentioned and keep you posted on the progress. Looking forward to collaborating on this!