Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add vercel example to show a portal to private snowflake api #8706

Merged
merged 1 commit into from
Dec 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions examples/command/portals/vercel/example-1/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
*.ticket
.vercel
114 changes: 114 additions & 0 deletions examples/command/portals/vercel/example-1/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
# Call private APIs in Vercel

![Architecture](./diagram.png)


## Snowflake: Setup a private API in Snowpark Container Services

- [Follow](https://quickstarts.snowflake.com/guide/build_a_private_custom_api_in_python/index.html?index=..%2F..index#0) to setup a private API in Snowpark Container Services

- Create a database, table and insert some data

```sql
-- Use the accountadmin role to create the database, table and insert some data
USE ROLE ACCOUNTADMIN;

-- Grant the DATA_API_ROLE the necessary permissions to use the database and schema
GRANT USAGE ON DATABASE TEST_DATABASE TO ROLE DATA_API_ROLE;
GRANT USAGE ON SCHEMA TEST_DATABASE.PUBLIC TO ROLE DATA_API_ROLE;

-- Create the table
CREATE OR REPLACE TABLE TEST_DATABASE.PUBLIC.PRODUCTS (
id INTEGER PRIMARY KEY,
product_name VARCHAR(255),
price NUMBER(10,2)
);

-- Grant the DATA_API_ROLE the necessary permissions to use the table
GRANT SELECT, INSERT, UPDATE, DELETE ON TABLE TEST_DATABASE.PUBLIC.PRODUCTS TO ROLE DATA_API_ROLE;


-- Insert sample data
INSERT INTO TEST_DATABASE.PUBLIC.PRODUCTS (id, product_name, price)
VALUES (1, 'Ergonomic Keyboard', 10.99);

INSERT INTO TEST_DATABASE.PUBLIC.PRODUCTS (id, product_name, price)
VALUES (2, 'Wireless Mouse', 20.99);

INSERT INTO TEST_DATABASE.PUBLIC.PRODUCTS (id, product_name, price)
VALUES (3, 'LED Monitor 27"', 30.99);
```

- Create two new endpoints `/projects` and `/products/update`. Update `connector.py` to include the new endpoints.

```python
@connector.route('/products')
def get_products():
sql = '''
SELECT id, product_name, price
FROM TEST_DATABASE.public.products
'''
try:
res = conn.cursor(DictCursor).execute(sql)
return make_response(jsonify(res.fetchall()))
except:
abort(500, "Error reading from Snowflake. Check the logs for details.")

@connector.route('/products/update', methods=['POST'])
def update_products():
sql = '''
UPDATE TEST_DATABASE.public.products
SET price = ROUND(UNIFORM(10, 100, RANDOM()), 2)
'''
try:
conn.cursor().execute(sql)
conn.commit()
return make_response(jsonify({"message": "Products prices updated successfully"}), 200)
except:
conn.rollback()
abort(500, "Error updating prices in Snowflake. Check the logs for details.")
```

- Rebuild and publish the dataapi image (`/api/private/api/dataapi`) with new endpoints. Drop and create `API.PRIVATE.API` service with new image.


## Vercel: Setup Vercel Serverless functions to access the private API.

### Download latest ockam binary

- Download ockam binary (x86_64-unknown-linux-gnu) from the [Ockam](https://github.com/build-trust/ockam/releases) github repository and place it in the `data/linux-x86_64` directory. Rename the binary to `ockam`.

### Create an enrollment ticket for the Vercel function

```sh
# Generate an inlet ticket for the Vercel function.
ockam project ticket --usage-count 100 --expires-in 10h --attribute snowflake-api-service-inlet > vercel-inlet.ticket
```

### Setup a Vercel project and add the inlet ticket as a secret

- Select `Project Settings`, `Environments`, `Production
- Click `Add Environment Variable`
- Select `Sensitive`
- Add `OCKAM_SNOWFLAKE_INLET_ENROLLMENT_TICKET` as the name
- Add the value of `vercel-inlet.ticket` as the value
- Click `Add`

### Deploy the Vercel function

- Setup vercel cli and select the project
- Deploy the function inside the `api` directory

```sh
vercel --prod
```

### Test the Vercel function

- Use the `/api` endpoint to get the products
- Use the `/api/update` endpoint to update the products


## Setup a Next.js frontend deployed to Vercel to access vercel function

- Optionally you can setup a Next.js frontend to access the vercel function via it's public URLs.
182 changes: 182 additions & 0 deletions examples/command/portals/vercel/example-1/api/index.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,182 @@
"""
Simple API handler for accessing Snowflake private API endpoints via Ockam portal
"""

import json
import os
import subprocess
import threading
import time
from http.server import BaseHTTPRequestHandler
import requests

# Constants
OCKAM_BINARY_PATH = '/var/task/data/linux-x86_64/ockam' if os.environ.get('VERCEL_ENV') == 'production' else 'ockam'
MAX_RETRIES = 60
RETRY_DELAY = 1 # seconds
LOCAL_API_ENDPOINT = "http://127.0.0.1:8081"
REQUEST_TIMEOUT = 3 # seconds

# Track if Ockam node creation has been attempted
# Only one Ockam node should run per serverless function runtime - this flag prevents multiple node creation attempts.
NODE_INITIALIZED = False

def is_production() -> bool:
"""Check if the environment is production."""
return os.environ.get('VERCEL_ENV') == 'production'

def get_ockam_version() -> str:
"""
Get the installed Ockam version.
Returns version string or error message.
"""
try:
ockam_path = OCKAM_BINARY_PATH
if is_production() and not os.path.exists(ockam_path):
return f"Error: Ockam binary not found at {os.path.abspath(ockam_path)}"

result = subprocess.run([ockam_path, '--version'], capture_output=True, text=True)

return result.stdout.strip() if result.returncode == 0 else f"Error running ockam: {result.stderr}"
except Exception as e:
return f"Error: {str(e)} (CWD: {os.getcwd()})"

class handler(BaseHTTPRequestHandler):
"""Handler for Snowflake API requests via Ockam secure channel."""

def create_ockam_node(self, enrollment_ticket: str) -> bool:
"""
Initialize Ockam node with the provided enrollment ticket.
Returns True if successful or already attempted, False otherwise.
"""
if not is_production():
return False

global NODE_INITIALIZED
request_id = self.headers.get('x-vercel-id', os.urandom(8).hex())

print(f"[{request_id}] Starting with NODE_INITIALIZED = {NODE_INITIALIZED}")

if not NODE_INITIALIZED:
NODE_INITIALIZED = True

def run_ockam_node():
try:
config = '''{
"tcp-inlet": {
"from": "127.0.0.1:8081",
"via": "snowflake-api-service-relay",
"allow": "snowflake-api-service-outlet"
}
}'''

print(f"[{request_id}] Starting ockam node with config: {config}", flush=True)

subprocess.Popen([
OCKAM_BINARY_PATH,
'node',
'create',
'--configuration',
config,
'--enrollment-ticket',
enrollment_ticket.strip()
], env={
**os.environ,
'OCKAM_HOME': '/tmp',
'OCKAM_OPENTELEMETRY_EXPORT': 'false',
'OCKAM_DISABLE_UPGRADE_CHECK': 'true'
})

except Exception as e:
print(f"[{request_id}] Error in node creation: {str(e)}", flush=True)

thread = threading.Thread(target=run_ockam_node)
thread.daemon = True
print(f"[{request_id}] Starting ockam node thread", flush=True)
thread.start()

return True

def handle_select(self, request_id: str) -> dict:
"""Handle GET requests for product data."""
for retry in range(MAX_RETRIES):
try:
response = requests.get(
f"{LOCAL_API_ENDPOINT}/connector/products",
timeout=REQUEST_TIMEOUT
)
response.raise_for_status()
print(f"[{request_id}] Connected successfully after {retry} retries")
return {"status": "success", "data": response.json()}

except Exception as e:
print(f"[{request_id}] Attempt {retry + 1} failed: {str(e)}")
if retry < MAX_RETRIES - 1:
time.sleep(RETRY_DELAY)
else:
return {"status": "error", "message": f"Connection failed: {str(e)}"}

def handle_update(self, request_id: str) -> dict:
"""Handle POST requests for product updates."""
for retry in range(MAX_RETRIES):
try:
response = requests.post(
f"{LOCAL_API_ENDPOINT}/connector/products/update",
timeout=REQUEST_TIMEOUT
)
response.raise_for_status()
print(f"[{request_id}] Update successful after {retry} retries")

updated_values = self.handle_select(request_id)
return {
"status": "success",
"update_result": response.json(),
"current_values": updated_values["data"]
}

except Exception as e:
print(f"[{request_id}] Attempt {retry + 1} failed: {str(e)}")
if retry < MAX_RETRIES - 1:
time.sleep(RETRY_DELAY)
else:
return {"status": "error", "message": f"Update failed: {str(e)}"}

def do_GET(self):
"""Handle incoming GET requests."""
try:
# Verify Ockam installation
version_result = get_ockam_version()
if 'Error' in version_result:
self._send_error(500, version_result)
return

# Verify enrollment ticket
enrollment_ticket = os.environ.get('OCKAM_SNOWFLAKE_INLET_ENROLLMENT_TICKET')
if not enrollment_ticket:
self._send_error(500, 'OCKAM_SNOWFLAKE_INLET_ENROLLMENT_TICKET not configured')
return

# Initialize Ockam node
if not self.create_ockam_node(enrollment_ticket):
self._send_error(500, 'Failed to initialize Ockam node')
return

# Handle the request
request_id = self.headers.get('x-vercel-id', os.urandom(8).hex())
result = self.handle_update(request_id) if self.path == '/api/update' else self.handle_select(request_id)

self._send_response(200, result)

except Exception as e:
self._send_error(500, str(e))

def _send_response(self, status: int, data: dict):
"""Helper method to send JSON responses."""
self.send_response(status)
self.send_header('Content-Type', 'application/json')
self.end_headers()
self.wfile.write(json.dumps(data).encode())

def _send_error(self, status: int, message: str):
"""Helper method to send error responses."""
self._send_response(status, {'error': message})
Empty file.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions examples/command/portals/vercel/example-1/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Flask==3.0.3
psycopg2-binary==2.9.9
requests==2.31.0
19 changes: 19 additions & 0 deletions examples/command/portals/vercel/example-1/vercel.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
{
"redirects": [
{
"source": "/",
"destination": "/api"
}
],
"rewrites": [
{
"source": "/api/update",
"destination": "/api"
}
],
"functions": {
"api/**/*": {
"maxDuration": 300
}
}
}
Loading