Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: http2 support #515

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ COPY --from=production-deps /myapp/node_modules /myapp/node_modules
COPY --from=build /myapp/node_modules/.prisma /myapp/node_modules/.prisma

COPY --from=build /myapp/build /myapp/build
COPY --from=build /myapp/server /myapp/server
COPY --from=build /myapp/public /myapp/public
ADD . .

Expand Down
32 changes: 27 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,9 @@
Self-host your [turborepo remote cache](https://turborepo.org/docs/features/remote-caching) powerred by [Remix](https://remix.run/)

## Features
- Store artifacts where you want
- fs: file storage ([example](./docker-compose.fs.yml))

- Store artifacts where you want
- fs: file storage ([example](./docker-compose.fs.yml))
- s3: [https://aws.amazon.com/s3/](https://aws.amazon.com/s3/) ([example](./docker-compose.s3.yml))
- azure: [https://azure.microsoft.com/en-us/services/storage/blobs/](https://azure.microsoft.com/en-us/services/storage/blobs/)
- Manage users & teams
Expand All @@ -13,14 +14,13 @@ Self-host your [turborepo remote cache](https://turborepo.org/docs/features/remo
- See sessions globally, by user or by teams
- See artifacts globally, by user or by teams
- Display times saved by using the remote caching
- Probably can be deployed anywhere that support Remix ([How top deploy a Remix app?](https://remix.run/docs/en/v1/guides/deployment))
- Probably can be deployed anywhere that support Remix ([How top deploy a Remix app?](https://remix.run/docs/en/v1/guides/deployment))
- Docker Image support [thibmarechal/turborepo-remote-cache](https://hub.docker.com/r/thibmarechal/turborepo-remote-cache)



## Configuration

### USER configuration

- ADMIN_USERNAME : admin
- ADMIN_NAME : Admin
- ADMIN_PASSWORD : turbo
Expand All @@ -29,23 +29,31 @@ Self-host your [turborepo remote cache](https://turborepo.org/docs/features/remo
### TURBO configuration

### Storage configuration

- STORAGE_TYPE : the type of storage to use (default: fs, options: fs ,s3, azure)

#### fs (File Storage)

- STORAGE_FS_PATH : the path where to storage the cache,

#### s3 (Amazon S3)

- STORAGE_S3_ACCESS_KEY_ID
- STORAGE_S3_SECRET_ACCESS_KEY
- STORAGE_S3_FORCE_PATH_STYLE
- STORAGE_S3_ENDPOINT
- STORAGE_S3_REGION
- STORAGE_S3_SSL_ENABLED
- STORAGE_S3_BUCKET

#### azure (Azure blob storage)

- STORAGE_AZURE_STORAGE_ACCOUNT
- STORAGE_AZURE_STORAGE_ACCESS_KEY
- STORAGE_AZURE_STORAGE_CONTAINER

### Postgres configuration

- DATABASE_URL

## Repository configuration
Expand All @@ -57,7 +65,9 @@ Self-host your [turborepo remote cache](https://turborepo.org/docs/features/remo
"loginUrl": "http://localhost:8080/turbo/login"
}
```

// Link the repository to this remote server caching

```
npx turbo login
npx turbo link
Expand All @@ -66,17 +76,20 @@ npx turbo link
## Development

- Install dependencies

```sh
yarn install
```

- Launch a postgres database
You can use Docker with the docker-commpose.db.yml file if you want

```sh
docker-compose -f docker-compose.db.yml up -d
```

- Launche the remix dev server

```sh
yarn dev
```
Expand All @@ -100,3 +113,12 @@ yarn start
Now you'll need to pick a host to deploy it.

You can also use the Dockerfile

## HTTP2 support (e.g. for Google CloudRun)

In some cases you may need to use HTTP/2. To do so set the environment variable USE_HTTP2_NO_TLS=1.

E.g. **Google CloudRun** will have a payload limit of 32MB if you use HTTP/1.1, but some artifacts may be bigger than that.
There is no limit when using HTTP/2.

Make sure to set `--use-http2` on cloud run. See https://cloud.google.com/run/docs/configuring/http2
3 changes: 2 additions & 1 deletion app/routes/turbo.api.v8.artifacts.$hash.ts
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,8 @@ export const action: ActionFunction = async ({ request, params, context }) => {
}

const storage = new CacheStorage();
const contentLength = Number.parseInt(request.headers.get('Content-Length') as string);

const contentLength = Number.parseInt((request.headers.get('Content-Length') as string) ?? 0);
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

on http2 clients won't send the content-length header.

this will currently save the length as 0. This looks ugly in the ui, but it has no other negative effect.

it can be fixed later by updating the size after streaming the body into the store

try {
await Promise.all([
// The real type of request.body is ReadableStream. Somehow ReadableStream can be used as AsyncIterator
Expand Down
16 changes: 9 additions & 7 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
"type": "module",
"scripts": {
"build": "remix build",
"dev": "remix dev",
"dev": "remix dev -c 'node server/index.mjs'",
"docker": "docker-compose -f docker-compose.db.yml up -d",
"lint": "eslint ./app --color --max-warnings=0",
"setup": "prisma migrate dev && prisma db seed",
Expand All @@ -23,14 +23,13 @@
"typecheck": "tsc -b && tsc -b cypress && tsc -b prisma",
"validate": "run-p \"test -- --run\" lint typecheck test:e2e:run",
"prestart": "prisma migrate deploy && prisma db seed",
"start": "cross-env NODE_ENV=production remix-serve build/index.js"
"start": "cross-env NODE_ENV=production node server/index.mjs"
},
"dependencies": {
"@heroicons/react": "2.0.18",
"@prisma/client": "^5.8.1",
"@remix-run/node": "2.3.1",
"@remix-run/react": "2.3.1",
"@remix-run/serve": "2.3.1",
"@remix-run/node": "2.10.3",
"@remix-run/react": "2.10.3",
"@tanstack/react-table": "8.10.7",
"abstract-blob-store": "3.3.5",
"aws-sdk": "2.1551.0",
Expand All @@ -43,6 +42,8 @@
"domain-functions": "2.5.1",
"fs-blob-store": "6.0.0",
"isbot": "3.7.1",
"koa": "^2.15.3",
"koa-static": "^5.0.0",
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i had no succes with express, but koa supports http2 without issues

"npm-run-all": "4.1.5",
"prisma": "^5.8.1",
"prop-types": "15.8.1",
Expand All @@ -58,6 +59,7 @@
"remix-auth-microsoft": "^2.0.1",
"remix-auth-oidc": "1.0.0",
"remix-forms": "2.2.1",
"remix-koa-adapter": "^2.0.0",
"remix-utils": "^7.3.0",
"s3-blob-store": "4.1.1",
"superjson": "2.2.1",
Expand All @@ -69,8 +71,8 @@
"zod": "3.22.4"
},
"devDependencies": {
"@remix-run/dev": "2.3.1",
"@remix-run/eslint-config": "2.3.1",
"@remix-run/dev": "2.10.3",
"@remix-run/eslint-config": "2.10.3",
"@tailwindcss/typography": "0.5.10",
"@testing-library/cypress": "^10.0.1",
"@testing-library/dom": "9.3.4",
Expand Down
26 changes: 26 additions & 0 deletions server/index.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
import Koa from "koa";
import serve from "koa-static";
import http2 from "node:http2";
import { createRequestHandler } from "remix-koa-adapter";
import * as build from "../build/index.js";

const { PORT, USE_HTTP2_NO_TLS } = process.env;
const app = new Koa();

app.use(serve("public"));

app.use(
createRequestHandler({
build,
})
);

if (USE_HTTP2_NO_TLS) {
console.log("Using HTTP2 without TLS");
}
const server = USE_HTTP2_NO_TLS ? http2.createServer(app.callback()) : app;

server.listen(PORT, () => {
console.log(`server listening on port ${PORT}`);
// ... code to run after your server is running goes here ...
});
Loading