Building Docker container without database connection?

[Manouchehri]

I'm a bit confused on why we need to set DATABASE_URL, NEXTAUTH_SECRET, SALT, and NEXTAUTH_URL when just building the container. Shouldn't these just be needed for runtime?

langfuse/.github/workflows/pipeline.yml

Lines 197 to 200 in 37c0737

	DATABASE_URL=${{ env.DATABASE_URL }}
	NEXTAUTH_SECRET=${{ env.NEXTAUTH_SECRET }}
	NEXTAUTH_URL=${{ env.NEXTAUTH_URL }}
	SALT=${{ env.SALT }}

My team uses Google Cloud Build and we usually don't put secrets into build environments.

[marcklingen]

Agree, this does not really make sense as none of the secrets are used to build the container. This is a weird fallout of validating environment variables/secrets using https://github.com/t3-oss/t3-env

In the meantime you can use fake strings at build time. Here's what we use to build the application in CD:

langfuse/.github/workflows/pipeline.yml

Lines 150 to 153 in 20d00e5

	DATABASE_URL: postgresql://postgres:postgres@localhost:5432/postgres
	NEXTAUTH_SECRET: "secret"
	SALT: "salt"
	NEXTAUTH_URL: "http://localhost:3030"

There is a fix outlined here: t3-oss/t3-env#85 to skip validation at build time. I created an issue for this.

[marcklingen]

That's really annoying, good feature request for Google Cloud Run :) Pulling/pushing images to their registry seems tedious.

The main branch is not meant to be used in this way as it is kind of a staging branch until a version is tagged/released. All CI tests need to pass before changes are merged to main but sometimes we find issues and fix them before creating a new release.

Options

• Safest option: Can you configure to only build tagged (v*) commits on main?
• Less safe but still much better than using main: You could use the production branch which we use for the Langfuse Cloud product. This branch is updated whenever a new release is created but we sometimes also release new changes to this branch before creating a new docker release. This is less safe as we do not need to consider all potential risks before merging to production since we migrate the database ourselves.

[Manouchehri]

Eh, to be honest I doubt Google will address it, since they already support Docker Hub. (I agree they should support GitHub's container registry since it's getting more popular. I wouldn't hold my breath though.)

So what we do / have done, is:

1. Mirrored 37c0737 to our private GitHub repo
2. Whenever there's a new tagged release, we'll merge it into the main of our private GitHub repo
3. We push main (which contains the new tagged release) and that automatically deploys

We don't automatically pick up commits from langfuse/langfuse.

We have the same workflow for other projects like LiteLLM and it works quite well. =)

[marcklingen]

Awesome, this is perfect. Just wanted to make sure that you don't run into any unexpected issues.

[Manouchehri]

For anyone else reading this, we use the Slack RSS app to monitor the Langfuse release feed at https://github.com/langfuse/langfuse/releases.atom

Yeah, I've been burned too many times by unexpected breaking changes in various projects. I like new features, but only when I have time to test them out. =P

[marcklingen]

Thanks for sharing, did not know about Slack RSS <> GitHub release feed, just added it to our slack 🙏

[marcklingen]

Just found this thread again. We have fixed this, you no longer need to provide these environments to build the container. (adding this here for reference in case someone finds this thread)