Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Prometheus's data directory mapping + fixed permissions error. #535

Merged
merged 1 commit into from
Dec 10, 2024

Conversation

mcteo
Copy link
Contributor

@mcteo mcteo commented Dec 8, 2024

A change to the Dockerfile for Prometheus has overridden the default data storage directory from /prometheus/data to /prometheus. This means any new containers created using this template will technically continue to work, but due to the data being stored in a transient docker volume, and not on the intended UnRAID appdata share, the data will not survive the container being recreated.

I noticed this myself when trying to find why Grafana only showed a weeks worth of data, despite the Prometheus being configured by default to retain 15 days. (and I clean up docker volumes every so often).

N.B. This will overwrite the data currently in your docker volume, with your presumedly empty appdata folder. To get around this, I manually copied /prometheus/* to /prometheus/data/ before applying this update.

After making this change on my own server, the Prometheus container refused to start, due to permission denied when reading the new data. I found this was because while the container's process is configured to run as nobody, the UID doesn't match the UID of UnRAID's nobody, therefore added the --user 99:100 to override the container's UID got things working smoothly. On UnRAID, UID 99 maps to nobody, and GID 100 maps to users, which should be the default permissions, but you might have to run chown -r nobody:users /mnt/user/appdata/prometheus/ if you copied the data from volume to share, as mentioned above.

A [change](prometheus/prometheus@ceb844a) to the Dockerfile for Prometheus has overridden the default data storage directory from `/prometheus/data` to `/prometheus`. This means any new containers created using this template will technically continue to work, but due to the data being stored in a transient docker volume, and not on the intended UnRAID appdata share, the data will not survive the container being recreated.

I noticed this myself when trying to find why Grafana only showed a weeks worth of data, despite the Prometheus being configured by default to retain 15 days. (and I clean up docker volumes every so often).

N.B. This will overwrite the data currently in your docker volume, with your presumedly empty appdata folder. To get around this, I manually copied `/prometheus/*` to `/prometheus/data/` before applying this update.

After making this change on my own server, the Prometheus container refused to start, due to permission denied when reading the new data. I found this was because while the container's process is configured to run as `nobody`, the UID doesn't match the UID of UnRAID's `nobody`, therefore added the `--user 99:100` to override the container's UID got things working smoothly.
@nwithan8 nwithan8 requested a review from Roxedus December 9, 2024 16:48
@Roxedus Roxedus merged commit 2f00c86 into selfhosters:master Dec 10, 2024
1 check passed
@mcteo mcteo deleted the patch-1 branch December 13, 2024 23:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants