Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide Debian image #34

Merged
merged 2 commits into from
Jul 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -1 +1,4 @@
test

Dockerfile.alpine
Dockerfile.bookworm
51 changes: 30 additions & 21 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@ jobs:
runs-on: ubuntu-latest
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just noticed this and the thought "should we be using Debian instead?" popped up?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

anecdotally, i would say ubuntu seems to be more standard and common for things like this. personally, i tend to think "ubuntu unless i have a reason not to"

Copy link
Member

@justinclift justinclift Jul 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that used to be the previous default for a lot of people, though it doesn't seem to be the case as much any more.

For me, when Canonical started including advertising in their cli is when I personally stopped defaulting to Ubuntu for anything and begun defaulting to Debian instead.

For an existing build process though (like here), it's unlikely to be worth the effort to go about changing the distribution due to trivial preference stuff like that. So yeah, no worries. 😉

strategy:
matrix:
flavor:
- alpine
- bookworm
pg_version:
- "9.5"
- "9.6"
Expand Down Expand Up @@ -40,14 +43,15 @@ jobs:
- name: Build and push image
uses: docker/build-push-action@v5
with:
file: "Dockerfile.${{ matrix.flavor }}"
push: true
platforms: linux/amd64,linux/arm64
build-args: |
"PGTARGET=16"
target: "build-${{ matrix.pg_version }}"
tags: "pgautoupgrade/pgautoupgrade:build-${{ matrix.pg_version }}"
tags: "pgautoupgrade/pgautoupgrade:build-${{ matrix.pg_version }}-${{ matrix.flavor }}"
cache-to: type=inline
cache-from: type=registry,ref=pgautoupgrade/pgautoupgrade:build-${{ matrix.pg_version }}
cache-from: type=registry,ref=pgautoupgrade/pgautoupgrade:build-${{ matrix.pg_version }}-${{ matrix.flavor }}

target-images:
runs-on: ubuntu-latest
Expand All @@ -57,24 +61,25 @@ jobs:
env:
# but still use our public caches in any case
# they might be outdated, in which case a full rebuild will be triggered
TARGET_TAG: ${{ github.ref == 'refs/heads/main' && 'alpine' || 'dev-alpine' }}
CACHE_FROM: |
type=registry,ref=pgautoupgrade/pgautoupgrade:build-9.5
type=registry,ref=pgautoupgrade/pgautoupgrade:build-9.6
type=registry,ref=pgautoupgrade/pgautoupgrade:build-10
type=registry,ref=pgautoupgrade/pgautoupgrade:build-11
type=registry,ref=pgautoupgrade/pgautoupgrade:build-12
type=registry,ref=pgautoupgrade/pgautoupgrade:build-13
type=registry,ref=pgautoupgrade/pgautoupgrade:build-14
type=registry,ref=pgautoupgrade/pgautoupgrade:build-15
type=registry,ref=pgautoupgrade/pgautoupgrade:build-16
type=registry,ref=pgautoupgrade/pgautoupgrade:${{ matrix.pg_target }}-alpine
type=registry,ref=pgautoupgrade/pgautoupgrade:${{ matrix.pg_target }}-alpine3.19
# we cannot access TARGET_TAG from env
# https://docs.github.com/en/actions/learn-github-actions/contexts#context-availability
type=registry,ref=pgautoupgrade/pgautoupgrade:build-9.5-${{ matrix.operating_system.flavor }}
type=registry,ref=pgautoupgrade/pgautoupgrade:build-9.6-${{ matrix.operating_system.flavor }}
type=registry,ref=pgautoupgrade/pgautoupgrade:build-10-${{ matrix.operating_system.flavor }}
type=registry,ref=pgautoupgrade/pgautoupgrade:build-11-${{ matrix.operating_system.flavor }}
type=registry,ref=pgautoupgrade/pgautoupgrade:build-12-${{ matrix.operating_system.flavor }}
type=registry,ref=pgautoupgrade/pgautoupgrade:build-13-${{ matrix.operating_system.flavor }}
type=registry,ref=pgautoupgrade/pgautoupgrade:build-14-${{ matrix.operating_system.flavor }}
type=registry,ref=pgautoupgrade/pgautoupgrade:build-15-${{ matrix.operating_system.flavor }}
type=registry,ref=pgautoupgrade/pgautoupgrade:build-16-${{ matrix.operating_system.flavor }}
type=registry,ref=pgautoupgrade/pgautoupgrade:${{ matrix.pg_target }}-${{ matrix.operating_system.flavor }}
type=registry,ref=pgautoupgrade/pgautoupgrade:${{ matrix.pg_target }}-${{ matrix.operating_system.flavor }}${{ matrix.operating_system.version }}
Comment on lines +74 to +75
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so for bookworm, no version is set in the GitHub Actions matrix. If I am not mistaken, this should work as GitHub Actions will substitute the missing version as an empty string.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets try it out. 😄


strategy:
matrix:
operating_system:
- flavor: "alpine"
version: "3.19"
- flavor: "bookworm"
pg_target:
- "12"
- "13"
Expand All @@ -99,10 +104,11 @@ jobs:
- name: Build image
uses: docker/build-push-action@v5
with:
file: "Dockerfile.${{ matrix.operating_system.flavor }}"
load: true
tags: |
"pgautoupgrade/pgautoupgrade:${{ matrix.pg_target }}-alpine"
"pgautoupgrade/pgautoupgrade:${{ matrix.pg_target }}-alpine3.19"
"pgautoupgrade/pgautoupgrade:${{ matrix.pg_target }}-${{ matrix.operating_system.flavor }}"
"pgautoupgrade/pgautoupgrade:${{ matrix.pg_target }}-${{ matrix.operating_system.flavor }}${{ matrix.operating_system.version }}"
build-args: |
"PGTARGET=${{ matrix.pg_target }}"
cache-to: type=inline
Expand All @@ -116,25 +122,28 @@ jobs:
make test
env:
PGTARGET: ${{ matrix.pg_target }}
OS_FLAVOR: ${{ matrix.operating_system.flavor }}

- name: Push image
if: github.repository == 'pgautoupgrade/docker-pgautoupgrade' && github.ref == 'refs/heads/main'
uses: docker/build-push-action@v5
with:
file: "Dockerfile.${{ matrix.operating_system.flavor }}"
platforms: linux/amd64,linux/arm64
tags: |
"pgautoupgrade/pgautoupgrade:${{ matrix.pg_target }}-alpine"
"pgautoupgrade/pgautoupgrade:${{ matrix.pg_target }}-alpine3.19"
"pgautoupgrade/pgautoupgrade:${{ matrix.pg_target }}-${{ matrix.operating_system.flavor }}"
"pgautoupgrade/pgautoupgrade:${{ matrix.pg_target }}-${{ matrix.operating_system.flavor }}${{ matrix.operating_system.version }}"
build-args: |
"PGTARGET=${{ matrix.pg_target }}"
push: true
cache-to: type=inline
cache-from: "${{ env.CACHE_FROM }}"

- name: Push latest image
if: github.repository == 'pgautoupgrade/docker-pgautoupgrade' && github.ref == 'refs/heads/main' && matrix.pg_target == '16'
if: github.repository == 'pgautoupgrade/docker-pgautoupgrade' && github.ref == 'refs/heads/main' && matrix.pg_target == '16' && matrix.operating_system.flavor == 'alpine'
uses: docker/build-push-action@v5
with:
file: "Dockerfile.${{ matrix.operating_system.flavor }}"
platforms: linux/amd64,linux/arm64
tags: |
"pgautoupgrade/pgautoupgrade:latest"
Expand Down
File renamed without changes.
174 changes: 174 additions & 0 deletions Dockerfile.bookworm
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not want to add a bunch of if statements to the Dockerfile, so I opted to add another Dockerfile.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All good, it makes sense to me.

My feeling is that this will probably be a thing that we try out and see how well it works (ie over a few weeks/months), and from that we can decide if it's best to keep two Dockerfiles or have it all in one, or something else. 😄

Original file line number Diff line number Diff line change
@@ -0,0 +1,174 @@
ARG PGTARGET=16

### Things we need in all build containers
FROM debian:bookworm as base-build

# The versions of PostgreSQL to use
ENV PG95=9.5.25
ENV PG96=9.6.24
ENV PG10=10.23
ENV PG11=11.22
ENV PG12=12.19
ENV PG13=13.15
ENV PG14=14.12
ENV PG15=15.7
ENV PG16=16.3

# Where we'll do all our compiling and similar
ENV BUILD_ROOT /buildroot

# Make the directory for building, and set it as the default for the following Docker commands
RUN mkdir ${BUILD_ROOT}
WORKDIR ${BUILD_ROOT}

# Install things needed for development
# We might want to install "alpine-sdk" instead of "build-base", if build-base
# doesn't have everything we need
RUN apt update && \
apt upgrade && \
apt install -y build-essential libicu-dev liblz4-dev locales tzdata zlib1g-dev libzstd-dev wget pkg-config && \
apt clean

### PostgreSQL 9.5
FROM base-build as build-9.5

RUN wget https://ftp.postgresql.org/pub/source/v${PG95}/postgresql-${PG95}.tar.bz2 && \
tar -xf postgresql-9.5*.tar.bz2

RUN cd postgresql-9.5.* && \
./configure --prefix=/usr/local-pg9.5 --with-openssl=no --without-readline --with-system-tzdata=/usr/share/zoneinfo --enable-debug=no CFLAGS="-Os" && \
make -j $(nproc) && \
make install-world && \
rm -rf /usr/local-pg9.5/include

### PostgreSQL 9.6
FROM base-build as build-9.6

RUN wget https://ftp.postgresql.org/pub/source/v${PG96}/postgresql-${PG96}.tar.bz2 && \
tar -xf postgresql-9.6*.tar.bz2

RUN cd postgresql-9.6.* && \
./configure --prefix=/usr/local-pg9.6 --with-openssl=no --without-readline --with-system-tzdata=/usr/share/zoneinfo --enable-debug=no CFLAGS="-Os" && \
make -j $(nproc) && \
make install-world && \
rm -rf /usr/local-pg9.6/include

### PostgreSQL 10
FROM base-build as build-10
RUN wget https://ftp.postgresql.org/pub/source/v${PG10}/postgresql-${PG10}.tar.bz2 && \
tar -xf postgresql-10*.tar.bz2

RUN cd postgresql-10.* && \
./configure --prefix=/usr/local-pg10 --with-openssl=no --without-readline --with-icu --with-system-tzdata=/usr/share/zoneinfo --enable-debug=no CFLAGS="-Os" && \
make -j $(nproc) && \
make install-world && \
rm -rf /usr/local-pg10/include

### PostgreSQL 11
FROM base-build as build-11
RUN wget https://ftp.postgresql.org/pub/source/v${PG11}/postgresql-${PG11}.tar.bz2 && \
tar -xf postgresql-11*.tar.bz2

RUN cd postgresql-11.* && \
./configure --prefix=/usr/local-pg11 --with-openssl=no --without-readline --with-icu --with-system-tzdata=/usr/share/zoneinfo --enable-debug=no CFLAGS="-Os" && \
make -j $(nproc) && \
make install-world && \
rm -rf /usr/local-pg11/include

### PostgreSQL 12
FROM base-build as build-12
RUN wget https://ftp.postgresql.org/pub/source/v${PG12}/postgresql-${PG12}.tar.bz2 && \
tar -xf postgresql-12*.tar.bz2

RUN cd postgresql-12.* && \
./configure --prefix=/usr/local-pg12 --with-openssl=no --without-readline --with-icu --with-system-tzdata=/usr/share/zoneinfo --enable-debug=no CFLAGS="-Os" && \
make -j $(nproc) && \
make install-world && \
rm -rf /usr/local-pg12/include

### PostgreSQL 13
FROM base-build as build-13

RUN wget https://ftp.postgresql.org/pub/source/v${PG13}/postgresql-${PG13}.tar.bz2 && \
tar -xf postgresql-13*.tar.bz2

RUN cd postgresql-13.* && \
./configure --prefix=/usr/local-pg13 --with-openssl=no --without-readline --with-icu --with-system-tzdata=/usr/share/zoneinfo --enable-debug=no CFLAGS="-Os" && \
make -j $(nproc) && \
make install-world && \
rm -rf /usr/local-pg13/include

### PostgreSQL 14
FROM base-build as build-14

RUN wget https://ftp.postgresql.org/pub/source/v${PG14}/postgresql-${PG14}.tar.bz2 && \
tar -xf postgresql-14*.tar.bz2

RUN cd postgresql-14.* && \
./configure --prefix=/usr/local-pg14 --with-openssl=no --without-readline --with-icu --with-lz4 --with-system-tzdata=/usr/share/zoneinfo --enable-debug=no CFLAGS="-Os" && \
make -j $(nproc) && \
make install-world && \
rm -rf /usr/local-pg14/include

### PostgreSQL 15
FROM base-build as build-15

RUN wget https://ftp.postgresql.org/pub/source/v${PG15}/postgresql-${PG15}.tar.bz2 && \
tar -xf postgresql-15*.tar.bz2

RUN cd postgresql-15.* && \
./configure --prefix=/usr/local-pg15 --with-openssl=no --without-readline --with-icu --with-lz4 --with-system-tzdata=/usr/share/zoneinfo --enable-debug=no CFLAGS="-Os" && \
make -j $(nproc) && \
make install-world && \
rm -rf /usr/local-pg15/include

### PostgreSQL 16
FROM base-build as build-16

RUN wget https://ftp.postgresql.org/pub/source/v${PG16}/postgresql-${PG16}.tar.gz && \
tar -xf postgresql-16*.tar.gz

RUN cd postgresql-16.* && \
./configure --prefix=/usr/local-pg16 --with-openssl=no --without-readline --with-icu --with-lz4 --with-system-tzdata=/usr/share/zoneinfo --enable-debug=no CFLAGS="-Os" && \
make -j $(nproc) && \
make install-world && \
rm -rf /usr/local-pg16/include

# Use the PostgreSQL Alpine image as our output image base
FROM postgres:${PGTARGET}-bookworm

# We need to define this here, to make the above PGTARGET available after the FROM
ARG PGTARGET

# Copy across our compiled files
COPY --from=build-9.5 /usr/local-pg9.5 /usr/local-pg9.5
COPY --from=build-9.6 /usr/local-pg9.6 /usr/local-pg9.6
COPY --from=build-10 /usr/local-pg10 /usr/local-pg10
COPY --from=build-11 /usr/local-pg11 /usr/local-pg11
COPY --from=build-12 /usr/local-pg12 /usr/local-pg12
COPY --from=build-13 /usr/local-pg13 /usr/local-pg13
COPY --from=build-14 /usr/local-pg14 /usr/local-pg14
COPY --from=build-15 /usr/local-pg15 /usr/local-pg15
COPY --from=build-16 /usr/local-pg16 /usr/local-pg16

# Remove any left over PG directory stubs. Doesn't help with image size, just with clarity on what's in the image.
RUN if [ "${PGTARGET}" -eq 12 ]; then rm -rf /usr/local-pg12 /usr/local-pg13 /usr/local-pg14 /usr/local-pg15 /usr/local-pg16; fi
RUN if [ "${PGTARGET}" -eq 13 ]; then rm -rf /usr/local-pg13 /usr/local-pg14 /usr/local-pg15 /usr/local-pg16; fi
RUN if [ "${PGTARGET}" -eq 14 ]; then rm -rf /usr/local-pg14 /usr/local-pg15 /usr/local-pg16; fi
RUN if [ "${PGTARGET}" -eq 15 ]; then rm -rf /usr/local-pg15 /usr/local-pg16; fi
RUN if [ "${PGTARGET}" -eq 16 ]; then rm -rf /usr/local-pg16; fi

# Install locale
RUN apt update && \
apt install -y icu-devtools locales tzdata && \
apt clean

# Pass the PG build target through to the running image
ENV PGTARGET=${PGTARGET}

# Set up the script run by the container when it starts
WORKDIR /var/lib/postgresql
COPY docker-entrypoint.sh /usr/local/bin/
ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"]

CMD ["postgres"]
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,9 @@ To always use the latest version of PostgreSQL, use the tag

pgautoupgrade/pgautoupgrade:latest

Please note that our `latest` tag is based on Alpine. The one by
Postgres is based on Debian.

If you instead want to run a specific version of PostgreSQL
then pick a matching tag on our Docker Hub. For example, to
use PostgreSQL 15 you can use:
Expand Down
23 changes: 19 additions & 4 deletions docker-entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -279,7 +279,8 @@ docker_temp_server_stop() {
# Initialise PG data directory in a temp location with a specific locale
initdb_locale() {
echo "Initialising PostgreSQL ${PGTARGET} data directory"
/usr/local/bin/initdb --username="${POSTGRES_USER}" ${POSTGRES_INITDB_ARGS} ${PGDATA}/new/
bin_path=$(get_bin_path)
${bin_path}/initdb --username="${POSTGRES_USER}" ${POSTGRES_INITDB_ARGS} ${PGDATA}/new/
}

# check arguments for an option that would cause postgres to stop
Expand All @@ -299,6 +300,14 @@ _pg_want_help() {
return 1
}

get_bin_path() {
if [ -f /etc/alpine-release ]; then
echo "/usr/local/bin"
else
echo "/usr/lib/postgresql/${PGTARGET}/bin"
fi
}

_main() {
# if first arg looks like a flag, assume we want to run postgres server
if [ "${1:0:1}" = '-' ]; then
Expand All @@ -310,8 +319,13 @@ _main() {
# setup data directories and permissions (when run as root)
docker_create_db_directories
if [ "$(id -u)" = '0' ]; then
# then restart script as postgres user
exec su-exec postgres "$BASH_SOURCE" "$@"
if [ -f /etc/alpine-release ]; then
# If running on Alpine, use su-exec
exec su-exec postgres "$BASH_SOURCE" "$@"
else
# Otherwise, use gosu
exec gosu postgres "$BASH_SOURCE" "$@"
fi
fi

# only run initialization on an empty data directory
Expand Down Expand Up @@ -518,7 +532,8 @@ _main() {
echo "---------------------------------------"
echo "Running pg_upgrade command, from $(pwd)"
echo "---------------------------------------"
/usr/local/bin/pg_upgrade --username="${POSTGRES_USER}" --link -d "${OLD}" -D "${NEW}" -b "${OLDPATH}/bin" -B /usr/local/bin --socketdir="/var/run/postgresql"
bin_path=$(get_bin_path)
${bin_path}/pg_upgrade --username="${POSTGRES_USER}" --link -d "${OLD}" -D "${NEW}" -b "${OLDPATH}/bin" -B "${bin_path}" --socketdir="/var/run/postgresql"
echo "--------------------------------------"
echo "Running pg_upgrade command is complete"
echo "--------------------------------------"
Expand Down
5 changes: 3 additions & 2 deletions test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ test_down() {
test_run() {
VERSION=$1
TARGET=$2
FLAVOR=$3

# Delete any existing test PostgreSQL data
if [ -d postgres-data ]; then
Expand All @@ -25,7 +26,7 @@ test_run() {
docker compose -f "docker-compose-pg${VERSION}.yml" run --rm server create_db

# Start Redash normally, using an "autoupdate" version of PostgreSQL
TARGET_TAG="${TARGET}-alpine" docker compose -f docker-compose-pgauto.yml up --wait -d
TARGET_TAG="${TARGET}-${FLAVOR}" docker compose -f docker-compose-pgauto.yml up --wait -d

# Verify the PostgreSQL data files are now the target version
PGVER=$(sudo cat postgres-data/PG_VERSION)
Expand Down Expand Up @@ -64,7 +65,7 @@ cd test || exit 1
for version in "${PG_VERSIONS[@]}"; do
# Only test if the version is less than the latest version
if [[ $(echo "$version < $PGTARGET" | bc) -eq 1 ]]; then
test_run "$version" "$PGTARGET"
test_run "$version" "$PGTARGET" "$OS_FLAVOR"
fi
done

Expand Down