MDBF-866 - Create bintar builder from non-EOL distro #690
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
RazvanLiviuVarzaru
force-pushed
the
feature/new-bintar
branch
from
6f6f515 to
eb56650
Compare
RazvanLiviuVarzaru
changed the title
RazvanLiviuVarzaru
force-pushed
the
feature/new-bintar
branch
5 times, most recently
from
f4b106b to
dc0ce2a
Compare
|
@cvicentiu If you want to inspect the resulting container image, please see: |
RazvanLiviuVarzaru
force-pushed
the
feature/new-bintar
branch
from
b544c66 to
7c1fdcd
Compare
|
Hi @cvicentiu The PR is ready for review, I've added 7c1fdcd to make GnuTLS work. See the commit message / gnutls.sh comments. If you would like to test the server, I've updated to what this PR will produce. |
cvicentiu
reviewed
Jan 22, 2025
RazvanLiviuVarzaru
force-pushed
the
feature/new-bintar
branch
from
7c1fdcd to
20046a3
Compare
|
Hi, @cvicentiu I pushed 2 more commits: 20046a3 introduced almalinux 8 bintar builder Please see the commit's messages for more details. This is all that's needed for the new bintar builder so far. |
cvicentiu
approved these changes
Jan 23, 2025
There was a problem hiding this comment.
Approve with the following changes required as discussed over the 2 hour pair-debugging session:
- Use GnuTLS 3.7.11 in the
./gnutls.sh - Separate dropping eigen3-devel as a separate commit, mentioning that there is no need for eigen3 since the MariaDB Vector code did not show significant improvements.
- Separate the almalinux build steps of the libraries in separate RUN commands + hadolint exception.
AlmaLinux 8 was chosen as the oldest non-eol distro for a new bintar which will build server versions >= 11.7 Buildbot configuration changes will follow after this PR is merged. (new builder,worker). A dedicated workflow was defined, with a workflow_call directive to allow us to rebuild the image anytime we like. About bintar.Dockerfile - it is based on the centos.Dockerfile with small changes to remove redundant code - use galera-4 from RHEL-8 because we no longer build on alma/rocky, neither the server or the galera library. Compatibility wise, should be ok, because we also test RHEL packages on alma/rocky VM's. About static libs: - static libraries are build in a separate stage to reduce the image size - the general rule is to install the -devel packages (provide headers) then to compile the static library and move it in a path used then by CMAKE - the strategy to compile static is to take the source rpm and then install deps + re-build. Different libs have different needs in terms of what custom modifications of the SPEC files are needed. Testing: - one can build the image from scratch, compile the server and check resulted binaries with LDD - the list of *.sh files introduced by this patch is the minimum I found that will produce a binary package similar with what we've released so far.
From GnuTLS I had to disable: --without-p11-kit --without-idn for 11.8 server compilation to work. GnuTLS cannot link nettle/gmp if we don't compile these libraries statically. The first approach was to use .src.rpm for all of them, but the distro version of nettle has a bug causing OOM during build. In the end, I bumped the versions to the one dbart used in buildbot.mariadb.net. Hopefully this workaround will work until almalinux-8 is EOL. We should probably try again to compile from distro .src.rpm when nettle is updated, if ever, during the lifetime of the distro.
- never released - not built according to bintar specs - eol distro If we ever need an aarch bintar we will create one according to the bintar specs.
- should run on currently idle hz-bbw7. bbw5 is sometimes unstable (OOM) - switch centos-7 bintar to run on hz-bbw7, for the same reason as above CMAKE: - introduce %(kw:additional_args)s - centos7 bintar still needs -DCURSES_NEED_NCURSES=1 flag because curses -> ncurses symlink is missing on the image. That's not the case for almalinux-8-bintar - already released packages have galera shared library included. The library is already present in the almalinux-8-bintar container image so include it as extra files for CPACK
There is no need for eigen3 since the MariaDB Vector code did not show significant improvements.
RazvanLiviuVarzaru
force-pushed
the
feature/new-bintar
branch
from
20046a3 to
50555c5
Compare
Thanks for your help @cvicentiu |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
AlmaLinux 8 was chosen as the oldest non-eol distro for a new bintar which will build server versions
>= 11.7Buildbot configuration changes will follow after this PR is merged. (new builder,worker).
A dedicated workflow was defined, with a
workflow_calldirective to allow us to rebuild the image anytime we like.About bintar.Dockerfile
About static libs:
-devel packages(they provide headers) then to compile the static library and move it in a path used then by CMAKETesting:
LDD