-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(docker): compile toolchains separately #368
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
5 tasks
github-actions
bot
added
feature
New feature or request
testing
Testing related
github_actions
Pull requests that update GitHub Actions code
labels
Oct 14, 2024
AtomicFS
force-pushed
the
feat/compile-toolchains-separatly
branch
24 times, most recently
from
October 15, 2024 13:40
22d585d
to
808affb
Compare
AtomicFS
force-pushed
the
feat/compile-toolchains-separatly
branch
10 times, most recently
from
October 17, 2024 13:03
af65761
to
f2f9442
Compare
AtomicFS
changed the title
feat: compile toolchains separately
feat(docker): compile toolchains separately
Oct 17, 2024
@MDr164 I think that this is ready for preview. Next PR will replace the in |
AtomicFS
force-pushed
the
feat/compile-toolchains-separatly
branch
2 times, most recently
from
October 21, 2024 14:09
e16041d
to
0cb3751
Compare
- ubuntu uses different yq parser which does act a bit oddly, this should make it act in more predictable way - add check to validate the JSON Signed-off-by: AtomicFS <[email protected]>
AtomicFS
force-pushed
the
feat/compile-toolchains-separatly
branch
3 times, most recently
from
October 26, 2024 18:58
3a78766
to
4a0f830
Compare
Signed-off-by: AtomicFS <[email protected]>
- I did try to pre-compile edk2 BaseTools, just like we do with coreobot, but there were simply too many problems - udk2017 uses Ubuntu Bionic (18.04) which is simply too old for GitHub actions, which require modern NodeJS - uploading almost 1GB of artifacts per edk2 variant is just too much for the CI and takes forever - since the BaseTools do not take that long to compile (under 1 minute), it will be better to run the compilation on the first use by the user Signed-off-by: AtomicFS <[email protected]>
Signed-off-by: AtomicFS <[email protected]>
Signed-off-by: AtomicFS <[email protected]>
AtomicFS
force-pushed
the
feat/compile-toolchains-separatly
branch
from
October 26, 2024 23:32
4a0f830
to
727199c
Compare
MDr164
reviewed
Oct 31, 2024
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks alright, gonna run some local tests once those updates are in
MDr164
approved these changes
Oct 31, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
docker
docker related
feature
New feature or request
github_actions
Pull requests that update GitHub Actions code
testing
Testing related
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
At the moment this is not used yet, but will be.
Basically I am moving the compilation of the tool-chains out of the Dockerfile, as I believe this to be the only reasonable way to achieve multi-arch docker containers.
There is a lot to comprehend, so here are some notes as to why:
u-root
andlinux
are OK to build the "normal way", however withedk2
andcoreboot
I get errors when trying to compile to tool-chains - I suspect the problem to be native emulation not supporting all of the instructions requiredx86
onx86
machine andarm64
onarm64
works fine, but buildingarm64
onx86
machine is brokenedk2
orcoreboot
multi-arch container on singlex86
machine as I canu-root
orlinux
x86
part onx86
machine andarm64
part onarm64
machine), publish them separately and then publish hand-crafted manifest connecting them together ... (seeThe hard way with docker manifest
in Multi-arch build and images, the simple waycoreboot
and speed up the build by one order of magnitude (normally they take around 1 hour to compile)UPDATE:
edk2
BaseTools
is not that simple, there are 2 big problems:edk2
that we have isudk2017
which is usingUbuntu Bionic
(18.04
) which is simply too old to run GitHub action inside it due to obsoleteNodeJS
(see GLIBC_2.28 not found)edk2
BaseTools
gets compile on the first use by the user, and since it does not take long, it should be fine I guess