Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create builder_cache repo before first build #27

Open
ashpect opened this issue Aug 18, 2024 · 11 comments
Open

Create builder_cache repo before first build #27

ashpect opened this issue Aug 18, 2024 · 11 comments
Labels
good first issue Good for newcomers

Comments

@ashpect
Copy link
Contributor

ashpect commented Aug 18, 2024

The code works throwing warning if we do not create one, as pushing cache after the first build automatically creates one, but would be safer to build before using a small script.

@ashpect
Copy link
Contributor Author

ashpect commented Aug 26, 2024

Todo : mark as good first issue

@viferga viferga added the good first issue Good for newcomers label Aug 27, 2024
@viferga
Copy link
Member

viferga commented Aug 27, 2024

Todo : mark as good first issue

Done.

@SaranshBaniyal
Copy link
Contributor

Hey @ashpect! Should I give this a try?

@viferga
Copy link
Member

viferga commented Nov 21, 2024

Is this related to this: #38 ?

@SaranshBaniyal
Copy link
Contributor

@viferga I am not sure, is this the reason why the CI is failing in #38 ? @ashpect can you please elaborate this issue a bit more or maybe point me out to some resource to look into to better understand this

@ashpect
Copy link
Contributor Author

ashpect commented Nov 24, 2024

no this is not related to #38. the idea here is simple , basically when you initiate the registry, it doesn't have any repo inside obviously, now this isn't a issue if you push a image to a repo not existing (a repo is created by default when you push and it doesn't exist), the issue comes if you pull, it will throw error obviously.
Now if you look at builder.sh, to make things generic we have added both export and import registry

${BUILDER_BINARY} $@ | ${BUILDKIT_COMMAND} build \

Now take a look at test.sh, the startup tests don't need to pull anything as it's the startup when we push everything, but we still give some import registry cause we need to parse into Dockerfile.
And it isn't a breaking change and ci works fine anyways, it just throws an error that cache import from this repo failed an continues with the command and etc.
So it will be good and overall solid code if we have some way to either add a repo initialisation command which can then be triggered after setup and takes both export and import registry as arguments cause even if it doesn't break now, in future it might if buildkit decides to makes this error as major terminated error and its anyways good to write a whole clean flow without errors.

@ashpect
Copy link
Contributor Author

ashpect commented Nov 24, 2024

It's a standalone minor issue, nothing breaking so you can try it if you want and let me know if you have any further questions. @SaranshBaniyal

@SaranshBaniyal
Copy link
Contributor

@ashpect should I call the initialize function from within the setupRegistry() function or should I call it in startupTests(). Which one would make more sense?

@ashpect
Copy link
Contributor Author

ashpect commented Dec 18, 2024

@SaranshBaniyal Since buildkit takes import reg in all arguments, it would be better to iniialize it when you create registry, inside setupRegistry

@SaranshBaniyal
Copy link
Contributor

SaranshBaniyal commented Dec 18, 2024

The push refers to repository [registry:5000/metacall/builder_startup] Get "https://registry:5000/v2/": dial tcp: lookup registry on 127.0.0.53:53: server misbehaving

I am getting this while testing with my solution, do you have any idea what might be causing this.
Here is a snippet of my solution:

setupRegistry(){
	
	${DOCKER_CMD} up -d registry

	while [ ! "$(docker inspect --format '{{json .State.Health.Status }}' metacall_builder_registry)" = "\"healthy\"" ]; do
		sleep 5
	done

	# To Pre-initialize Import and Export Registry
	initializeRegistry "registry:5000" "metacall/builder_startup"
	initializeRegistry "registry:5000" "metacall/builder_cache"
}
initializeRegistry() {
	local registry=$1
	local repo_name=$2

	docker pull busybox
	docker tag busybox ${registry}/${repo_name}:dummy
	docker push ${registry}/${repo_name}:dummy
}

@ashpect
Copy link
Contributor Author

ashpect commented Jan 4, 2025

First does one really need to push something to a regsitry to make it ? Iing makes one by default, but try to find a command that creates one without pushing anything (the reason is we'll be pulling everything in builder cache) - it works by pushing for now as well, you can do that later.

while [ ! "$(docker inspect --format '{{json .State.Health.Status }}' metacall_builder_registry)" = ""healthy"" ]; do

here we enable docker to inspect the health of the registry
so if this passes, it means the registry should be accessibble
and if you check docker-compose, its 5000 port is mapped to device's 5000
so try localhost:5000 once
and also if you see we dont have certs as well, so you should try with http://local/...

try all this and lemme know if it works
also after you setup registry you could just put a simple curl/wget or whatever to the regsitry address manually in terminal to confirm its accessible by that and then add that url here

lemme know if you have any doubts @SaranshBaniyal

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

3 participants