Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gpu-memory is just for a gpu memery claim? #14

Open
hougangliu opened this issue Jun 8, 2021 · 2 comments
Open

gpu-memory is just for a gpu memery claim? #14

hougangliu opened this issue Jun 8, 2021 · 2 comments
Assignees

Comments

@hougangliu
Copy link

hougangliu commented Jun 8, 2021

for now, “volcano.sh/gpu-memory” can essentially only set 3 environment variables based on https://github.com/volcano-sh/devices/blob/master/pkg/plugin/nvidia/server.go#L324 to tell user how much gpu memory he can use in current container?

for example, when set “volcano.sh/gpu-memory: 1024”, environment variables looks like below in container, in fact, process in this container can use at most 4096M gpu memory, even VOLCANO_GPU_ALLOCATED is 1024?

NVIDIA_VISIBLE_DEVICES: “0” 
VOLCANO_GPU_ALLOCATED: “1024” 
VOLCANO_GPU_TOTAL: “4096”

can anyone help me confirm if I understand correctly?

@Thor-wl
Copy link
Contributor

Thor-wl commented Jun 9, 2021

/cc @wpeng102

@wpeng102
Copy link
Member

Currently it only support schedule pod base on pod's “volcano.sh/gpu-memory”. Not support memory hard limit and isolation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants