az containerapp compose create is stuck due to a specific docker compose file

[horihiro]

Describe the bug

I tried az containerapp compose create with the following docker compose file, then the command was stuck.
https://github.com/langgenius/dify/blob/main/docker/docker-compose.yaml

It seems that loading yaml fails because this log doesn't display even with --debug.

azure-cli/src/azure-cli/azure/cli/command_modules/containerapp/custom.py

Line 4927 in f0b5572

logger.info(parsed_compose_file)

Related command

az containerapp compose create

Errors

No error message because the command is stuck

Issue script & Debug output

No error message

Expected behavior

The container apps would be created related the YAML file

Environment Summary

azure-cli 2.68.0

core 2.68.0
telemetry 1.1.0

Extensions:
authV2 0.1.3
bastion 1.3.1
devcenter 6.2.0
spring 1.6.4
ssh 2.0.6

Dependencies:
msal 1.31.1
azure-mgmt-resource 23.1.1

Python location '/opt/az/bin/python3'
Extensions directory '/home/horihiro/.azure/cliextensions'

Python (Linux) 3.12.8 (main, Jan 8 2025, 03:38:17) [GCC 11.4.0]

Legal docs and information: aka.ms/AzureCliLegal

Your CLI is up-to-date.

Additional context

No response

[yonzhan]

Thank you for opening this issue, we will look into it.

[horihiro]

The cause of this issue is smurawski/pycomposefile#37, and the latest version also has the same issue.
To fix this, pycomposefile needs to be replaced with a newer version if it is released, or with another package that has the same functionality.

[Greedygre]

Hi @horihiro
I can repro this issue.

I think this behavior is the same as this issue: Azure/azure-cli-extensions#7007 (comment)

This is because the variables and variable values are not set, after executing the assignment statement, it can execute normally.
My suggestion is please make sure all the variables and variable values have been set.

[horihiro]

Thanks @Greedygre
Do you mean you cannot repro this even with https://github.com/langgenius/dify/blob/main/docker/docker-compose.yaml ?

[Greedygre]

Thanks @Greedygre Do you mean you cannot repro this even with https://github.com/langgenius/dify/blob/main/docker/docker-compose.yaml ?

Sorry, I can repro it with https://github.com/langgenius/dify/blob/main/docker/docker-compose.yaml, my previous reply has typo error, I have updated it.

I think this behavior is the same as this issue: Azure/azure-cli-extensions#7007 (comment)

This is because the variables and variable values are not set, after executing the assignment statement, it can execute normally.
My suggestion is please make sure all the variables and variable values have been set.

[horihiro]

Thanks.
I also think these are same issues.

I suggest this behavior is describled as limitation of this command if the variable values must be set, because Docker Compose spec allows this.
https://docs.docker.com/compose/how-tos/environment-variables/variable-interpolation/#interpolation-syntax

[Greedygre]

Thanks. I also think these are same issues.

I suggest this behavior is describled as limitation of this command if the variable values must be set, because Docker Compose spec allows this. https://docs.docker.com/compose/how-tos/environment-variables/variable-interpolation/#interpolation-syntax

Hi @horihiro
I did more tests base on the yaml file, It's not that all of it is not supported, just that some of it is not supported.

I found following code make the parse stuck:

1. WEB_API_CORS_ALLOW_ORIGINS: ${WEB_API_CORS_ALLOW_ORIGINS:-*}
2. CONSOLE_CORS_ALLOW_ORIGINS: ${CONSOLE_CORS_ALLOW_ORIGINS:-*}
3. NGINX_SERVER_NAME: ${NGINX_SERVER_NAME:-_}
4. OPENSEARCH_JAVA_OPTS: -Xms${OPENSEARCH_JAVA_OPTS_MIN:-512m} -Xmx${OPENSEARCH_JAVA_OPTS_MAX:-1024m}
5. ports:
   • '${EXPOSE_NGINX_PORT:-80}:${NGINX_PORT:-80}'

Base on 4 and 5, maybe it cannot parse one line include 2 vars.

Others I did some test.
For example all vars in x-shared-env: &shared-api-worker-env, only these 2 is not supported.

WEB_API_CORS_ALLOW_ORIGINS: ${WEB_API_CORS_ALLOW_ORIGINS:-*}
CONSOLE_CORS_ALLOW_ORIGINS: ${CONSOLE_CORS_ALLOW_ORIGINS:-*}

some type for example REDISCLI_AUTH: ${REDIS_PASSWORD:-difyai123456} works fine.

For 5,

I tested change to - '${EXPOSE_NGINX_PORT:-80}' or - 80 or - 80:80 or - ${EXPOSE_NGINX_PORT:-9200}:9200, it can work :

ports:
      - '${EXPOSE_NGINX_PORT:-80}'

This code is

azure-cli/src/azure-cli/azure/cli/command_modules/containerapp/custom.py

Line 4926 in f0b5572

parsed_compose_file = ComposeFile(compose_yaml)

It is the limitation of pycomposefile I think. I changed to 0.3.32 (the latest version) locally, It still stuck, which means the pattern var format are not supported yet.

[horihiro]

Yes I reached same causes, so I filed issue smurawski/pycomposefile#37 and PR smurawski/pycomposefile#38
I hope this PR gets merged.