Unexpected parameter "build_type" when running deploy-model-training-pipeline in CV pattern

[changwei0314]

Describe the bug or the issue that you are facing

I encounter an error when running deploy-model-training-pipeline.yaml

Steps/Code to Reproduce

I use the following configuration to generate the required code for deploying the pipeline.

Expected Output

Expect to run the deploy-model-training pipeline successfully.

Versions

main branch

Which platform are you using for deploying your infrastrucutre?

Azure DevOps (ADO)

If you mentioned Others, please mention which platformm are you using?

No response

What are you using for deploying your infrastrucutre?

Bicep

Are you using Azure ML CLI v2 or Azure ML Python SDK v2

Azure ML CLI v2

Describe the example that you are trying to run?

CV

Tasks

Preview Give feedback

1. Add Fix to remove build type from the CV model training pipeline for ADO.

[patcharees]

Hi. I am having this problem as well. Any workaround?

[setuc]

The template needs a fix

You will notice that the build_type was not setup or passed as a parameter to the register environment file that can be found here.
https://github.com/Azure/mlops-templates/blob/34dc9a07612cf920db5e1377a90f672b43ab8afd/.github/workflows/register-environment.yml

You can remove the build type or use a similar approach from classical template.

https://github.com/Azure/mlops-project-template/blob/acd9c8afbcd477cf85da819ace5e3ccd1dadd105/classical/aml-cli-v2/mlops/devops-pipelines/deploy-model-training-pipeline.yml#L51

I will need to retest the pipelines to fix the error. Give me some time to test it and address this issue.

[patcharees]

Hi @setuc,
Thanks for info. I found another issue. It is missing /templates/aml-cli-v2/register-dataset.yml in mlops-templates repo. Any workaround?

[setuc]

@patcharees let me review the CV templates to see what is causing these issues?

There is register dataset but CV doesn’t use that as we download the data on the fly. Do you have the error message that you get

[patcharees]

Below is my template mlops-project-template\cv\aml-cli-v2\mlops\devops-pipelines\deploy-model-training-pipeline.yml. You can see it requires register-dataset.yml. When I run this pipeline to deploy model training, I got this error

/mlops/devops-pipelines/deploy-model-training-pipeline.yml: File /templates/aml-cli-v2/register-dataset.yml not found in repository https://dev.azure.com/xxx/xxx/_git/mlops-templates branch refs/heads/main version fe222c36eaa8c0949d99e97e6ba5f17361f1f108.

as register-dataset.yml does not really exist in aml-cli-v2 folder

---

name: deploy-model-training-pipeline

variables:

• ${{ if eq(variables['Build.SourceBranchName'], 'main') }}:
  • template: ../../config-infra-prod.yml
• ${{ if ne(variables['Build.SourceBranchName'], 'main') }}:
  • template: ../../config-infra-dev.yml
• name: version
  value: aml-cli-v2

trigger: none

pool:
vmImage: ubuntu-20.04

resources:
repositories:
- repository: mlops-templates # Template Repo
name: mlops-templates
type: git
ref: main

stages:

• stage: DeployTrainingPipeline
  displayName: Deploy Training Pipeline
  jobs:
  • job: DeployTrainingPipeline
    steps:
    • checkout: self
      path: s/
    • task: Bash@3
      displayName: "Create checkout repository folder(s)"
      inputs:
      targetType: "inline"
      script: |
      set -e
      mkdir "$(Build.Repository.Name)"
      mkdir "mlops-templates"
    • checkout: mlops-templates
      path: s/templates/
    • template: templates/${{ variables.version }}/install-az-cli.yml@mlops-templates
    • template: templates/${{ variables.version }}/install-aml-cli.yml@mlops-templates
    • template: templates/${{ variables.version }}/connect-to-workspace.yml@mlops-templates
    • template: templates/${{ variables.version }}/create-compute.yml@mlops-templates
      parameters:
      cluster_name: gpu-cluster
      size: Standard_NC6
      min_instances: 0
      max_instances: 1
      cluster_tier: dedicated
    • template: templates/${{ variables.version }}/register-environment.yml@mlops-templates
      parameters:
      build_type: docker
      environment_name: nvidia_pytorch # Not used for docker builds
      environment_file: mlops/azureml/train/train-env.yaml
    • template: templates/${{ variables.version }}/register-dataset.yml@mlops-templates
      parameters:
      data_type: training
      environment_file: mlops/azureml/train/create_stanford_dogs_dataset.yaml
    • template: templates/${{ variables.version }}/run-pipeline.yml@mlops-templates
      parameters:
      pipeline_file: mlops/azureml/train/pipeline.yaml
      experiment_name: $(environment)cv_train$(Build.SourceBranchName)
      display_name: $(environment)cv_run$(Build.BuildID)

[Dan-RT]

Hi guys, I am facing the same bug with the same settings, @setuc did you get the chance to look at this issue?
@changwei0314 @patcharees did one of you figure out a workaround? @changwei0314 I saw you have a similar project but using terraform and github

[setuc]

@Dan-RT Try removing build-type from the line 58 here

https://github.com/Azure/mlops-project-template/blob/acd9c8afbcd477cf85da819ace5e3ccd1dadd105/cv/aml-cli-v2/mlops/devops-pipelines/deploy-model-training-pipeline.yml#L58

I will issue a patch for it.

[changwei0314]

Hi @Dan-RT, If you don't necessarily need to use Bicep and Azure DevOps, I would suggest referring to the guidance provided by this repository.

I have successfully built an MLOps engine using this guidance. However, please note that this project utilizes Terraform and GitHub Actions.

[setuc]

@changwei0314 @Dan-RT Is Bicep really a priority for you?