Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migration of Keras Rest Protocol test on triton for Kserve (UI -> API) #2182

Merged
merged 3 commits into from
Jan 20, 2025
Merged

Migration of Keras Rest Protocol test on triton for Kserve (UI -> API) #2182

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b5bfbd5
Migration of Keras Rest Protocol test on triton for Kserve (UI -> API)
rpancham Jan 17, 2025
f778865
Merge branch 'master' into keras_api_rest
rpancham Jan 17, 2025
a7ab88f
Merge branch 'master' into keras_api_rest
tarukumar Jan 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions ods_ci/tests/Resources/Files/triton/kserve-triton-keras-rest-input.json
View file
Open in desktop

Large diffs are not rendered by default.

48 changes: 46 additions & 2 deletions ...rving/1009__model_serving_triton_on_kserve/1009__model_serving_triton_on_kserve_api.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,9 @@
${KSERVE_RUNTIME_REST_NAME}= triton-kserve-runtime
${PYTORCH_MODEL_NAME}= resnet50
${INFERENCE_REST_INPUT_PYTORCH}= @tests/Resources/Files/triton/kserve-triton-resnet-rest-input.json
${EXPECTED_INFERENCE_REST_OUTPUT_FILE__PYTORCH}= tests/Resources/Files/triton/kserve-triton-resnet-rest-output.json
${EXPECTED_INFERENCE_REST_OUTPUT_FILE_PYTORCH}= tests/Resources/Files/triton/kserve-triton-resnet-rest-output.json

Check warning

Code scanning / Robocop

Line is too long ({{ line_length }}/{{ allowed_length }}) Warning test

Line is too long (121/120)
${INFERENCE_REST_INPUT_KERAS}= @tests/Resources/Files/triton/kserve-triton-keras-rest-input.json
${EXPECTED_INFERENCE_REST_OUTPUT_FILE_KERAS}= tests/Resources/Files/triton/kserve-triton-keras-rest-output.json
${PATTERN}= https:\/\/([^\/:]+)
${PROTOBUFF_FILE}= tests/Resources/Files/triton/grpc_predict_v2.proto

Expand Down Expand Up @@ -88,7 +90,7 @@
... Run Keyword If "${KSERVE_MODE}"=="RawDeployment" Terminate Process triton-process kill=true

Test Pytorch Model Rest Inference Via API (Triton on Kserve) # robocop: off=too-long-test-case
[Documentation] Test the deployment of python model in Kserve using Triton
[Documentation] Test the deployment of pytorch model in Kserve using Triton
[Tags] Tier2 RHOAIENG-16909
Setup Test Variables model_name=${PYTORCH_MODEL_NAME} use_pvc=${FALSE} use_gpu=${FALSE}
... kserve_mode=${KSERVE_MODE} model_path=triton/model_repository/
Expand Down Expand Up @@ -255,6 +257,48 @@
... isvc_names=${models_names} wait_prj_deletion=${FALSE} kserve_mode=${KSERVE_MODE}
... AND
... Run Keyword If "${KSERVE_MODE}"=="RawDeployment" Terminate Process triton-process kill=true

Check warning

Code scanning / Robocop

Trailing whitespace at the end of line Warning test

Trailing whitespace at the end of line
Test Keras Model Rest Inference Via API (Triton on Kserve) # robocop: off=too-long-test-case

Check warning

Code scanning / Robocop

Test case '{{ test_name }}' has too many keywords inside ({{ keyword_count }}/{{ max_allowed_count }}) Warning test

Test case 'Test Keras Model Rest Inference Via API (Triton on Kserve)' has too many keywords inside (12/10)
[Documentation] Test the deployment of keras model in Kserve using Triton
[Tags] Tier2 RHOAIENG-16911
Setup Test Variables model_name=${PYTORCH_MODEL_NAME} use_pvc=${FALSE} use_gpu=${FALSE}
... kserve_mode=${KSERVE_MODE} model_path=tritonkeras/model_repository/
Set Project And Runtime runtime=${KSERVE_RUNTIME_REST_NAME} protocol=${PROTOCOL} namespace=${test_namespace}

Check warning

Code scanning / Robocop

Line is too long ({{ line_length }}/{{ allowed_length }}) Warning test

Line is too long (123/120)
... download_in_pvc=${DOWNLOAD_IN_PVC} model_name=${PYTORCH_MODEL_NAME}
... storage_size=100Mi memory_request=100Mi
${requests}= Create Dictionary memory=1Gi

Check notice

Code scanning / Robocop

{{ create_keyword }} can be replaced with VAR Note test

Create Dictionary can be replaced with VAR
Compile Inference Service YAML isvc_name=${PYTORCH_MODEL_NAME}
... sa_name=models-bucket-sa
... model_storage_uri=${storage_uri}
... model_format=python serving_runtime=${KSERVE_RUNTIME_REST_NAME}
... version="1"
... limits_dict=${limits} requests_dict=${requests} kserve_mode=${KSERVE_MODE}
Deploy Model Via CLI isvc_filepath=${INFERENCESERVICE_FILLED_FILEPATH}
... namespace=${test_namespace}
# File is not needed anymore after applying
Remove File ${INFERENCESERVICE_FILLED_FILEPATH}
Wait For Pods To Be Ready label_selector=serving.kserve.io/inferenceservice=${PYTORCH_MODEL_NAME}
... namespace=${test_namespace}
${pod_name}= Get Pod Name namespace=${test_namespace}
... label_selector=serving.kserve.io/inferenceservice=${PYTORCH_MODEL_NAME}
${service_port}= Extract Service Port service_name=${PYTORCH_MODEL_NAME}-predictor protocol=TCP
... namespace=${test_namespace}
IF "${KSERVE_MODE}"=="RawDeployment"
Start Port-forwarding namespace=${test_namespace} pod_name=${pod_name} local_port=${service_port}
... remote_port=${service_port} process_alias=triton-process
END
${EXPECTED_INFERENCE_REST_OUTPUT_KERAS}= Load Json File
... file_path=${EXPECTED_INFERENCE_REST_OUTPUT_FILE_KERAS} as_string=${TRUE}
Verify Model Inference With Retries model_name=${PYTORCH_MODEL_NAME} inference_input=${INFERENCE_REST_INPUT_KERAS}

Check warning

Code scanning / Robocop

Line is too long ({{ line_length }}/{{ allowed_length }}) Warning test

Line is too long (123/120)
... expected_inference_output=${EXPECTED_INFERENCE_REST_OUTPUT_KERAS} project_title=${test_namespace}
... deployment_mode=Cli kserve_mode=${KSERVE_MODE} service_port=${service_port}
... end_point=/v2/models/${model_name}/infer retries=3
[Teardown] Run Keywords
... Clean Up Test Project test_ns=${test_namespace}
... isvc_names=${models_names} wait_prj_deletion=${FALSE} kserve_mode=${KSERVE_MODE}
... AND
... Run Keyword If "${KSERVE_MODE}"=="RawDeployment" Terminate Process triton-process kill=true


*** Keywords ***
Suite Setup
Expand Down
Loading