New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

enhancement: add monitoring test to check rhoai dashboard metrics by code #2175

Merged

CFSNM merged 1 commit into red-hat-data-services:master from CFSNM:ods-195-dashboard-metrics-code

Jan 17, 2025

+42 −1

Contributor

CFSNM commented Jan 13, 2025

Related JIRA: https://issues.redhat.com/browse/RHOAIENG-13261

CFSNM self-assigned this

github-advanced-security bot found potential problems

View reviewed changes

ods_ci/tests/Tests/0100__platform/0102__monitor_and_manage/0101__metrics/0101__metrics.robot

@@ @@ -123,6 +122,33 @@ @@
                   List Should Contain Value    ${targets}    user_facing_endpoints_status_rhods_dashboard
                   List Should Contain Value    ${targets}    user_facing_endpoints_status_workbenches
+              Test RHOAI Dashboard Metrics By Code Are Defined

Check warning

Code scanning / Robocop

Test case '{{ test_name }}' is too long ({{ test_length }}/{{ allowed_length }}) Warning test

Test case 'Test RHOAI Dashboard Metrics By Code Are Defined' is too long (40/20)

ods_ci/tests/Tests/0100__platform/0102__monitor_and_manage/0101__metrics/0101__metrics.robot Fixed Show fixed Hide fixed

ods_ci/tests/Tests/0100__platform/0102__monitor_and_manage/0101__metrics/0101__metrics.robot Fixed Show fixed Hide fixed

ods_ci/tests/Tests/0100__platform/0102__monitor_and_manage/0101__metrics/0101__metrics.robot Fixed Show fixed Hide fixed

ods_ci/tests/Tests/0100__platform/0102__monitor_and_manage/0101__metrics/0101__metrics.robot Fixed Show fixed Hide fixed

ods_ci/tests/Tests/0100__platform/0102__monitor_and_manage/0101__metrics/0101__metrics.robot Fixed Show fixed Hide fixed

ods_ci/tests/Tests/0100__platform/0102__monitor_and_manage/0101__metrics/0101__metrics.robot Fixed Show fixed Hide fixed

ods_ci/tests/Tests/0100__platform/0102__monitor_and_manage/0101__metrics/0101__metrics.robot Fixed Show fixed Hide fixed

ods_ci/tests/Tests/0100__platform/0102__monitor_and_manage/0101__metrics/0101__metrics.robot Fixed Show fixed Hide fixed

ods_ci/tests/Tests/0100__platform/0102__monitor_and_manage/0101__metrics/0101__metrics.robot Fixed Show fixed Hide fixed

Contributor

github-actions bot commented Jan 13, 2025 •

edited

Loading

Robot Results

✅ Passed	❌ Failed	⏭️ Skipped	Total	Pass %
599	0	0	599	100

CFSNM force-pushed the ods-195-dashboard-metrics-code branch from 4af3679 to 8a84e2e Compare

January 13, 2025 12:15

github-advanced-security bot found potential problems

View reviewed changes

ods_ci/tests/Tests/0100__platform/0102__monitor_and_manage/0101__metrics/0101__metrics.robot

@@ @@ -123,6 +122,48 @@ @@
                   List Should Contain Value    ${targets}    user_facing_endpoints_status_rhods_dashboard
                   List Should Contain Value    ${targets}    user_facing_endpoints_status_workbenches
+              Test RHOAI Dashboard Metrics By Code Are Defined

Check warning

Code scanning / Robocop

Test case '{{ test_name }}' has too many keywords inside ({{ keyword_count }}/{{ max_allowed_count }}) Warning test

Test case 'Test RHOAI Dashboard Metrics By Code Are Defined' has too many keywords inside (19/10)

ods_ci/tests/Tests/0100__platform/0102__monitor_and_manage/0101__metrics/0101__metrics.robot Fixed Show fixed Hide fixed

ods_ci/tests/Tests/0100__platform/0102__monitor_and_manage/0101__metrics/0101__metrics.robot Fixed Show fixed Hide fixed

ods_ci/tests/Tests/0100__platform/0102__monitor_and_manage/0101__metrics/0101__metrics.robot Fixed Show fixed Hide fixed

ods_ci/tests/Tests/0100__platform/0102__monitor_and_manage/0101__metrics/0101__metrics.robot Fixed Show fixed Hide fixed

ods_ci/tests/Tests/0100__platform/0102__monitor_and_manage/0101__metrics/0101__metrics.robot Fixed Show fixed Hide fixed

ods_ci/tests/Tests/0100__platform/0102__monitor_and_manage/0101__metrics/0101__metrics.robot Fixed Show fixed Hide fixed

ods_ci/tests/Tests/0100__platform/0102__monitor_and_manage/0101__metrics/0101__metrics.robot Fixed Show fixed Hide fixed

ods_ci/tests/Tests/0100__platform/0102__monitor_and_manage/0101__metrics/0101__metrics.robot Fixed Show fixed Hide fixed

ods_ci/tests/Tests/0100__platform/0102__monitor_and_manage/0101__metrics/0101__metrics.robot Fixed Show fixed Hide fixed

CFSNM requested review from MarianMacik and asanzgom

January 13, 2025 12:16

CFSNM force-pushed the ods-195-dashboard-metrics-code branch 2 times, most recently from c7d394b to 9974835 Compare

January 13, 2025 14:53

github-advanced-security bot found potential problems

View reviewed changes

ods_ci/tests/Tests/0100__platform/0102__monitor_and_manage/0101__metrics/0101__metrics.robot Fixed Show fixed Hide fixed

ods_ci/tests/Tests/0100__platform/0102__monitor_and_manage/0101__metrics/0101__metrics.robot Fixed Show fixed Hide fixed

ods_ci/tests/Tests/0100__platform/0102__monitor_and_manage/0101__metrics/0101__metrics.robot Fixed Show fixed Hide fixed

ods_ci/tests/Tests/0100__platform/0102__monitor_and_manage/0101__metrics/0101__metrics.robot Fixed Show fixed Hide fixed

CFSNM force-pushed the ods-195-dashboard-metrics-code branch 9 times, most recently from 0419bfc to 6494a14 Compare

January 16, 2025 10:34

github-advanced-security bot found potential problems

View reviewed changes

ods_ci/tests/Resources/Page/ODH/JupyterHub/JupyterLabLauncher.robot Outdated

    
            @@ -178,10 +178,9 @@
          
                  Should Match    "${ls_server}"    "${EMPTY}"

              Get User Notebook Pod Name

                [Documentation]   Returns notebook pod name for given username  (e.g. for user ldap-admin1 it will be jupyterhub-nb-ldap-2dadmin1)

                [Documentation]   Returns notebook pod name for given username  (e.g. for user ldap-admin1 it will be jupyter-nb-ldap-2dadmin1-0)

Check warning

Code scanning / Robocop

Line is too long ({{ line_length }}/{{ allowed_length }}) Warning test

Line is too long (131/120)

ods_ci/tests/Tests/0200__rhoai_upgrade/0201__pre_upgrade.robot Outdated

+                  # Get the notebook pod creation timestamp
+                  ${notebook_pod_name}=    Get User Notebook Pod Name    ${TEST_USER2.USERNAME}
+                  ${return_code}    ${ntb_creation_timestamp} =    Run And Return Rc And Output

Check warning

Code scanning / Robocop

The assignment sign is not consistent within the file. Expected '{{ expected_sign }}' but got '{{ actual_sign }}' instead Warning test

The assignment sign is not consistent within the file. Expected '=' but got ' =' instead

ods_ci/tests/Tests/0200__rhoai_upgrade/0201__pre_upgrade.robot Outdated

+                  Should Be Equal As Integers     ${return_code}    0    msg=${ntb_creation_timestamp}
+                  # Save the timestamp to the OpenShift ConfigMap so it can be used in test in the next phase
+                  ${return_code}    ${cmd_output} =    Run And Return Rc And Output

Check warning

Code scanning / Robocop

The assignment sign is not consistent within the file. Expected '{{ expected_sign }}' but got '{{ actual_sign }}' instead Warning test

The assignment sign is not consistent within the file. Expected '=' but got ' =' instead

ods_ci/tests/Tests/0200__rhoai_upgrade/0201__pre_upgrade.robot Outdated

                   [Documentation]    Basic suite setup
                   Set Library Search Order    SeleniumLibrary
                   RHOSi Setup
+                  # Prepare a namespace for storing values that should be shared between different upgrade test phases
+                  # 1. if the namespace exists already, let's remove it
+                  ${return_code}    ${cmd_output} =    Run And Return Rc And Output

Check warning

Code scanning / Robocop

The assignment sign is not consistent within the file. Expected '{{ expected_sign }}' but got '{{ actual_sign }}' instead Warning test

The assignment sign is not consistent within the file. Expected '=' but got ' =' instead

ods_ci/tests/Tests/0200__rhoai_upgrade/0201__pre_upgrade.robot Outdated

+                  ...    oc delete namespace --wait --ignore-not-found ${UPGRADE_NS}
+                  Should Be Equal As Integers     ${return_code}    0    msg=${cmd_output}
+                  # 2. create the namespace now
+                  ${return_code}    ${cmd_output} =    Run And Return Rc And Output

Check warning

Code scanning / Robocop

The assignment sign is not consistent within the file. Expected '{{ expected_sign }}' but got '{{ actual_sign }}' instead Warning test

The assignment sign is not consistent within the file. Expected '=' but got ' =' instead

..._serving/1009__model_serving_triton_on_kserve/1009__model_serving_triton_on_kserve_api.robot Outdated

@@ @@ -210,6 +212,49 @@ @@
                   ...    AND
                   ...    Run Keyword If    "${KSERVE_MODE}"=="RawDeployment"    Terminate Process    triton-process    kill=true
+              Test Onnx Model Grpc Inference Via API (Triton on Kserve)    # robocop: off=too-long-test-case

Check warning

Code scanning / Robocop

Test case '{{ test_name }}' has too many keywords inside ({{ keyword_count }}/{{ max_allowed_count }}) Warning test

Test case 'Test Onnx Model Grpc Inference Via API (Triton on Kserve)' has too many keywords inside (19/10)

..._serving/1009__model_serving_triton_on_kserve/1009__model_serving_triton_on_kserve_api.robot Outdated

+                  [Tags]    Tier2    RHOAIENG-16908
+                  Setup Test Variables    model_name=${ONNX_MODEL_NAME}    use_pvc=${FALSE}    use_gpu=${FALSE}
+                  ...    kserve_mode=${KSERVE_MODE}   model_path=triton/model_repository/
+                  Set Project And Runtime    runtime=${KSERVE_RUNTIME_REST_NAME}     protocol=${PROTOCOL_GRPC}     namespace=${test_namespace}

Check warning

Code scanning / Robocop

Line is too long ({{ line_length }}/{{ allowed_length }}) Warning test

Line is too long (128/120)

..._serving/1009__model_serving_triton_on_kserve/1009__model_serving_triton_on_kserve_api.robot Outdated

+                  Set Project And Runtime    runtime=${KSERVE_RUNTIME_REST_NAME}     protocol=${PROTOCOL_GRPC}     namespace=${test_namespace}
+                  ...    download_in_pvc=${DOWNLOAD_IN_PVC}    model_name=${ONNX_MODEL_NAME}
+                  ...    storage_size=100Mi    memory_request=100Mi
+                  ${requests}=    Create Dictionary    memory=1Gi

Check notice

Code scanning / Robocop

{{ create_keyword }} can be replaced with VAR Note test

Create Dictionary can be replaced with VAR

..._serving/1009__model_serving_triton_on_kserve/1009__model_serving_triton_on_kserve_api.robot Outdated

+                  Remove File    ${INFERENCESERVICE_FILLED_FILEPATH}
+                  Wait For Pods To Be Ready    label_selector=serving.kserve.io/inferenceservice=${ONNX_MODEL_NAME}
+                  ...    namespace=${test_namespace}
+                  ${pod_name}=  Get Pod Name    namespace=${test_namespace}

Check notice

Code scanning / Robocop

Variable '{{ name }}' is assigned but not used Note test

Variable '${pod_name}' is assigned but not used

..._serving/1009__model_serving_triton_on_kserve/1009__model_serving_triton_on_kserve_api.robot Outdated

+                  ...    namespace=${test_namespace}
+                  ${pod_name}=  Get Pod Name    namespace=${test_namespace}
+                  ...    label_selector=serving.kserve.io/inferenceservice=${ONNX_MODEL_NAME}
+                  ${valued}  ${host}=    Run And Return Rc And Output    oc get ksvc ${ONNX_MODEL_NAME}-predictor -o jsonpath='{.status.url}'

Check warning

Code scanning / Robocop

Line is too long ({{ line_length }}/{{ allowed_length }}) Warning test

Line is too long (127/120)

CFSNM force-pushed the ods-195-dashboard-metrics-code branch from 6494a14 to c97c64d Compare

January 16, 2025 11:09

MarianMacik previously approved these changes

View reviewed changes

ods_ci/tests/Tests/0100__platform/0102__monitor_and_manage/0101__metrics/0101__metrics.robot

Comment on lines +163 to +164

		Should Be True ${metrics_by_code_5xx} == ${metrics_5xx}
		Should Be True ${metrics_by_code_2xx} == ${metrics_2xx}

Contributor

MarianMacik Jan 16, 2025

This is more testing Prometheus aggregation than the actual metrics as this is the same metric just queried multiple times with different filters but I guess that is the only meaningful check we can do that there is any data. :)

Contributor Author

CFSNM Jan 16, 2025

Yeah, know its a bit redundant, but is the only check that we can do using these data

openshift-ci bot assigned MarianMacik

openshift-ci bot added the lgtm label


          enhancement: add monitoring test to check rhoai dashboard metrics by …

fbc25d8

…code

CFSNM dismissed MarianMacik’s stale review via

fbc25d8

January 16, 2025 12:46

CFSNM force-pushed the ods-195-dashboard-metrics-code branch from c97c64d to fbc25d8 Compare

January 16, 2025 12:46

openshift-ci bot removed the lgtm label

sonarqubecloud bot commented Jan 16, 2025

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

CFSNM requested a review from MarianMacik

January 16, 2025 12:54

asanzgom approved these changes

View reviewed changes

openshift-ci bot assigned asanzgom

openshift-ci bot added the lgtm label

CFSNM enabled auto-merge

January 16, 2025 15:37

MarianMacik approved these changes

View reviewed changes

CFSNM merged commit c8a057d into red-hat-data-services:master

11 of 12 checks passed

openshift-ci bot commented Jan 17, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: asanzgom, CFSNM, MarianMacik

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

CFSNM deleted the ods-195-dashboard-metrics-code branch

January 17, 2025 12:09

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lgtm