diff --git a/content/modules/ROOT/assets/images/modeler-8.png b/content/modules/ROOT/assets/images/modeler-8.png index 94c4f67..cf0eec7 100644 Binary files a/content/modules/ROOT/assets/images/modeler-8.png and b/content/modules/ROOT/assets/images/modeler-8.png differ diff --git a/content/modules/ROOT/pages/12-AI-chatbot-flow.adoc b/content/modules/ROOT/pages/12-AI-chatbot-flow.adoc index 76c6291..9a7c3fd 100644 --- a/content/modules/ROOT/pages/12-AI-chatbot-flow.adoc +++ b/content/modules/ROOT/pages/12-AI-chatbot-flow.adoc @@ -307,11 +307,11 @@ Click *Yes* for periodically run "git fetch"? image::chatbot-51.png[width=471,height=160] -Click back to RHDH and the *CI* tab to see the Trusted Application Pipeline running +Click back to Developer Hub and the *CI* tab to see the Trusted Application Pipeline running image::chatbot-52.png[] -This process takes between two minutes to four minutes. Details about this pipeline are included in module *Pipeline Exploration* +This process takes between two minutes to four minutes. Details about this pipeline are included in module *6. Pipeline Exploration*. image::chatbot-53.png[] @@ -326,7 +326,9 @@ image::chatbot-55.png[] That URL might be shared with your audience so they can interact with your new LLM-infused application. Note: Make sure load test your application a bit before sharing with a large audience. -=== Model inside of RHDH +Promotion to -pre-prod and -prod namespaces is covered in *7. Release and Promotion* + +=== Model inside of Developer Hub As a developer, you can also interact directly with the model server's API. @@ -346,10 +348,12 @@ Click on *Definition* image::chatbot-59.png[] -This is an OpenAPI definition, specifically the vLLM API that is serving the model. +This is an OpenAPI definition (think Swagger, not to be confused with OpenAI the creators of ChatGPT), specifically the vLLM API that is serving the model. image::chatbot-60.png[] Narrator: Now it is time to change hats. Up to this point we have been acting more like an enterprise application developer. The coder who sees models as API endpoints and who focuses on integrating LLMs with existing enterprise systems. +Let's switch into modeler mode. + diff --git a/content/modules/ROOT/pages/13-AI-model-exploration.adoc b/content/modules/ROOT/pages/13-AI-model-exploration.adoc index 2927a97..42d508a 100644 --- a/content/modules/ROOT/pages/13-AI-model-exploration.adoc +++ b/content/modules/ROOT/pages/13-AI-model-exploration.adoc @@ -32,7 +32,7 @@ Sources: *Data Science Projects*. The *marketingbot-ai* project was auto provisioned via gitops (ArgoCD) when the template was used earlier in the demonstration. -*Workbench*: A workbench, in the context provided, is an instance of a development and experimentation environment for data science projects. It serves as an isolated area where you can examine and work with Machine Learning (ML) models, work with data, and run programs. A workbench is not always required, but it is needed for most data science workflow tasks such as writing code to process data or training a model. You can create multiple workbenches within a project, each with its own configuration and purpose, to separate tasks or activities. A workbench is created by selecting a notebook image, which is optimized with the tools and libraries needed for model development. Supported IDEs include JupyterLab, code-server (Technology Preview), and RStudio (Technology Preview). +*Workbench*: A workbench is an instance of a development and experimentation environment for data science projects. It serves as an isolated area where you can examine and work with Machine Learning (ML) models, work with data, and run programs. A workbench is not always required, but it is needed for most data science workflow tasks such as writing code to process data or training a model. You can create multiple workbenches within a project, each with its own configuration and purpose, to separate tasks or activities. A workbench is created by selecting a notebook image, which is optimized with the tools and libraries needed for model development. Supported IDEs include JupyterLab, Visual Studio Code (Technology Preview), and RStudio (Technology Preview). Sources: @@ -42,7 +42,7 @@ Sources: * https://docs.redhat.com/en/documentation/red_hat_openshift_ai_self-managed/2.11/html/openshift_ai_tutorial_-_fraud_detection_example/index -*Deployed Models*: You can see the model server based on "vLLM custom" that is running, in a pod, the model that was originally downloaded from Huggingface.co. +*Deployed Models*: You can see the model server based on "vLLM custom" running, in a pod, the model that was originally downloaded from Huggingface.co. Narrator: As a modeler, I like working in the world of Python and Red Hat OpenShift AI gives me self-service access and the Workbench gives me instant access to a properly configured Notebook. @@ -64,7 +64,7 @@ image::modeler-6.png[] image::modeler-7.png[] -Narrator: Now I am in a Jupyter Notebook, run as a container, as a pod, on the OpenShift cluster. Where the underlying container image has been properly configured with the appropriate versions of python and needed libraries. +Narrator: Now I am in a Jupyter Notebook, run as a container, as a pod, on the OpenShift cluster. Where the underlying container image has been properly configured with the appropriate versions of python and the needed libraries. Double click on the folder *check-confidence-self-hosted-llm* to expand it out. And double click on *test_response_quality.py* @@ -74,9 +74,11 @@ One common thing to help manage the curation and lifecycle of a private LLM is t image::modeler-8.png[] -This code will print out "Response Quality OK" if the similiarity check is *<= 0.8* +This code will print out "Response Quality OK" if the similiarity check is not *less than or equal to 0.8* -Note: It does sometimes fail this test, you might lower the number to .7 +Note: It does some times fail this test, you might lower the number to .7 + +LLMs rarely return identical strings therefore the similiarity test is a way to deal with the varying responses. image::modeler-9.png[] @@ -84,12 +86,12 @@ image::modeler-9.png[] Double click on *confidence-check.pipeline* -Narrator: A MLOps pipeline, based on Kubeflow, can be used to create a scheduled set of checks/tests against the LLM. In the case of the confidence-check.pipeline, it tests for quality, response time and also has a simple identity verification step. - -image::modeler-10.png[] +Narrator: A MLOps pipeline, based on Kubeflow, can be used to create a scheduled set of tests against the LLM. In the case of the confidence-check.pipeline, it tests for quality, response time and also has a simple identity verification step. Click the "run" button +image::modeler-10.png[] + image::modeler-11.png[] Click *OK* @@ -100,7 +102,7 @@ Click *Run Details* image::modeler-13.png[] -And if you do get a failure on the quality test, it is very likely to be that .8 bar is too high. LLMs are non-deterministic their responses are almost always unique per invocation, even if the input prompt is identical. +And if you do get a failure on the quality test, it is very likely to be that .8 bar is too high. LLMs have non-deterministic their responses are almost always unique per invocation, even if the input prompt is identical. image::modeler-14.png[] @@ -132,19 +134,19 @@ Narrator: Now, how did this model come to life? If we scroll down toward the bot image::modeler-17.png[] -Click *Cancel* +Click *Cancel* as we do not need to make any changes Click *Data connections* and *Edit data connection* for marketingbot-models. image::modeler-18.png[] -You can see the endpoint is mapped to minio, an in-cluster, S3 alternative. +You can see the endpoint is mapped to Minio, an in-cluster, S3 solution. And the bucket is called *models*. image::modeler-19.png[] -These models are simply sitting in storage and are visible using your S3 browser, in the case of Minio, it is a nice GUI for viewing the artifacts. +These models are simply sitting in storage and are visible using your S3 browser, in the case of Minio, it has a nice GUI for viewing the artifacts. image::modeler-20.png[]