![\"Google](\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\"){kind=logo}
Open in Colab\n", - " \n", - "
![\"Google](\"https://cloud.google.com/ml-engine/images/colab-enterprise-logo-32px.png\"){kind=logo}
Open in Colab Enterprise\n", - " \n", - "
![\"Vertex](\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\")
Open in Workbench\n", - " \n", - "
![\"GitHub](\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\"){kind=logo}
View on GitHub\n", - " \n", - "
| \n",
- " \n",
- " Open in Colab\n", - " \n", - " | \n",
- " \n",
- " \n",
- " Open in Colab Enterprise\n", - " \n", - " | \n",
- " \n",
- " \n",
- " Open in Workbench\n", - " \n", - " | \n",
- " \n",
- " \n",
- " View on GitHub\n", - " \n", - " | \n",
- "
![](\"https://storage.googleapis.com/github-repo/generative-ai/gemini/reasoning-engine/images/cloud-trace-agent.png\")
\n",
- "\n",
- "This notebook demonstrates how to:\n",
- "\n",
- "- **Learn Key Concepts**: Learn about the fundamental building blocks of tracing.\n",
- "- **Deploy Your Agent**: Make your tracing-enabled agent available in a production-like environment on Reasoning Engine.\n",
- "- **Enable Tracing**: Enable tracing in a simple agent\n",
- "- **Examine Traces**: Use the Cloud Console and Cloud Trace SDK to access and analyze a specific trace.\n",
- "\n",
- "By the end of this notebook, you'll be able to leverage tracing to build more robust and efficient AI agents on Vertex AI."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Concepts\n",
- "\n",
- "Here are some of the key concepts and terminology related to tracing, which will be helpful to understand as we explore traces generated by an agent in Reasoning Engine:\n",
- "\n",
- "Below is an example of a trace in JSON format, showing a single span. This span represents a call to a large language model (LLM). Notice how the trace data captures important details:\n",
- "\n",
- "### Example trace\n",
- "\n",
- "```json\n",
- "{\n",
- " \"name\": \"llm\",\n",
- " \"context\": {\n",
- " \"trace_id\": \"ed7b336d-e71a-46f0-a334-5f2e87cb6cfc\",\n",
- " \"span_id\": \"ad67332a-38bd-428e-9f62-538ba2fa90d4\"\n",
- " },\n",
- " \"span_kind\": \"LLM\",\n",
- " \"parent_id\": \"f89ebb7c-10f6-4bf8-8a74-57324d2556ef\",\n",
- " \"start_time\": \"2023-09-07T12:54:47.597121-06:00\",\n",
- " \"end_time\": \"2023-09-07T12:54:49.321811-06:00\",\n",
- " \"status_code\": \"OK\",\n",
- " \"status_message\": \"\",\n",
- " \"attributes\": {\n",
- " \"llm.input_messages\": [\n",
- " {\n",
- " \"message.role\": \"system\",\n",
- " \"message.content\": \"You are an expert Q&A system that is trusted around the world.\\nAlways answer the query using the provided context information, and not prior knowledge.\\nSome rules to follow:\\n1. Never directly reference the given context in your answer.\\n2. Avoid statements like 'Based on the context, ...' or 'The context information ...' or anything along those lines.\"\n",
- " },\n",
- " {\n",
- " \"message.role\": \"user\",\n",
- " \"message.content\": \"Hello?\"\n",
- " }\n",
- " ],\n",
- " \"output.value\": \"assistant: Yes I am here\",\n",
- " \"output.mime_type\": \"text/plain\"\n",
- " },\n",
- " \"events\": [],\n",
- "}\n",
- "```\n",
- "\n",
- "### Trace\n",
- "\n",
- "You can think of a [trace](https://opentelemetry.io/docs/concepts/signals/traces/) like a timeline of requests as they travel through your application. A trace is composed of individual spans, with the first span representing the overall request. Each span provides details about a specific operation within the request.\n",
- "\n",
- "### Span\n",
- "\n",
- "A [span](https://opentelemetry.io/docs/concepts/signals/traces/#spans) represents a single unit of work, like a function call or an interaction with an LLM. It captures information such as the operation's name, start and end times, and any relevant attributes (metadata). Spans can be nested, showing parent-child relationships between operations.\n",
- "\n",
- "### Span Attribute\n",
- "\n",
- "[Span attributes](https://opentelemetry.io/docs/concepts/signals/traces/#attributes) are key-value pairs that provide additional context about a span. For instance, an LLM span might have attributes like the model name, prompt text, and token count.\n",
- "\n",
- "### Span Kind\n",
- "\n",
- "[Span kind](https://opentelemetry.io/docs/concepts/signals/traces/#span-kind) categorizes the type of operation a span represents. Common kinds include:\n",
- "\n",
- "- `CHAIN`: Links between LLM application steps or the start of a request.\n",
- "- `LLM`: A call to a large language model.\n",
- "- `TOOL`: An interaction with an external tool (API, database, etc.).\n",
- "- `AGENT`: A reasoning block that combines LLM and tool interactions."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "61RBz8LLbxCR"
- },
- "source": [
- "## Get started"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "No17Cw5hgx12"
- },
- "source": [
- "### Install Vertex AI SDK and other required packages\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 1,
- "metadata": {
- "id": "tFy3H3aPgx12"
- },
- "outputs": [],
- "source": [
- "%pip install --upgrade --user --quiet \\\n",
- " \"google-cloud-aiplatform[langchain,reasoningengine]\" \\\n",
- " cloudpickle==3.0.0 \\\n",
- " pydantic==2.7.4 \\\n",
- " google-cloud-trace"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "R5Xep4W9lq-Z"
- },
- "source": [
- "### Restart runtime\n",
- "\n",
- "To use the newly installed packages in this Jupyter runtime, you must restart the runtime. You can do this by running the cell below, which restarts the current kernel.\n",
- "\n",
- "The restart might take a minute or longer. After it's restarted, continue to the next step."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 2,
- "metadata": {
- "id": "XRvKdaPDTznN"
- },
- "outputs": [],
- "source": [
- "import IPython\n",
- "\n",
- "app = IPython.Application.instance()\n",
- "app.kernel.do_shutdown(True)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "SbmM4z7FOBpM"
- },
- "source": [
- "| \n",
+ " \n",
+ " Open in Colab\n", + " \n", + " | \n",
+ " \n",
+ " \n",
+ " Open in Colab Enterprise\n", + " \n", + " | \n",
+ " \n",
+ " \n",
+ " Open in Workbench\n", + " \n", + " | \n",
+ " \n",
+ " \n",
+ " View on GitHub\n", + " \n", + " | \n",
+ "
\n",
+ "\n",
+ "This notebook demonstrates how to:\n",
+ "\n",
+ "- **Learn Key Concepts**: Learn about the fundamental building blocks of tracing.\n",
+ "- **Deploy Your Agent**: Make your tracing-enabled agent available in a production-like environment on Reasoning Engine.\n",
+ "- **Enable Tracing**: Enable tracing in a simple agent\n",
+ "- **Examine Traces**: Use the Cloud Console and Cloud Trace SDK to access and analyze a specific trace.\n",
+ "\n",
+ "By the end of this notebook, you'll be able to leverage tracing to build more robust and efficient AI agents on Vertex AI."
]
- },
- "execution_count": 13,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
- "source": [
- "remote_agent.query(\n",
- " input=\"\"\"\n",
- " Classify the following ticket into a category and route the customer accordingly:\n",
- " \n",
- " Support ticket text:\n",
- " I am unable to make any API calls and I need to report an outage in the system\n",
- " \"\"\",\n",
- ")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "id": "RYwmTcdgIlza",
- "jp-MarkdownHeadingCollapsed": true
- },
- "source": [
- "## Exploring traces in the Cloud Console\n",
- "\n",
- "The Cloud Trace console provides a powerful and intuitive visual interface for exploring trace data, including visualizing, filtering, and analyzing your traces.\n",
- "\n",
- "Accessing the Trace Console:\n",
- "\n",
- "- **Project-Level View**: To see all traces for your Google Cloud project (replace `your-project-id` with your actual project ID), go to: https://console.cloud.google.com/traces/list?project=your-project-id\n",
- "\n",
- "- **Specific Trace**: If you know the unique Trace ID for a specific trace you want to examine, you can view it directly (replace your-trace-id with the actual Trace ID): https://console.cloud.google.com/traces/list?project=your-project-id&tid=your-trace-id\n",
- "\n",
- "Features to Explore in the Console:\n",
- "\n",
- "- **Trace List**: View a list of traces, sorted by start time, along with summary information (duration, number of spans).\n",
- "- **Waterfall View**: Visualize the spans within a trace as a timeline, showing the duration of each operation and their relationships.\n",
- "- **Span Details**: Click on a span to view its attributes, including the input and output data, and any custom metadata you've added.\n",
- "- **Filtering and Search**: The console provides powerful options for filtering traces by time range, service, span name, and other criteria. You can also search for specific traces using keywords or attributes.\n",
- "\n",
- "For a detailed guide to working with traces in the console, refer to the [Cloud Trace documentation on finding traces](https://cloud.google.com/trace/docs/finding-traces). Experiment with the Cloud Trace console to gain a deeper understanding of your agent's behavior and how it's executing within Reasoning Engine."
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Working with traces using `pandas`\n",
- "\n",
- "For more programmatic analysis, you can use the pandas library to work with trace data. You'll fetch traces, convert them to DataFrames, and then use pandas' functionality to explore the trace data."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 14,
- "metadata": {},
- "outputs": [],
- "source": [
- "result = [\n",
- " r\n",
- " for r in client.list_traces(\n",
- " request=trace.types.ListTracesRequest(\n",
- " project_id=PROJECT_ID,\n",
- " # Return all traces containing `labels {key: \"openinference.span.kind\" value: \"AGENT\"}`\n",
- " filter=\"openinference.span.kind:AGENT\",\n",
- " )\n",
- " )\n",
- "]"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 15,
- "metadata": {},
- "outputs": [],
- "source": [
- "trace_data = client.get_trace(project_id=PROJECT_ID, trace_id=result[0].trace_id)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 16,
- "metadata": {},
- "outputs": [
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "d38b3d918526"
+ },
+ "source": [
+ "## Concepts\n",
+ "\n",
+ "Here are some of the key concepts and terminology related to tracing, which will be helpful to understand as we explore traces generated by an agent in Reasoning Engine:\n",
+ "\n",
+ "Below is an example of a trace in JSON format, showing a single span. This span represents a call to a large language model (LLM). Notice how the trace data captures important details:\n",
+ "\n",
+ "### Example trace\n",
+ "\n",
+ "```json\n",
+ "{\n",
+ " \"name\": \"llm\",\n",
+ " \"context\": {\n",
+ " \"trace_id\": \"ed7b336d-e71a-46f0-a334-5f2e87cb6cfc\",\n",
+ " \"span_id\": \"ad67332a-38bd-428e-9f62-538ba2fa90d4\"\n",
+ " },\n",
+ " \"span_kind\": \"LLM\",\n",
+ " \"parent_id\": \"f89ebb7c-10f6-4bf8-8a74-57324d2556ef\",\n",
+ " \"start_time\": \"2023-09-07T12:54:47.597121-06:00\",\n",
+ " \"end_time\": \"2023-09-07T12:54:49.321811-06:00\",\n",
+ " \"status_code\": \"OK\",\n",
+ " \"status_message\": \"\",\n",
+ " \"attributes\": {\n",
+ " \"llm.input_messages\": [\n",
+ " {\n",
+ " \"message.role\": \"system\",\n",
+ " \"message.content\": \"You are an expert Q&A system that is trusted around the world.\\nAlways answer the query using the provided context information, and not prior knowledge.\\nSome rules to follow:\\n1. Never directly reference the given context in your answer.\\n2. Avoid statements like 'Based on the context, ...' or 'The context information ...' or anything along those lines.\"\n",
+ " },\n",
+ " {\n",
+ " \"message.role\": \"user\",\n",
+ " \"message.content\": \"Hello?\"\n",
+ " }\n",
+ " ],\n",
+ " \"output.value\": \"assistant: Yes I am here\",\n",
+ " \"output.mime_type\": \"text/plain\"\n",
+ " },\n",
+ " \"events\": [],\n",
+ "}\n",
+ "```\n",
+ "\n",
+ "### Trace\n",
+ "\n",
+ "You can think of a [trace](https://opentelemetry.io/docs/concepts/signals/traces/) like a timeline of requests as they travel through your application. A trace is composed of individual spans, with the first span representing the overall request. Each span provides details about a specific operation within the request.\n",
+ "\n",
+ "### Span\n",
+ "\n",
+ "A [span](https://opentelemetry.io/docs/concepts/signals/traces/#spans) represents a single unit of work, like a function call or an interaction with an LLM. It captures information such as the operation's name, start and end times, and any relevant attributes (metadata). Spans can be nested, showing parent-child relationships between operations.\n",
+ "\n",
+ "### Span Attribute\n",
+ "\n",
+ "[Span attributes](https://opentelemetry.io/docs/concepts/signals/traces/#attributes) are key-value pairs that provide additional context about a span. For instance, an LLM span might have attributes like the model name, prompt text, and token count.\n",
+ "\n",
+ "### Span Kind\n",
+ "\n",
+ "[Span kind](https://opentelemetry.io/docs/concepts/signals/traces/#span-kind) categorizes the type of operation a span represents. Common kinds include:\n",
+ "\n",
+ "- `CHAIN`: Links between LLM application steps or the start of a request.\n",
+ "- `LLM`: A call to a large language model.\n",
+ "- `TOOL`: An interaction with an external tool (API, database, etc.).\n",
+ "- `AGENT`: A reasoning block that combines LLM and tool interactions."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "61RBz8LLbxCR"
+ },
+ "source": [
+ "## Get started"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "No17Cw5hgx12"
+ },
+ "source": [
+ "### Install Vertex AI SDK and other required packages\n"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {
+ "id": "tFy3H3aPgx12"
+ },
+ "outputs": [],
+ "source": [
+ "%pip install --upgrade --user --quiet \\\n",
+ " \"google-cloud-aiplatform[langchain,reasoningengine]\" \\\n",
+ " cloudpickle==3.0.0 \\\n",
+ " pydantic==2.7.4 \\\n",
+ " google-cloud-trace"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "R5Xep4W9lq-Z"
+ },
+ "source": [
+ "### Restart runtime\n",
+ "\n",
+ "To use the newly installed packages in this Jupyter runtime, you must restart the runtime. You can do this by running the cell below, which restarts the current kernel.\n",
+ "\n",
+ "The restart might take a minute or longer. After it's restarted, continue to the next step."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {
+ "id": "XRvKdaPDTznN"
+ },
+ "outputs": [],
+ "source": [
+ "import IPython\n",
+ "\n",
+ "app = IPython.Application.instance()\n",
+ "app.kernel.do_shutdown(True)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "SbmM4z7FOBpM"
+ },
+ "source": [
+ "| \n", - " | spanId | \n", - "name | \n", - "startTime | \n", - "endTime | \n", - "labels | \n", - "parentSpanId | \n", - "
|---|---|---|---|---|---|---|
| 0 | \n", - "12108864055932459456 | \n", - "AgentExecutor | \n", - "2024-06-29T04:33:40.426806016Z | \n", - "2024-06-29T04:33:49.886738176Z | \n", - "{'g.co/agent': 'opentelemetry-python 1.25.0; g... | \n", - "NaN | \n", - "
| 1 | \n", - "1514793182933739503 | \n", - "RunnableParallel<input,agent_scratchpad> | \n", - "2024-06-29T04:33:40.435836160Z | \n", - "2024-06-29T04:33:40.816612096Z | \n", - "{'g.co/agent': 'opentelemetry-python 1.25.0; g... | \n", - "10997585570109326486 | \n", - "
| 2 | \n", - "2419854474582761855 | \n", - "ChatPromptTemplate | \n", - "2024-06-29T04:33:40.988899840Z | \n", - "2024-06-29T04:33:40.990212864Z | \n", - "{'g.co/agent': 'opentelemetry-python 1.25.0; g... | \n", - "10997585570109326486 | \n", - "
| 3 | \n", - "883609395272937079 | \n", - "ChatVertexAI | \n", - "2024-06-29T04:33:41.174941184Z | \n", - "2024-06-29T04:33:43.206272Z | \n", - "{'llm.invocation_parameters': '{\"model_name\": ... | \n", - "10997585570109326486 | \n", - "
| 4 | \n", - "10997585570109326486 | \n", - "RunnableSequence | \n", - "2024-06-29T04:33:40.431934208Z | \n", - "2024-06-29T04:33:43.667400960Z | \n", - "{'g.co/agent': 'opentelemetry-python 1.25.0; g... | \n", - "12108864055932459456 | \n", - "
| \n", - " | spanId | \n", - "name | \n", - "startTime | \n", - "endTime | \n", - "labels | \n", - "parentSpanId | \n", - "
|---|---|---|---|---|---|---|
| 3 | \n", - "883609395272937079 | \n", - "ChatVertexAI | \n", - "2024-06-29T04:33:41.174941184Z | \n", - "2024-06-29T04:33:43.206272Z | \n", - "{'llm.invocation_parameters': '{\"model_name\": ... | \n", - "10997585570109326486 | \n", - "
| 14 | \n", - "1267786026295004320 | \n", - "ChatVertexAI | \n", - "2024-06-29T04:33:44.592414976Z | \n", - "2024-06-29T04:33:46.278656Z | \n", - "{'g.co/agent': 'opentelemetry-python 1.25.0; g... | \n", - "8170159531257482779 | \n", - "
| 22 | \n", - "15517842396867663478 | \n", - "ChatVertexAI | \n", - "2024-06-29T04:33:47.494297856Z | \n", - "2024-06-29T04:33:49.470009088Z | \n", - "{'g.co/agent': 'opentelemetry-python 1.25.0; g... | \n", - "4922396732366218475 | \n", - "
| \n", - " | llm.invocation_parameters | \n", - "g.co/agent | \n", - "output.mime_type | \n", - "llm.input_messages.0.message.content | \n", - "llm.output_messages.0.message.function_call_arguments_json | \n", - "output.value | \n", - "input.mime_type | \n", - "llm.input_messages.0.message.role | \n", - "input.value | \n", - "llm.output_messages.0.message.function_call_name | \n", - "... | \n", - "llm.input_messages.1.message.role | \n", - "llm.input_messages.2.message.role | \n", - "llm.input_messages.1.message.function_call_arguments_json | \n", - "llm.input_messages.1.message.function_call_name | \n", - "llm.input_messages.4.message.content | \n", - "llm.input_messages.3.message.role | \n", - "llm.input_messages.4.message.role | \n", - "llm.output_messages.0.message.content | \n", - "llm.input_messages.3.message.function_call_name | \n", - "llm.input_messages.3.message.function_call_arguments_json | \n", - "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 3 | \n", - "{\"model_name\": \"gemini-1.5-pro-001\", \"temperature... | \n", - "opentelemetry-python 1.25.0; google-cloud-trac... | \n", - "application/json | \n", - "\\n Classify the following ticket into a cat... | \n", - "{\"ticket_text\": \"I need to update my billing i... | \n", - "{\"generations\": [[{\"text\": \"\", \"generation_inf... | \n", - "application/json | \n", - "user | \n", - "{\"messages\": [[{\"lc\": 1, \"type\": \"constructor\"... | \n", - "classify_ticket | \n", - "... | \n", - "NaN | \n", - "NaN | \n", - "NaN | \n", - "NaN | \n", - "NaN | \n", - "NaN | \n", - "NaN | \n", - "NaN | \n", - "NaN | \n", - "NaN | \n", - "
| 14 | \n", - "{\"model_name\": \"gemini-1.5-pro-001\", \"temperature... | \n", - "opentelemetry-python 1.25.0; google-cloud-trac... | \n", - "application/json | \n", - "\\n Classify the following ticket into a cat... | \n", - "{\"category\": \"billing\"} | \n", - "{\"generations\": [[{\"text\": \"\", \"generation_inf... | \n", - "application/json | \n", - "user | \n", - "{\"messages\": [[{\"lc\": 1, \"type\": \"constructor\"... | \n", - "query_knowledge_base | \n", - "... | \n", - "assistant | \n", - "tool | \n", - "{\"ticket_text\": \"I need to update my billing i... | \n", - "classify_ticket | \n", - "NaN | \n", - "NaN | \n", - "NaN | \n", - "NaN | \n", - "NaN | \n", - "NaN | \n", - "
| 22 | \n", - "{\"model_name\": \"gemini-1.5-pro-001\", \"temperature... | \n", - "opentelemetry-python 1.25.0; google-cloud-trac... | \n", - "application/json | \n", - "\\n Classify the following ticket into a cat... | \n", - "NaN | \n", - "{\"generations\": [[{\"text\": \"The ticket is related... | \n", - "application/json | \n", - "user | \n", - "{\"messages\": [[{\"lc\": 1, \"type\": \"constructor\"... | \n", - "NaN | \n", - "... | \n", - "assistant | \n", - "tool | \n", - "{\"ticket_text\": \"I need to update my billing i... | \n", - "classify_ticket | \n", - "[{\"title\": \"How to update your payment information... | \n", - "assistant | \n", - "tool | \n", - "The ticket is related to **billing**. Here's a... | \n", - "query_knowledge_base | \n", - "{\"category\": \"billing\"} | \n", - "
3 rows × 24 columns
\n", - "| \n", + " | spanId | \n", + "name | \n", + "startTime | \n", + "endTime | \n", + "labels | \n", + "parentSpanId | \n", + "
|---|---|---|---|---|---|---|
| 0 | \n", + "12108864055932459456 | \n", + "AgentExecutor | \n", + "2024-06-29T04:33:40.426806016Z | \n", + "2024-06-29T04:33:49.886738176Z | \n", + "{'g.co/agent': 'opentelemetry-python 1.25.0; g... | \n", + "NaN | \n", + "
| 1 | \n", + "1514793182933739503 | \n", + "RunnableParallel<input,agent_scratchpad> | \n", + "2024-06-29T04:33:40.435836160Z | \n", + "2024-06-29T04:33:40.816612096Z | \n", + "{'g.co/agent': 'opentelemetry-python 1.25.0; g... | \n", + "10997585570109326486 | \n", + "
| 2 | \n", + "2419854474582761855 | \n", + "ChatPromptTemplate | \n", + "2024-06-29T04:33:40.988899840Z | \n", + "2024-06-29T04:33:40.990212864Z | \n", + "{'g.co/agent': 'opentelemetry-python 1.25.0; g... | \n", + "10997585570109326486 | \n", + "
| 3 | \n", + "883609395272937079 | \n", + "ChatVertexAI | \n", + "2024-06-29T04:33:41.174941184Z | \n", + "2024-06-29T04:33:43.206272Z | \n", + "{'llm.invocation_parameters': '{\"model_name\": ... | \n", + "10997585570109326486 | \n", + "
| 4 | \n", + "10997585570109326486 | \n", + "RunnableSequence | \n", + "2024-06-29T04:33:40.431934208Z | \n", + "2024-06-29T04:33:43.667400960Z | \n", + "{'g.co/agent': 'opentelemetry-python 1.25.0; g... | \n", + "12108864055932459456 | \n", + "
| \n", + " | spanId | \n", + "name | \n", + "startTime | \n", + "endTime | \n", + "labels | \n", + "parentSpanId | \n", + "
|---|---|---|---|---|---|---|
| 3 | \n", + "883609395272937079 | \n", + "ChatVertexAI | \n", + "2024-06-29T04:33:41.174941184Z | \n", + "2024-06-29T04:33:43.206272Z | \n", + "{'llm.invocation_parameters': '{\"model_name\": ... | \n", + "10997585570109326486 | \n", + "
| 14 | \n", + "1267786026295004320 | \n", + "ChatVertexAI | \n", + "2024-06-29T04:33:44.592414976Z | \n", + "2024-06-29T04:33:46.278656Z | \n", + "{'g.co/agent': 'opentelemetry-python 1.25.0; g... | \n", + "8170159531257482779 | \n", + "
| 22 | \n", + "15517842396867663478 | \n", + "ChatVertexAI | \n", + "2024-06-29T04:33:47.494297856Z | \n", + "2024-06-29T04:33:49.470009088Z | \n", + "{'g.co/agent': 'opentelemetry-python 1.25.0; g... | \n", + "4922396732366218475 | \n", + "
| \n", + " | llm.invocation_parameters | \n", + "g.co/agent | \n", + "output.mime_type | \n", + "llm.input_messages.0.message.content | \n", + "llm.output_messages.0.message.function_call_arguments_json | \n", + "output.value | \n", + "input.mime_type | \n", + "llm.input_messages.0.message.role | \n", + "input.value | \n", + "llm.output_messages.0.message.function_call_name | \n", + "... | \n", + "llm.input_messages.1.message.role | \n", + "llm.input_messages.2.message.role | \n", + "llm.input_messages.1.message.function_call_arguments_json | \n", + "llm.input_messages.1.message.function_call_name | \n", + "llm.input_messages.4.message.content | \n", + "llm.input_messages.3.message.role | \n", + "llm.input_messages.4.message.role | \n", + "llm.output_messages.0.message.content | \n", + "llm.input_messages.3.message.function_call_name | \n", + "llm.input_messages.3.message.function_call_arguments_json | \n", + "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 3 | \n", + "{\"model_name\": \"gemini-1.5-pro-001\", \"temperature... | \n", + "opentelemetry-python 1.25.0; google-cloud-trac... | \n", + "application/json | \n", + "\\n Classify the following ticket into a cat... | \n", + "{\"ticket_text\": \"I need to update my billing i... | \n", + "{\"generations\": [[{\"text\": \"\", \"generation_inf... | \n", + "application/json | \n", + "user | \n", + "{\"messages\": [[{\"lc\": 1, \"type\": \"constructor\"... | \n", + "classify_ticket | \n", + "... | \n", + "NaN | \n", + "NaN | \n", + "NaN | \n", + "NaN | \n", + "NaN | \n", + "NaN | \n", + "NaN | \n", + "NaN | \n", + "NaN | \n", + "NaN | \n", + "
| 14 | \n", + "{\"model_name\": \"gemini-1.5-pro-001\", \"temperature... | \n", + "opentelemetry-python 1.25.0; google-cloud-trac... | \n", + "application/json | \n", + "\\n Classify the following ticket into a cat... | \n", + "{\"category\": \"billing\"} | \n", + "{\"generations\": [[{\"text\": \"\", \"generation_inf... | \n", + "application/json | \n", + "user | \n", + "{\"messages\": [[{\"lc\": 1, \"type\": \"constructor\"... | \n", + "query_knowledge_base | \n", + "... | \n", + "assistant | \n", + "tool | \n", + "{\"ticket_text\": \"I need to update my billing i... | \n", + "classify_ticket | \n", + "NaN | \n", + "NaN | \n", + "NaN | \n", + "NaN | \n", + "NaN | \n", + "NaN | \n", + "
| 22 | \n", + "{\"model_name\": \"gemini-1.5-pro-001\", \"temperature... | \n", + "opentelemetry-python 1.25.0; google-cloud-trac... | \n", + "application/json | \n", + "\\n Classify the following ticket into a cat... | \n", + "NaN | \n", + "{\"generations\": [[{\"text\": \"The ticket is related... | \n", + "application/json | \n", + "user | \n", + "{\"messages\": [[{\"lc\": 1, \"type\": \"constructor\"... | \n", + "NaN | \n", + "... | \n", + "assistant | \n", + "tool | \n", + "{\"ticket_text\": \"I need to update my billing i... | \n", + "classify_ticket | \n", + "[{\"title\": \"How to update your payment information... | \n", + "assistant | \n", + "tool | \n", + "The ticket is related to **billing**. Here's a... | \n", + "query_knowledge_base | \n", + "{\"category\": \"billing\"} | \n", + "
3 rows × 24 columns
\n", + "| \n",
- " \n",
- " Run in Colab\n", - " \n", - " | \n",
- " \n",
- " \n",
- " View on GitHub\n", - " \n", - " | \n",
- " \n",
- " \n",
- " Open in Vertex AI Workbench\n", - " \n", - " | \n",
- "
| \n",
+ " \n",
+ " Run in Colab\n", + " \n", + " | \n",
+ " \n",
+ " \n",
+ " Run in Colab Enterprise\n", + " \n", + " | \n",
+ " \n",
+ " \n",
+ " View on GitHub\n", + " \n", + " | \n",
+ " \n",
+ " \n",
+ " Open in Vertex AI Workbench\n", + " \n", + " | \n",
+ "