Run documents in batch when the prompt has tool calls #784

andresgutgon · 2025-01-20T10:05:27Z

What?

We implemented tool calling mocking in our playground. But we also need to handle tool calling when the running of the prompt is in batch

Create a document for gen tool responses in development workspace
Implement AI service to auto-generate tool responses based on tool calls
Handle tool calling when running in batch from the playground
Handle tool calling when running an evaluation in batch
~~Provision missing environment variables in production~~ I'll postpone from now. I did a default variable if the env is not there.
Create a document for gen tool responses in the production workspace. Copy from dev

andresgutgon · 2025-01-20T16:49:22Z

.../core/src/jobs/job-definitions/documents/runDocumentAtCommit/generateToolResponseMessages.ts

+    )
+  }
+
+  const toolCallResponses = response.object as ToolCallResponse[]


Validate that the returned object is like an array of ToolCallResponse. AI could hallucinate

andresgutgon added the 🚧 wip Work in progress label Jan 20, 2025

andresgutgon force-pushed the fix/run-batch-with-tool-calls branch from 4e1610e to 23ce1dd Compare January 20, 2025 14:33

andresgutgon commented Jan 20, 2025

View reviewed changes

andresgutgon force-pushed the fix/run-batch-with-tool-calls branch 2 times, most recently from c5d36c4 to 38445e8 Compare January 22, 2025 15:10

Run documents in batch when the prompt has tool calls

43337e7

andresgutgon force-pushed the fix/run-batch-with-tool-calls branch from 38445e8 to 43337e7 Compare January 22, 2025 15:15

andresgutgon removed the 🚧 wip Work in progress label Jan 22, 2025

csansoon approved these changes Jan 22, 2025

View reviewed changes

andresgutgon merged commit 1849d44 into main Jan 22, 2025
4 checks passed

andresgutgon deleted the fix/run-batch-with-tool-calls branch January 22, 2025 16:15

github-actions bot locked and limited conversation to collaborators Jan 22, 2025