Skip to content

Commit

Permalink
feat(ai): Allow customizing the LLM request settings
Browse files Browse the repository at this point in the history
With this change, we enable more easily customizing the LLM settings, such as `temperature`, for the LLM requests in a chat agent.

Contributed on behalf of STMicroelectronics
  • Loading branch information
planger committed Oct 10, 2024
1 parent eb5b395 commit 2be612f
Showing 1 changed file with 9 additions and 0 deletions.
9 changes: 9 additions & 0 deletions packages/ai-chat/src/common/chat-agents.ts
Original file line number Diff line number Diff line change
Expand Up @@ -279,13 +279,22 @@ export abstract class AbstractChatAgent {
tools: ToolRequest[] | undefined,
token: CancellationToken
): Promise<LanguageModelResponse> {
const settings = this.getLlmSettings();
const languageModelResponse = languageModel.request({
messages,
tools,
settings,
}, token);
return languageModelResponse;
}

/**
* @returns the settings, such as `temperature`, to be used in all language model requests. Returns `undefined` by default.
*/
protected getLlmSettings(): { [key: string]: unknown; } | undefined {
return undefined;
}

protected abstract addContentsToResponse(languageModelResponse: LanguageModelResponse, request: ChatRequestModelImpl): Promise<void>;
}

Expand Down

0 comments on commit 2be612f

Please sign in to comment.