Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix issue #4629: [Bug]: Replace claude-3-5-sonnet-20240620 with claude-3-5-sonnet-20241022 #4631

Merged
merged 8 commits into from
Oct 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ WORKSPACE_BASE=$(pwd)/workspace
2. Définissez `LLM_MODEL` sur le modèle que vous souhaitez utiliser :

```bash
LLM_MODEL="anthropic/claude-3-5-sonnet-20240620"
LLM_MODEL="anthropic/claude-3-5-sonnet-20241022"
```

3. Définissez `LLM_API_KEY` sur votre clé API :
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@ Voici un exemple de fichier de configuration que vous pouvez utiliser pour défi
```toml
[llm]
# IMPORTANT : ajoutez votre clé API ici et définissez le modèle que vous souhaitez évaluer
model = "claude-3-5-sonnet-20240620"
model = "claude-3-5-sonnet-20241022"

api_key = "sk-XXX"

[llm.eval_gpt4_1106_preview_llm]
Expand Down Expand Up @@ -278,3 +279,4 @@ Cette fonction fait ce qui suit :
3. Si l'agent a fait plusieurs tentatives, il lui donne la possibilité d'abandonner

En utilisant cette fonction, vous pouvez garantir un comportement cohérent sur plusieurs exécutions d'évaluation et empêcher l'agent de rester bloqué en attendant une entrée humaine.

Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ WORKSPACE_BASE=$(pwd)/workspace
2. Définissez `LLM_MODEL` sur le modèle que vous voulez utiliser :

```bash
LLM_MODEL="anthropic/claude-3-5-sonnet-20240620"
LLM_MODEL="anthropic/claude-3-5-sonnet-20241022"
```

3. Définissez `LLM_API_KEY` sur votre clé API :
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ WORKSPACE_BASE=$(pwd)/workspace
2.`LLM_MODEL` 设置为你要使用的模型:

```bash
LLM_MODEL="anthropic/claude-3-5-sonnet-20240620"
LLM_MODEL="anthropic/claude-3-5-sonnet-20241022"
```

3.`LLM_API_KEY` 设置为你的 API 密钥:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
```toml
[llm]
# 重要:在此处添加您的 API 密钥,并将模型设置为您要评估的模型
model = "claude-3-5-sonnet-20240620"
model = "claude-3-5-sonnet-20241022"
api_key = "sk-XXX"

[llm.eval_gpt4_1106_preview_llm]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@ WORKSPACE_BASE=$(pwd)/workspace
2.`LLM_MODEL` 设置为你要使用的模型:

```bash
LLM_MODEL="anthropic/claude-3-5-sonnet-20240620"
LLM_MODEL="anthropic/claude-3-5-sonnet-20241022"

```

3.`LLM_API_KEY` 设置为你的 API 密钥:
Expand All @@ -57,3 +58,4 @@ docker run -it \
ghcr.io/all-hands-ai/openhands:0.11 \
python -m openhands.core.main -t "write a bash script that prints hi"
```

4 changes: 3 additions & 1 deletion docs/modules/usage/how-to/cli-mode.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,8 @@ WORKSPACE_BASE=$(pwd)/workspace
2. Set `LLM_MODEL` to the model you want to use:

```bash
LLM_MODEL="anthropic/claude-3-5-sonnet-20240620"
LLM_MODEL="anthropic/claude-3-5-sonnet-20241022"

```

3. Set `LLM_API_KEY` to your API key:
Expand Down Expand Up @@ -106,3 +107,4 @@ Expected Output:
```bash
🤖 An error occurred. Please try again.
```

2 changes: 1 addition & 1 deletion docs/modules/usage/how-to/evaluation-harness.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Here's an example configuration file you can use to define and use multiple LLMs
```toml
[llm]
# IMPORTANT: add your API key here, and set the model to the one you want to evaluate
model = "claude-3-5-sonnet-20240620"
model = "claude-3-5-sonnet-20241022"
api_key = "sk-XXX"

[llm.eval_gpt4_1106_preview_llm]
Expand Down
4 changes: 3 additions & 1 deletion docs/modules/usage/how-to/headless-mode.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,8 @@ WORKSPACE_BASE=$(pwd)/workspace
2. Set `LLM_MODEL` to the model you want to use:

```bash
LLM_MODEL="anthropic/claude-3-5-sonnet-20240620"
LLM_MODEL="anthropic/claude-3-5-sonnet-20241022"

```

3. Set `LLM_API_KEY` to your API key:
Expand All @@ -54,3 +55,4 @@ docker run -it \
ghcr.io/all-hands-ai/openhands:0.11 \
python -m openhands.core.main -t "write a bash script that prints hi"
```

5 changes: 3 additions & 2 deletions frontend/__tests__/utils/extractModelAndProvider.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -59,9 +59,9 @@ describe("extractModelAndProvider", () => {
separator: "/",
});

expect(extractModelAndProvider("claude-3-5-sonnet-20240620")).toEqual({
expect(extractModelAndProvider("claude-3-5-sonnet-20241022")).toEqual({
provider: "anthropic",
model: "claude-3-5-sonnet-20240620",
model: "claude-3-5-sonnet-20241022",
enyst marked this conversation as resolved.
Show resolved Hide resolved
separator: "/",
});

Expand All @@ -78,3 +78,4 @@ describe("extractModelAndProvider", () => {
});
});
});

5 changes: 3 additions & 2 deletions frontend/__tests__/utils/organizeModelsAndProviders.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ test("organizeModelsAndProviders", () => {
"gpt-4o",
"together-ai-21.1b-41b",
"gpt-4o-mini",
"claude-3-5-sonnet-20240620",
"claude-3-5-sonnet-20241022",
"claude-3-haiku-20240307",
"claude-2",
"claude-2.1",
Expand Down Expand Up @@ -51,7 +51,7 @@ test("organizeModelsAndProviders", () => {
anthropic: {
separator: "/",
models: [
"claude-3-5-sonnet-20240620",
"claude-3-5-sonnet-20241022",
"claude-3-haiku-20240307",
"claude-2",
"claude-2.1",
Expand All @@ -63,3 +63,4 @@ test("organizeModelsAndProviders", () => {
},
});
});

8 changes: 6 additions & 2 deletions frontend/src/utils/verified-models.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
// Here are the list of verified models and providers that we know work well with OpenHands.
export const VERIFIED_PROVIDERS = ["openai", "azure", "anthropic"];
export const VERIFIED_MODELS = ["gpt-4o", "claude-3-5-sonnet-20240620"];
export const VERIFIED_MODELS = [
"gpt-4o",
"claude-3-5-sonnet-20240620",
"claude-3-5-sonnet-20241022",
];

// LiteLLM does not return OpenAI models with the provider, so we list them here to set them ourselves for consistency
// (e.g., they return `gpt-4o` instead of `openai/gpt-4o`)
Expand All @@ -15,7 +19,7 @@ export const VERIFIED_OPENAI_MODELS = [
];

// LiteLLM does not return the compatible Anthropic models with the provider, so we list them here to set them ourselves
// (e.g., they return `claude-3-5-sonnet-20240620` instead of `anthropic/claude-3-5-sonnet-20240620`)
// (e.g., they return `claude-3-5-sonnet-20241022` instead of `anthropic/claude-3-5-sonnet-20241022`)
export const VERIFIED_ANTHROPIC_MODELS = [
"claude-2",
"claude-2.1",
Expand Down
2 changes: 1 addition & 1 deletion openhands/llm/llm.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,8 @@
# cache prompt supporting models
# remove this when we gemini and deepseek are supported
CACHE_PROMPT_SUPPORTED_MODELS = [
'claude-3-5-sonnet-20240620',
'claude-3-5-sonnet-20241022',
'claude-3-5-sonnet-20240620',
'claude-3-haiku-20240307',
'claude-3-opus-20240229',
]
Expand Down
2 changes: 1 addition & 1 deletion tests/unit/test_prompt_caching.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
@pytest.fixture
def mock_llm():
llm = Mock(spec=LLM)
llm.config = LLMConfig(model='claude-3-5-sonnet-20240620', caching_prompt=True)
llm.config = LLMConfig(model='claude-3-5-sonnet-20241022', caching_prompt=True)
llm.is_caching_prompt_active.return_value = True
return llm

Expand Down
Loading