Skip to content

armgit / letta

Public Imported from GitHub
0

Platform for stateful agents: AI with advanced memory that can learn and self-improve over time.

update branch ref

Kian Jones8e370c101dbe
diff --git a/.github/scripts/model-sweep/generate_model_sweep_markdown.py b/.github/scripts/model-sweep/generate_model_sweep_markdown.py
index 588a205ad..7382939d2 100644
--- a/.github/scripts/model-sweep/generate_model_sweep_markdown.py
+++ b/.github/scripts/model-sweep/generate_model_sweep_markdown.py
@@ -145,17 +145,8 @@ def generate_test_details(model_info, feature_mapping):
test_file_path = os.path.join(script_dir, 'model_sweep.py')
test_line_numbers = get_test_function_line_numbers(test_file_path)
- # Get GitHub repo info
- repo_path = get_github_repo_info()
-
- # Get current commit hash for links
- try:
- import subprocess
- result = subprocess.run(['git', 'rev-parse', 'HEAD'],
- capture_output=True, text=True, cwd=script_dir)
- commit_hash = result.stdout.strip() if result.returncode == 0 else 'main'
- except:
- commit_hash = 'main'
+ # Use the specific GitHub URL for the branch
+ base_github_url = "https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py"
for feature, tests in model_info['categorized_tests'].items():
if not tests:
@@ -173,9 +164,9 @@ def generate_test_details(model_info, feature_mapping):
status = "❓"
# Create GitHub link if we have line number info
- if test in test_line_numbers and repo_path:
+ if test in test_line_numbers:
line_num = test_line_numbers[test]
- github_link = f"https://github.com/{repo_path}/blob/{commit_hash}/.github/scripts/model-sweep/model_sweep.py#L{line_num}"
+ github_link = f"{base_github_url}#L{line_num}"
details.append(f"- {status} [`{test}`]({github_link})")
else:
details.append(f"- {status} `{test}`")
diff --git a/.github/scripts/model-sweep/model_sweep_report.mdx b/.github/scripts/model-sweep/model_sweep_report.mdx
index 6458d98ba..350ce7e57 100644
--- a/.github/scripts/model-sweep/model_sweep_report.mdx
+++ b/.github/scripts/model-sweep/model_sweep_report.mdx
@@ -1,6 +1,6 @@
---
title: Support Models
-generated: 2025-06-20T16:10:08.575462
+generated: 2025-06-20T16:18:18.561937
---
# Supported Models
@@ -32,244 +32,244 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### claude-3-5-sonnet-20241022 {#anthropic_claude_3_5_sonnet_20241022_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### claude-3-7-sonnet-20250219 {#anthropic_claude_3_7_sonnet_20250219_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### claude-sonnet-4-20250514 {#anthropic_claude_sonnet_4_20250514_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### claude-opus-4-20250514 {#anthropic_claude_opus_4_20250514_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### claude-3-5-sonnet-20240620 {#anthropic_claude_3_5_sonnet_20240620_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### claude-3-haiku-20240307 {#anthropic_claude_3_haiku_20240307_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### claude-3-opus-20240229 {#anthropic_claude_3_opus_20240229_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### claude-3-sonnet-20240229 {#anthropic_claude_3_sonnet_20240229_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
---
@@ -314,784 +314,784 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gpt-4.1-2025-04-14 {#openai_gpt_4.1_2025_04_14_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gpt-4.1-nano-2025-04-14 {#openai_gpt_4.1_nano_2025_04_14_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gpt-4o {#openai_gpt_4o_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gpt-4o-2024-05-13 {#openai_gpt_4o_2024_05_13_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gpt-4-turbo {#openai_gpt_4_turbo_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gpt-4.1-mini {#openai_gpt_4.1_mini_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gpt-4.5-preview {#openai_gpt_4.5_preview_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gpt-4.5-preview-2025-02-27 {#openai_gpt_4.5_preview_2025_02_27_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gpt-4o-2024-08-06 {#openai_gpt_4o_2024_08_06_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gpt-4-0613 {#openai_gpt_4_0613_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gpt-4-1106-preview {#openai_gpt_4_1106_preview_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gpt-4-turbo-2024-04-09 {#openai_gpt_4_turbo_2024_04_09_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gpt-4.1-mini-2025-04-14 {#openai_gpt_4.1_mini_2025_04_14_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gpt-4.1-nano {#openai_gpt_4.1_nano_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gpt-4o-2024-11-20 {#openai_gpt_4o_2024_11_20_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gpt-4-turbo-preview {#openai_gpt_4_turbo_preview_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gpt-4-0125-preview {#openai_gpt_4_0125_preview_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gpt-4o-mini {#openai_gpt_4o_mini_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gpt-4o-mini-2024-07-18 {#openai_gpt_4o_mini_2024_07_18_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gpt-4 {#openai_gpt_4_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### o1 {#openai_o1_details}
### Basic
-- ❌ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ❌ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### o1-2024-12-17 {#openai_o1_2024_12_17_details}
### Basic
-- ❌ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ❌ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### o3 {#openai_o3_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### o3-2025-04-16 {#openai_o3_2025_04_16_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### o3-mini {#openai_o3_mini_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### o3-mini-2025-01-31 {#openai_o3_mini_2025_01_31_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### o3-pro {#openai_o3_pro_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### o3-pro-2025-06-10 {#openai_o3_pro_2025_06_10_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
---
@@ -1142,946 +1142,946 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-1.5-pro-002 {#google_ai_gemini_1.5_pro_002_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-1.5-pro-latest {#google_ai_gemini_1.5_pro_latest_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-2.5-flash-preview-04-17-thinking {#google_ai_gemini_2.5_flash_preview_04_17_thinking_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-2.5-pro-preview-03-25 {#google_ai_gemini_2.5_pro_preview_03_25_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-2.5-pro-preview-05-06 {#google_ai_gemini_2.5_pro_preview_05_06_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-2.5-flash-preview-05-20 {#google_ai_gemini_2.5_flash_preview_05_20_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-2.0-flash-thinking-exp {#google_ai_gemini_2.0_flash_thinking_exp_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-2.0-flash-thinking-exp-1219 {#google_ai_gemini_2.0_flash_thinking_exp_1219_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-2.0-flash-thinking-exp-01-21 {#google_ai_gemini_2.0_flash_thinking_exp_01_21_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-2.5-flash-preview-04-17 {#google_ai_gemini_2.5_flash_preview_04_17_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-2.5-pro-preview-06-05 {#google_ai_gemini_2.5_pro_preview_06_05_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-1.0-pro-vision-latest {#google_ai_gemini_1.0_pro_vision_latest_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-1.5-flash {#google_ai_gemini_1.5_flash_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-1.5-flash-002 {#google_ai_gemini_1.5_flash_002_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-1.5-flash-8b {#google_ai_gemini_1.5_flash_8b_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-1.5-flash-8b-001 {#google_ai_gemini_1.5_flash_8b_001_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-1.5-flash-8b-latest {#google_ai_gemini_1.5_flash_8b_latest_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-1.5-flash-latest {#google_ai_gemini_1.5_flash_latest_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-2.0-flash {#google_ai_gemini_2.0_flash_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-2.0-flash-001 {#google_ai_gemini_2.0_flash_001_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-2.0-flash-exp {#google_ai_gemini_2.0_flash_exp_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-2.0-flash-exp-image-generation {#google_ai_gemini_2.0_flash_exp_image_generation_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-2.0-flash-lite {#google_ai_gemini_2.0_flash_lite_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-2.0-flash-lite-001 {#google_ai_gemini_2.0_flash_lite_001_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-2.0-flash-lite-preview {#google_ai_gemini_2.0_flash_lite_preview_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-2.0-flash-lite-preview-02-05 {#google_ai_gemini_2.0_flash_lite_preview_02_05_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-2.0-flash-preview-image-generation {#google_ai_gemini_2.0_flash_preview_image_generation_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-2.0-pro-exp {#google_ai_gemini_2.0_pro_exp_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-2.0-pro-exp-02-05 {#google_ai_gemini_2.0_pro_exp_02_05_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-2.5-flash-preview-tts {#google_ai_gemini_2.5_flash_preview_tts_details}
### Basic
-- ❌ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ❌ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-2.5-pro-exp-03-25 {#google_ai_gemini_2.5_pro_exp_03_25_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-2.5-pro-preview-tts {#google_ai_gemini_2.5_pro_preview_tts_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-exp-1206 {#google_ai_gemini_exp_1206_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemini-pro-vision {#google_ai_gemini_pro_vision_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
---
@@ -2098,28 +2098,28 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
---
@@ -2190,1486 +2190,1486 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### arcee-ai/virtuoso-large {#together_arcee_ai_virtuoso_large_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### Qwen/QwQ-32B {#together_qwen_qwq_32b_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### Qwen/Qwen2.5-7B-Instruct-Turbo {#together_qwen_qwen2.5_7b_instruct_turbo_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### Qwen/Qwen2.5-Coder-32B-Instruct {#together_qwen_qwen2.5_coder_32b_instruct_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### arcee-ai/coder-large {#together_arcee_ai_coder_large_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### arcee_ai/arcee-spotlight {#together_arcee_ai_arcee_spotlight_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### meta-llama/Llama-3.2-3B-Instruct-Turbo {#together_meta_llama_llama_3.2_3b_instruct_turbo_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### meta-llama/Llama-3.3-70B-Instruct-Turbo {#together_meta_llama_llama_3.3_70b_instruct_turbo_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### meta-llama/Llama-3.3-70B-Instruct-Turbo-Free {#together_meta_llama_llama_3.3_70b_instruct_turbo_free_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo {#together_meta_llama_meta_llama_3.1_405b_instruct_turbo_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo {#together_meta_llama_meta_llama_3.1_70b_instruct_turbo_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### nvidia/Llama-3.1-Nemotron-70B-Instruct-HF {#together_nvidia_llama_3.1_nemotron_70b_instruct_hf_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ✅ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### arcee-ai/virtuoso-medium-v2 {#together_arcee_ai_virtuoso_medium_v2_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8 {#together_meta_llama_llama_4_maverick_17b_128e_instruct_fp8_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ✅ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### Qwen/Qwen3-235B-A22B-fp8-tput {#together_qwen_qwen3_235b_a22b_fp8_tput_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### deepseek-ai/DeepSeek-V3 {#together_deepseek_ai_deepseek_v3_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### meta-llama/Llama-4-Scout-17B-16E-Instruct {#together_meta_llama_llama_4_scout_17b_16e_instruct_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo {#together_meta_llama_meta_llama_3.1_8b_instruct_turbo_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ✅ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### mistralai/Mixtral-8x7B-Instruct-v0.1 {#together_mistralai_mixtral_8x7b_instruct_v0.1_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ✅ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ✅ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ✅ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ✅ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ✅ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ✅ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### arcee-ai/caller {#together_arcee_ai_caller_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ✅ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### mistralai/Mistral-Small-24B-Instruct-2501 {#together_mistralai_mistral_small_24b_instruct_2501_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ✅ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO {#together_nousresearch_nous_hermes_2_mixtral_8x7b_dpo_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### Qwen/Qwen2-72B-Instruct {#together_qwen_qwen2_72b_instruct_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### Qwen/Qwen2-VL-72B-Instruct {#together_qwen_qwen2_vl_72b_instruct_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### Qwen/Qwen2.5-VL-72B-Instruct {#together_qwen_qwen2.5_vl_72b_instruct_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### arcee-ai/arcee-blitz {#together_arcee_ai_arcee_blitz_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### arcee-ai/maestro-reasoning {#together_arcee_ai_maestro_reasoning_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### deepseek-ai/DeepSeek-R1 {#together_deepseek_ai_deepseek_r1_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### deepseek-ai/DeepSeek-R1-Distill-Llama-70B {#together_deepseek_ai_deepseek_r1_distill_llama_70b_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### deepseek-ai/DeepSeek-R1-Distill-Llama-70B-free {#together_deepseek_ai_deepseek_r1_distill_llama_70b_free_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B {#together_deepseek_ai_deepseek_r1_distill_qwen_1.5b_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### deepseek-ai/DeepSeek-R1-Distill-Qwen-14B {#together_deepseek_ai_deepseek_r1_distill_qwen_14b_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### deepseek-ai/DeepSeek-V3-p-dp {#together_deepseek_ai_deepseek_v3_p_dp_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### google/gemma-2-27b-it {#together_google_gemma_2_27b_it_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### lgai/exaone-3-5-32b-instruct {#together_lgai_exaone_3_5_32b_instruct_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### lgai/exaone-deep-32b {#together_lgai_exaone_deep_32b_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### marin-community/marin-8b-instruct {#together_marin_community_marin_8b_instruct_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### meta-llama/Llama-3-70b-chat-hf {#together_meta_llama_llama_3_70b_chat_hf_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### meta-llama/Llama-3-8b-chat-hf {#together_meta_llama_llama_3_8b_chat_hf_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### meta-llama/Llama-3.2-11B-Vision-Instruct-Turbo {#together_meta_llama_llama_3.2_11b_vision_instruct_turbo_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### meta-llama/Llama-3.2-90B-Vision-Instruct-Turbo {#together_meta_llama_llama_3.2_90b_vision_instruct_turbo_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### meta-llama/Llama-Vision-Free {#together_meta_llama_llama_vision_free_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### meta-llama/Meta-Llama-3-70B-Instruct-Turbo {#together_meta_llama_meta_llama_3_70b_instruct_turbo_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### meta-llama/Meta-Llama-3-8B-Instruct-Lite {#together_meta_llama_meta_llama_3_8b_instruct_lite_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### mistralai/Mistral-7B-Instruct-v0.1 {#together_mistralai_mistral_7b_instruct_v0.1_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### mistralai/Mistral-7B-Instruct-v0.2 {#together_mistralai_mistral_7b_instruct_v0.2_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### mistralai/Mistral-7B-Instruct-v0.3 {#together_mistralai_mistral_7b_instruct_v0.3_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### perplexity-ai/r1-1776 {#together_perplexity_ai_r1_1776_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### scb10x/scb10x-llama3-1-typhoon2-70b-instruct {#together_scb10x_scb10x_llama3_1_typhoon2_70b_instruct_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### scb10x/scb10x-typhoon-2-1-gemma3-12b {#together_scb10x_scb10x_typhoon_2_1_gemma3_12b_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### togethercomputer/MoA-1 {#together_togethercomputer_moa_1_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### togethercomputer/MoA-1-Turbo {#together_togethercomputer_moa_1_turbo_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### togethercomputer/Refuel-Llm-V2 {#together_togethercomputer_refuel_llm_v2_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### togethercomputer/Refuel-Llm-V2-Small {#together_togethercomputer_refuel_llm_v2_small_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
---
@@ -3687,55 +3687,55 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### deepseek-reasoner {#deepseek_deepseek_reasoner_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
---
@@ -3774,622 +3774,622 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### compound-beta {#groq_compound_beta_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### compound-beta-mini {#groq_compound_beta_mini_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### deepseek-r1-distill-llama-70b {#groq_deepseek_r1_distill_llama_70b_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### distil-whisper-large-v3-en {#groq_distil_whisper_large_v3_en_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### gemma2-9b-it {#groq_gemma2_9b_it_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### llama-3.1-8b-instant {#groq_llama_3.1_8b_instant_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### llama-3.3-70b-versatile {#groq_llama_3.3_70b_versatile_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### llama-guard-3-8b {#groq_llama_guard_3_8b_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### llama3-70b-8192 {#groq_llama3_70b_8192_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### llama3-8b-8192 {#groq_llama3_8b_8192_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### meta-llama/llama-4-maverick-17b-128e-instruct {#groq_meta_llama_llama_4_maverick_17b_128e_instruct_details}
### Basic
-- ❌ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ❌ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### meta-llama/llama-4-scout-17b-16e-instruct {#groq_meta_llama_llama_4_scout_17b_16e_instruct_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### meta-llama/llama-guard-4-12b {#groq_meta_llama_llama_guard_4_12b_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### meta-llama/llama-prompt-guard-2-22m {#groq_meta_llama_llama_prompt_guard_2_22m_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### meta-llama/llama-prompt-guard-2-86m {#groq_meta_llama_llama_prompt_guard_2_86m_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### mistral-saba-24b {#groq_mistral_saba_24b_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### playai-tts {#groq_playai_tts_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### playai-tts-arabic {#groq_playai_tts_arabic_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### qwen-qwq-32b {#groq_qwen_qwq_32b_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### qwen/qwen3-32b {#groq_qwen_qwen3_32b_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### whisper-large-v3 {#groq_whisper_large_v3_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
#### whisper-large-v3-turbo {#groq_whisper_large_v3_turbo_details}
### Basic
-- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L463)
-- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L704)
-- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L731)
-- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L345)
-- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L368)
-- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L566)
-- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L488)
-- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L513)
-- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L539)
-- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L392)
+- ✅ [`test_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L463)
+- ❌ [`test_async_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L704)
+- ❌ [`test_auto_summarize`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L731)
+- ❌ [`test_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L345)
+- ❌ [`test_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L368)
+- ✅ [`test_step_stream_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L566)
+- ❌ [`test_step_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L488)
+- ❌ [`test_step_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L513)
+- ❌ [`test_step_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L539)
+- ❌ [`test_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L392)
### Token Streaming
-- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L674)
-- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L593)
-- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L619)
-- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L646)
+- ✅ [`test_token_streaming_agent_loop_error`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L674)
+- ❌ [`test_token_streaming_greeting_with_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L593)
+- ❌ [`test_token_streaming_greeting_without_assistant_message`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L619)
+- ❌ [`test_token_streaming_tool_call`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L646)
### Multimodal
-- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L440)
-- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/4b8c690fda9120641be656b11751f1c7fe232aee/.github/scripts/model-sweep/model_sweep.py#L417)
+- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)
+- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)
---