Platform for stateful agents: AI with advanced memory that can learn and self-improve over time.
tweak
Kian Jonesebe6aa68ce01
diff --git a/.github/scripts/model-sweep/generate_model_sweep_markdown.py b/.github/scripts/model-sweep/generate_model_sweep_markdown.pyindex 7382939d2..09aafa7e5 100644--- a/.github/scripts/model-sweep/generate_model_sweep_markdown.py+++ b/.github/scripts/model-sweep/generate_model_sweep_markdown.py@@ -406,7 +406,7 @@ def process_model_sweep_report(input_file, output_file, config_file=None, debug=model_anchor = model_info['name'].replace('/', '_').replace(':', '_').replace('-', '_').lower()# Build row with left-aligned first column, centered others- row_parts = [f"[`{model_info['name']}`](#{provider_name.lower().replace(' ', '_')}_{model_anchor}_details)".ljust(column_widths['model'] + 20)] # Add extra space for link syntax+ row_parts = [f"`{model_info['name']}`".ljust(column_widths['model'])]for feature in feature_names:row_parts.append(f"{model_info['feature_support'][feature]:^{column_widths[feature]}}")row_parts.extend([@@ -421,7 +421,7 @@ def process_model_sweep_report(input_file, output_file, config_file=None, debug=for model_info in model_data:model_anchor = model_info['name'].replace('/', '_').replace(':', '_').replace('-', '_').lower()- mdx_lines.append(f"#### {model_info['name']} {{#{provider_name.lower().replace(' ', '_')}_{model_anchor}_details}}")+ mdx_lines.append(f"#### {model_info['name']}")mdx_lines.append("")# Add test detailsdiff --git a/.github/scripts/model-sweep/model_sweep_report.mdx b/.github/scripts/model-sweep/model_sweep_report.mdxindex 350ce7e57..8990e4407 100644--- a/.github/scripts/model-sweep/model_sweep_report.mdx+++ b/.github/scripts/model-sweep/model_sweep_report.mdx@@ -1,6 +1,6 @@---title: Support Models-generated: 2025-06-20T16:18:18.561937+generated: 2025-06-20T16:24:07.581430---# Supported Models@@ -16,19 +16,19 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025| Model | Basic | Token Streaming | Multimodal | Context Window | Last Scanned ||---------------------------------------------------|:---:|:-------------:|:--------:|:------------:|:----------:|-| [`claude-3-5-haiku-20241022`](#anthropic_claude_3_5_haiku_20241022_details) | ✅ | ✅ | ✅ | 200,000 | 2025-06-20 |-| [`claude-3-5-sonnet-20241022`](#anthropic_claude_3_5_sonnet_20241022_details) | ✅ | ✅ | ✅ | 200,000 | 2025-06-20 |-| [`claude-3-7-sonnet-20250219`](#anthropic_claude_3_7_sonnet_20250219_details) | ✅ | ✅ | ✅ | 200,000 | 2025-06-20 |-| [`claude-sonnet-4-20250514`](#anthropic_claude_sonnet_4_20250514_details) | ✅ | ✅ | ✅ | 200,000 | 2025-06-20 |-| [`claude-opus-4-20250514`](#anthropic_claude_opus_4_20250514_details) | ✅ | ✅ | ⚠️ | 200,000 | 2025-06-20 |-| [`claude-3-5-sonnet-20240620`](#anthropic_claude_3_5_sonnet_20240620_details) | ⚠️ | ❌ | ✅ | 200,000 | 2025-06-20 |-| [`claude-3-haiku-20240307`](#anthropic_claude_3_haiku_20240307_details) | ⚠️ | ❌ | ✅ | 200,000 | 2025-06-20 |-| [`claude-3-opus-20240229`](#anthropic_claude_3_opus_20240229_details) | ⚠️ | ❌ | ✅ | 200,000 | 2025-06-20 |-| [`claude-3-sonnet-20240229`](#anthropic_claude_3_sonnet_20240229_details) | ❌ | ❌ | ❌ | 200,000 | 2025-06-20 |+| `claude-3-5-haiku-20241022` | ✅ | ✅ | ✅ | 200,000 | 2025-06-20 |+| `claude-3-5-sonnet-20241022` | ✅ | ✅ | ✅ | 200,000 | 2025-06-20 |+| `claude-3-7-sonnet-20250219` | ✅ | ✅ | ✅ | 200,000 | 2025-06-20 |+| `claude-sonnet-4-20250514` | ✅ | ✅ | ✅ | 200,000 | 2025-06-20 |+| `claude-opus-4-20250514` | ✅ | ✅ | ⚠️ | 200,000 | 2025-06-20 |+| `claude-3-5-sonnet-20240620` | ⚠️ | ❌ | ✅ | 200,000 | 2025-06-20 |+| `claude-3-haiku-20240307` | ⚠️ | ❌ | ✅ | 200,000 | 2025-06-20 |+| `claude-3-opus-20240229` | ⚠️ | ❌ | ✅ | 200,000 | 2025-06-20 |+| `claude-3-sonnet-20240229` | ❌ | ❌ | ❌ | 200,000 | 2025-06-20 |### Detailed Test Results-#### claude-3-5-haiku-20241022 {#anthropic_claude_3_5_haiku_20241022_details}+#### claude-3-5-haiku-20241022### Basic@@ -55,7 +55,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### claude-3-5-sonnet-20241022 {#anthropic_claude_3_5_sonnet_20241022_details}+#### claude-3-5-sonnet-20241022### Basic@@ -82,7 +82,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### claude-3-7-sonnet-20250219 {#anthropic_claude_3_7_sonnet_20250219_details}+#### claude-3-7-sonnet-20250219### Basic@@ -109,7 +109,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### claude-sonnet-4-20250514 {#anthropic_claude_sonnet_4_20250514_details}+#### claude-sonnet-4-20250514### Basic@@ -136,7 +136,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### claude-opus-4-20250514 {#anthropic_claude_opus_4_20250514_details}+#### claude-opus-4-20250514### Basic@@ -163,7 +163,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### claude-3-5-sonnet-20240620 {#anthropic_claude_3_5_sonnet_20240620_details}+#### claude-3-5-sonnet-20240620### Basic@@ -190,7 +190,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### claude-3-haiku-20240307 {#anthropic_claude_3_haiku_20240307_details}+#### claude-3-haiku-20240307### Basic@@ -217,7 +217,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### claude-3-opus-20240229 {#anthropic_claude_3_opus_20240229_details}+#### claude-3-opus-20240229### Basic@@ -244,7 +244,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### claude-3-sonnet-20240229 {#anthropic_claude_3_sonnet_20240229_details}+#### claude-3-sonnet-20240229### Basic@@ -278,39 +278,39 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025| Model | Basic | Token Streaming | Multimodal | Context Window | Last Scanned ||---------------------------------------------------|:---:|:-------------:|:--------:|:------------:|:----------:|-| [`gpt-4.1`](#openai_gpt_4.1_details) | ✅ | ✅ | ✅ | 1,047,576 | 2025-06-20 |-| [`gpt-4.1-2025-04-14`](#openai_gpt_4.1_2025_04_14_details) | ✅ | ✅ | ✅ | 1,047,576 | 2025-06-20 |-| [`gpt-4.1-nano-2025-04-14`](#openai_gpt_4.1_nano_2025_04_14_details) | ✅ | ✅ | ✅ | 1,047,576 | 2025-06-20 |-| [`gpt-4o`](#openai_gpt_4o_details) | ✅ | ✅ | ✅ | 128,000 | 2025-06-20 |-| [`gpt-4o-2024-05-13`](#openai_gpt_4o_2024_05_13_details) | ✅ | ✅ | ✅ | 128,000 | 2025-06-20 |-| [`gpt-4-turbo`](#openai_gpt_4_turbo_details) | ✅ | ✅ | ⚠️ | 8,192 | 2025-06-20 |-| [`gpt-4.1-mini`](#openai_gpt_4.1_mini_details) | ✅ | ✅ | ⚠️ | 1,047,576 | 2025-06-20 |-| [`gpt-4.5-preview`](#openai_gpt_4.5_preview_details) | ✅ | ✅ | ⚠️ | 128,000 | 2025-06-20 |-| [`gpt-4.5-preview-2025-02-27`](#openai_gpt_4.5_preview_2025_02_27_details) | ✅ | ✅ | ⚠️ | 128,000 | 2025-06-20 |-| [`gpt-4o-2024-08-06`](#openai_gpt_4o_2024_08_06_details) | ✅ | ✅ | ⚠️ | 128,000 | 2025-06-20 |-| [`gpt-4-0613`](#openai_gpt_4_0613_details) | ✅ | ✅ | ❌ | 8,192 | 2025-06-20 |-| [`gpt-4-1106-preview`](#openai_gpt_4_1106_preview_details) | ✅ | ✅ | ❌ | 128,000 | 2025-06-20 |-| [`gpt-4-turbo-2024-04-09`](#openai_gpt_4_turbo_2024_04_09_details) | ✅ | ⚠️ | ✅ | 128,000 | 2025-06-20 |-| [`gpt-4.1-mini-2025-04-14`](#openai_gpt_4.1_mini_2025_04_14_details) | ⚠️ | ✅ | ✅ | 1,047,576 | 2025-06-20 |-| [`gpt-4.1-nano`](#openai_gpt_4.1_nano_details) | ⚠️ | ✅ | ✅ | 1,047,576 | 2025-06-20 |-| [`gpt-4o-2024-11-20`](#openai_gpt_4o_2024_11_20_details) | ⚠️ | ✅ | ✅ | 8,192 | 2025-06-20 |-| [`gpt-4-turbo-preview`](#openai_gpt_4_turbo_preview_details) | ✅ | ⚠️ | ❌ | 128,000 | 2025-06-20 |-| [`gpt-4-0125-preview`](#openai_gpt_4_0125_preview_details) | ⚠️ | ✅ | ❌ | 128,000 | 2025-06-20 |-| [`gpt-4o-mini`](#openai_gpt_4o_mini_details) | ⚠️ | ⚠️ | ⚠️ | 128,000 | 2025-06-20 |-| [`gpt-4o-mini-2024-07-18`](#openai_gpt_4o_mini_2024_07_18_details) | ⚠️ | ⚠️ | ❌ | 128,000 | 2025-06-20 |-| [`gpt-4`](#openai_gpt_4_details) | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`o1`](#openai_o1_details) | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`o1-2024-12-17`](#openai_o1_2024_12_17_details) | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`o3`](#openai_o3_details) | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`o3-2025-04-16`](#openai_o3_2025_04_16_details) | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`o3-mini`](#openai_o3_mini_details) | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`o3-mini-2025-01-31`](#openai_o3_mini_2025_01_31_details) | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`o3-pro`](#openai_o3_pro_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`o3-pro-2025-06-10`](#openai_o3_pro_2025_06_10_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `gpt-4.1` | ✅ | ✅ | ✅ | 1,047,576 | 2025-06-20 |+| `gpt-4.1-2025-04-14` | ✅ | ✅ | ✅ | 1,047,576 | 2025-06-20 |+| `gpt-4.1-nano-2025-04-14` | ✅ | ✅ | ✅ | 1,047,576 | 2025-06-20 |+| `gpt-4o` | ✅ | ✅ | ✅ | 128,000 | 2025-06-20 |+| `gpt-4o-2024-05-13` | ✅ | ✅ | ✅ | 128,000 | 2025-06-20 |+| `gpt-4-turbo` | ✅ | ✅ | ⚠️ | 8,192 | 2025-06-20 |+| `gpt-4.1-mini` | ✅ | ✅ | ⚠️ | 1,047,576 | 2025-06-20 |+| `gpt-4.5-preview` | ✅ | ✅ | ⚠️ | 128,000 | 2025-06-20 |+| `gpt-4.5-preview-2025-02-27` | ✅ | ✅ | ⚠️ | 128,000 | 2025-06-20 |+| `gpt-4o-2024-08-06` | ✅ | ✅ | ⚠️ | 128,000 | 2025-06-20 |+| `gpt-4-0613` | ✅ | ✅ | ❌ | 8,192 | 2025-06-20 |+| `gpt-4-1106-preview` | ✅ | ✅ | ❌ | 128,000 | 2025-06-20 |+| `gpt-4-turbo-2024-04-09` | ✅ | ⚠️ | ✅ | 128,000 | 2025-06-20 |+| `gpt-4.1-mini-2025-04-14` | ⚠️ | ✅ | ✅ | 1,047,576 | 2025-06-20 |+| `gpt-4.1-nano` | ⚠️ | ✅ | ✅ | 1,047,576 | 2025-06-20 |+| `gpt-4o-2024-11-20` | ⚠️ | ✅ | ✅ | 8,192 | 2025-06-20 |+| `gpt-4-turbo-preview` | ✅ | ⚠️ | ❌ | 128,000 | 2025-06-20 |+| `gpt-4-0125-preview` | ⚠️ | ✅ | ❌ | 128,000 | 2025-06-20 |+| `gpt-4o-mini` | ⚠️ | ⚠️ | ⚠️ | 128,000 | 2025-06-20 |+| `gpt-4o-mini-2024-07-18` | ⚠️ | ⚠️ | ❌ | 128,000 | 2025-06-20 |+| `gpt-4` | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `o1` | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `o1-2024-12-17` | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `o3` | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `o3-2025-04-16` | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `o3-mini` | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `o3-mini-2025-01-31` | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `o3-pro` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `o3-pro-2025-06-10` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |### Detailed Test Results-#### gpt-4.1 {#openai_gpt_4.1_details}+#### gpt-4.1### Basic@@ -337,7 +337,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gpt-4.1-2025-04-14 {#openai_gpt_4.1_2025_04_14_details}+#### gpt-4.1-2025-04-14### Basic@@ -364,7 +364,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gpt-4.1-nano-2025-04-14 {#openai_gpt_4.1_nano_2025_04_14_details}+#### gpt-4.1-nano-2025-04-14### Basic@@ -391,7 +391,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gpt-4o {#openai_gpt_4o_details}+#### gpt-4o### Basic@@ -418,7 +418,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gpt-4o-2024-05-13 {#openai_gpt_4o_2024_05_13_details}+#### gpt-4o-2024-05-13### Basic@@ -445,7 +445,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gpt-4-turbo {#openai_gpt_4_turbo_details}+#### gpt-4-turbo### Basic@@ -472,7 +472,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gpt-4.1-mini {#openai_gpt_4.1_mini_details}+#### gpt-4.1-mini### Basic@@ -499,7 +499,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gpt-4.5-preview {#openai_gpt_4.5_preview_details}+#### gpt-4.5-preview### Basic@@ -526,7 +526,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gpt-4.5-preview-2025-02-27 {#openai_gpt_4.5_preview_2025_02_27_details}+#### gpt-4.5-preview-2025-02-27### Basic@@ -553,7 +553,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gpt-4o-2024-08-06 {#openai_gpt_4o_2024_08_06_details}+#### gpt-4o-2024-08-06### Basic@@ -580,7 +580,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gpt-4-0613 {#openai_gpt_4_0613_details}+#### gpt-4-0613### Basic@@ -607,7 +607,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gpt-4-1106-preview {#openai_gpt_4_1106_preview_details}+#### gpt-4-1106-preview### Basic@@ -634,7 +634,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gpt-4-turbo-2024-04-09 {#openai_gpt_4_turbo_2024_04_09_details}+#### gpt-4-turbo-2024-04-09### Basic@@ -661,7 +661,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gpt-4.1-mini-2025-04-14 {#openai_gpt_4.1_mini_2025_04_14_details}+#### gpt-4.1-mini-2025-04-14### Basic@@ -688,7 +688,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gpt-4.1-nano {#openai_gpt_4.1_nano_details}+#### gpt-4.1-nano### Basic@@ -715,7 +715,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gpt-4o-2024-11-20 {#openai_gpt_4o_2024_11_20_details}+#### gpt-4o-2024-11-20### Basic@@ -742,7 +742,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gpt-4-turbo-preview {#openai_gpt_4_turbo_preview_details}+#### gpt-4-turbo-preview### Basic@@ -769,7 +769,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gpt-4-0125-preview {#openai_gpt_4_0125_preview_details}+#### gpt-4-0125-preview### Basic@@ -796,7 +796,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gpt-4o-mini {#openai_gpt_4o_mini_details}+#### gpt-4o-mini### Basic@@ -823,7 +823,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gpt-4o-mini-2024-07-18 {#openai_gpt_4o_mini_2024_07_18_details}+#### gpt-4o-mini-2024-07-18### Basic@@ -850,7 +850,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gpt-4 {#openai_gpt_4_details}+#### gpt-4### Basic@@ -877,7 +877,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### o1 {#openai_o1_details}+#### o1### Basic@@ -904,7 +904,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### o1-2024-12-17 {#openai_o1_2024_12_17_details}+#### o1-2024-12-17### Basic@@ -931,7 +931,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### o3 {#openai_o3_details}+#### o3### Basic@@ -958,7 +958,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### o3-2025-04-16 {#openai_o3_2025_04_16_details}+#### o3-2025-04-16### Basic@@ -985,7 +985,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### o3-mini {#openai_o3_mini_details}+#### o3-mini### Basic@@ -1012,7 +1012,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### o3-mini-2025-01-31 {#openai_o3_mini_2025_01_31_details}+#### o3-mini-2025-01-31### Basic@@ -1039,7 +1039,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### o3-pro {#openai_o3_pro_details}+#### o3-pro### Basic@@ -1066,7 +1066,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### o3-pro-2025-06-10 {#openai_o3_pro_2025_06_10_details}+#### o3-pro-2025-06-10### Basic@@ -1100,45 +1100,45 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025| Model | Basic | Token Streaming | Multimodal | Context Window | Last Scanned ||---------------------------------------------------|:---:|:-------------:|:--------:|:------------:|:----------:|-| [`gemini-1.5-pro`](#google_ai_gemini_1.5_pro_details) | ✅ | ✅ | ✅ | 2,000,000 | 2025-06-20 |-| [`gemini-1.5-pro-002`](#google_ai_gemini_1.5_pro_002_details) | ✅ | ✅ | ✅ | 2,000,000 | 2025-06-20 |-| [`gemini-1.5-pro-latest`](#google_ai_gemini_1.5_pro_latest_details) | ✅ | ✅ | ✅ | 2,000,000 | 2025-06-20 |-| [`gemini-2.5-flash-preview-04-17-thinking`](#google_ai_gemini_2.5_flash_preview_04_17_thinking_details) | ✅ | ✅ | ✅ | 1,048,576 | 2025-06-20 |-| [`gemini-2.5-pro-preview-03-25`](#google_ai_gemini_2.5_pro_preview_03_25_details) | ✅ | ✅ | ✅ | 1,048,576 | 2025-06-20 |-| [`gemini-2.5-pro-preview-05-06`](#google_ai_gemini_2.5_pro_preview_05_06_details) | ✅ | ✅ | ✅ | 1,048,576 | 2025-06-20 |-| [`gemini-2.5-flash-preview-05-20`](#google_ai_gemini_2.5_flash_preview_05_20_details) | ✅ | ⚠️ | ✅ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-thinking-exp`](#google_ai_gemini_2.0_flash_thinking_exp_details) | ⚠️ | ✅ | ✅ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-thinking-exp-1219`](#google_ai_gemini_2.0_flash_thinking_exp_1219_details) | ⚠️ | ✅ | ✅ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-thinking-exp-01-21`](#google_ai_gemini_2.0_flash_thinking_exp_01_21_details) | ⚠️ | ✅ | ⚠️ | 1,048,576 | 2025-06-20 |-| [`gemini-2.5-flash-preview-04-17`](#google_ai_gemini_2.5_flash_preview_04_17_details) | ⚠️ | ✅ | ⚠️ | 1,048,576 | 2025-06-20 |-| [`gemini-2.5-pro-preview-06-05`](#google_ai_gemini_2.5_pro_preview_06_05_details) | ⚠️ | ✅ | ⚠️ | 1,048,576 | 2025-06-20 |-| [`gemini-1.0-pro-vision-latest`](#google_ai_gemini_1.0_pro_vision_latest_details) | ❌ | ❌ | ❌ | 12,288 | 2025-06-20 |-| [`gemini-1.5-flash`](#google_ai_gemini_1.5_flash_details) | ❌ | ❌ | ❌ | 1,000,000 | 2025-06-20 |-| [`gemini-1.5-flash-002`](#google_ai_gemini_1.5_flash_002_details) | ❌ | ❌ | ❌ | 1,000,000 | 2025-06-20 |-| [`gemini-1.5-flash-8b`](#google_ai_gemini_1.5_flash_8b_details) | ❌ | ❌ | ❌ | 1,000,000 | 2025-06-20 |-| [`gemini-1.5-flash-8b-001`](#google_ai_gemini_1.5_flash_8b_001_details) | ❌ | ❌ | ❌ | 1,000,000 | 2025-06-20 |-| [`gemini-1.5-flash-8b-latest`](#google_ai_gemini_1.5_flash_8b_latest_details) | ❌ | ❌ | ❌ | 1,000,000 | 2025-06-20 |-| [`gemini-1.5-flash-latest`](#google_ai_gemini_1.5_flash_latest_details) | ❌ | ❌ | ❌ | 1,000,000 | 2025-06-20 |-| [`gemini-2.0-flash`](#google_ai_gemini_2.0_flash_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-001`](#google_ai_gemini_2.0_flash_001_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-exp`](#google_ai_gemini_2.0_flash_exp_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-exp-image-generation`](#google_ai_gemini_2.0_flash_exp_image_generation_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-lite`](#google_ai_gemini_2.0_flash_lite_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-lite-001`](#google_ai_gemini_2.0_flash_lite_001_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-lite-preview`](#google_ai_gemini_2.0_flash_lite_preview_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-lite-preview-02-05`](#google_ai_gemini_2.0_flash_lite_preview_02_05_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-preview-image-generation`](#google_ai_gemini_2.0_flash_preview_image_generation_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`gemini-2.0-pro-exp`](#google_ai_gemini_2.0_pro_exp_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-pro-exp-02-05`](#google_ai_gemini_2.0_pro_exp_02_05_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.5-flash-preview-tts`](#google_ai_gemini_2.5_flash_preview_tts_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`gemini-2.5-pro-exp-03-25`](#google_ai_gemini_2.5_pro_exp_03_25_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.5-pro-preview-tts`](#google_ai_gemini_2.5_pro_preview_tts_details) | ❌ | ❌ | ❌ | 65,536 | 2025-06-20 |-| [`gemini-exp-1206`](#google_ai_gemini_exp_1206_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-pro-vision`](#google_ai_gemini_pro_vision_details) | ❌ | ❌ | ❌ | 12,288 | 2025-06-20 |+| `gemini-1.5-pro` | ✅ | ✅ | ✅ | 2,000,000 | 2025-06-20 |+| `gemini-1.5-pro-002` | ✅ | ✅ | ✅ | 2,000,000 | 2025-06-20 |+| `gemini-1.5-pro-latest` | ✅ | ✅ | ✅ | 2,000,000 | 2025-06-20 |+| `gemini-2.5-flash-preview-04-17-thinking` | ✅ | ✅ | ✅ | 1,048,576 | 2025-06-20 |+| `gemini-2.5-pro-preview-03-25` | ✅ | ✅ | ✅ | 1,048,576 | 2025-06-20 |+| `gemini-2.5-pro-preview-05-06` | ✅ | ✅ | ✅ | 1,048,576 | 2025-06-20 |+| `gemini-2.5-flash-preview-05-20` | ✅ | ⚠️ | ✅ | 1,048,576 | 2025-06-20 |+| `gemini-2.0-flash-thinking-exp` | ⚠️ | ✅ | ✅ | 1,048,576 | 2025-06-20 |+| `gemini-2.0-flash-thinking-exp-1219` | ⚠️ | ✅ | ✅ | 1,048,576 | 2025-06-20 |+| `gemini-2.0-flash-thinking-exp-01-21` | ⚠️ | ✅ | ⚠️ | 1,048,576 | 2025-06-20 |+| `gemini-2.5-flash-preview-04-17` | ⚠️ | ✅ | ⚠️ | 1,048,576 | 2025-06-20 |+| `gemini-2.5-pro-preview-06-05` | ⚠️ | ✅ | ⚠️ | 1,048,576 | 2025-06-20 |+| `gemini-1.0-pro-vision-latest` | ❌ | ❌ | ❌ | 12,288 | 2025-06-20 |+| `gemini-1.5-flash` | ❌ | ❌ | ❌ | 1,000,000 | 2025-06-20 |+| `gemini-1.5-flash-002` | ❌ | ❌ | ❌ | 1,000,000 | 2025-06-20 |+| `gemini-1.5-flash-8b` | ❌ | ❌ | ❌ | 1,000,000 | 2025-06-20 |+| `gemini-1.5-flash-8b-001` | ❌ | ❌ | ❌ | 1,000,000 | 2025-06-20 |+| `gemini-1.5-flash-8b-latest` | ❌ | ❌ | ❌ | 1,000,000 | 2025-06-20 |+| `gemini-1.5-flash-latest` | ❌ | ❌ | ❌ | 1,000,000 | 2025-06-20 |+| `gemini-2.0-flash` | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |+| `gemini-2.0-flash-001` | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |+| `gemini-2.0-flash-exp` | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |+| `gemini-2.0-flash-exp-image-generation` | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |+| `gemini-2.0-flash-lite` | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |+| `gemini-2.0-flash-lite-001` | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |+| `gemini-2.0-flash-lite-preview` | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |+| `gemini-2.0-flash-lite-preview-02-05` | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |+| `gemini-2.0-flash-preview-image-generation` | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |+| `gemini-2.0-pro-exp` | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |+| `gemini-2.0-pro-exp-02-05` | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |+| `gemini-2.5-flash-preview-tts` | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |+| `gemini-2.5-pro-exp-03-25` | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |+| `gemini-2.5-pro-preview-tts` | ❌ | ❌ | ❌ | 65,536 | 2025-06-20 |+| `gemini-exp-1206` | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |+| `gemini-pro-vision` | ❌ | ❌ | ❌ | 12,288 | 2025-06-20 |### Detailed Test Results-#### gemini-1.5-pro {#google_ai_gemini_1.5_pro_details}+#### gemini-1.5-pro### Basic@@ -1165,7 +1165,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-1.5-pro-002 {#google_ai_gemini_1.5_pro_002_details}+#### gemini-1.5-pro-002### Basic@@ -1192,7 +1192,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-1.5-pro-latest {#google_ai_gemini_1.5_pro_latest_details}+#### gemini-1.5-pro-latest### Basic@@ -1219,7 +1219,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-2.5-flash-preview-04-17-thinking {#google_ai_gemini_2.5_flash_preview_04_17_thinking_details}+#### gemini-2.5-flash-preview-04-17-thinking### Basic@@ -1246,7 +1246,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-2.5-pro-preview-03-25 {#google_ai_gemini_2.5_pro_preview_03_25_details}+#### gemini-2.5-pro-preview-03-25### Basic@@ -1273,7 +1273,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-2.5-pro-preview-05-06 {#google_ai_gemini_2.5_pro_preview_05_06_details}+#### gemini-2.5-pro-preview-05-06### Basic@@ -1300,7 +1300,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-2.5-flash-preview-05-20 {#google_ai_gemini_2.5_flash_preview_05_20_details}+#### gemini-2.5-flash-preview-05-20### Basic@@ -1327,7 +1327,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-2.0-flash-thinking-exp {#google_ai_gemini_2.0_flash_thinking_exp_details}+#### gemini-2.0-flash-thinking-exp### Basic@@ -1354,7 +1354,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-2.0-flash-thinking-exp-1219 {#google_ai_gemini_2.0_flash_thinking_exp_1219_details}+#### gemini-2.0-flash-thinking-exp-1219### Basic@@ -1381,7 +1381,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-2.0-flash-thinking-exp-01-21 {#google_ai_gemini_2.0_flash_thinking_exp_01_21_details}+#### gemini-2.0-flash-thinking-exp-01-21### Basic@@ -1408,7 +1408,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-2.5-flash-preview-04-17 {#google_ai_gemini_2.5_flash_preview_04_17_details}+#### gemini-2.5-flash-preview-04-17### Basic@@ -1435,7 +1435,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-2.5-pro-preview-06-05 {#google_ai_gemini_2.5_pro_preview_06_05_details}+#### gemini-2.5-pro-preview-06-05### Basic@@ -1462,7 +1462,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-1.0-pro-vision-latest {#google_ai_gemini_1.0_pro_vision_latest_details}+#### gemini-1.0-pro-vision-latest### Basic@@ -1489,7 +1489,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-1.5-flash {#google_ai_gemini_1.5_flash_details}+#### gemini-1.5-flash### Basic@@ -1516,7 +1516,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-1.5-flash-002 {#google_ai_gemini_1.5_flash_002_details}+#### gemini-1.5-flash-002### Basic@@ -1543,7 +1543,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-1.5-flash-8b {#google_ai_gemini_1.5_flash_8b_details}+#### gemini-1.5-flash-8b### Basic@@ -1570,7 +1570,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-1.5-flash-8b-001 {#google_ai_gemini_1.5_flash_8b_001_details}+#### gemini-1.5-flash-8b-001### Basic@@ -1597,7 +1597,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-1.5-flash-8b-latest {#google_ai_gemini_1.5_flash_8b_latest_details}+#### gemini-1.5-flash-8b-latest### Basic@@ -1624,7 +1624,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-1.5-flash-latest {#google_ai_gemini_1.5_flash_latest_details}+#### gemini-1.5-flash-latest### Basic@@ -1651,7 +1651,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-2.0-flash {#google_ai_gemini_2.0_flash_details}+#### gemini-2.0-flash### Basic@@ -1678,7 +1678,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-2.0-flash-001 {#google_ai_gemini_2.0_flash_001_details}+#### gemini-2.0-flash-001### Basic@@ -1705,7 +1705,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-2.0-flash-exp {#google_ai_gemini_2.0_flash_exp_details}+#### gemini-2.0-flash-exp### Basic@@ -1732,7 +1732,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-2.0-flash-exp-image-generation {#google_ai_gemini_2.0_flash_exp_image_generation_details}+#### gemini-2.0-flash-exp-image-generation### Basic@@ -1759,7 +1759,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-2.0-flash-lite {#google_ai_gemini_2.0_flash_lite_details}+#### gemini-2.0-flash-lite### Basic@@ -1786,7 +1786,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-2.0-flash-lite-001 {#google_ai_gemini_2.0_flash_lite_001_details}+#### gemini-2.0-flash-lite-001### Basic@@ -1813,7 +1813,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-2.0-flash-lite-preview {#google_ai_gemini_2.0_flash_lite_preview_details}+#### gemini-2.0-flash-lite-preview### Basic@@ -1840,7 +1840,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-2.0-flash-lite-preview-02-05 {#google_ai_gemini_2.0_flash_lite_preview_02_05_details}+#### gemini-2.0-flash-lite-preview-02-05### Basic@@ -1867,7 +1867,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-2.0-flash-preview-image-generation {#google_ai_gemini_2.0_flash_preview_image_generation_details}+#### gemini-2.0-flash-preview-image-generation### Basic@@ -1894,7 +1894,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-2.0-pro-exp {#google_ai_gemini_2.0_pro_exp_details}+#### gemini-2.0-pro-exp### Basic@@ -1921,7 +1921,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-2.0-pro-exp-02-05 {#google_ai_gemini_2.0_pro_exp_02_05_details}+#### gemini-2.0-pro-exp-02-05### Basic@@ -1948,7 +1948,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-2.5-flash-preview-tts {#google_ai_gemini_2.5_flash_preview_tts_details}+#### gemini-2.5-flash-preview-tts### Basic@@ -1975,7 +1975,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-2.5-pro-exp-03-25 {#google_ai_gemini_2.5_pro_exp_03_25_details}+#### gemini-2.5-pro-exp-03-25### Basic@@ -2002,7 +2002,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-2.5-pro-preview-tts {#google_ai_gemini_2.5_pro_preview_tts_details}+#### gemini-2.5-pro-preview-tts### Basic@@ -2029,7 +2029,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-exp-1206 {#google_ai_gemini_exp_1206_details}+#### gemini-exp-1206### Basic@@ -2056,7 +2056,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemini-pro-vision {#google_ai_gemini_pro_vision_details}+#### gemini-pro-vision### Basic@@ -2090,11 +2090,11 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025| Model | Basic | Token Streaming | Multimodal | Context Window | Last Scanned ||---------------------------------------------------|:---:|:-------------:|:--------:|:------------:|:----------:|-| [`letta-free`](#letta_letta_free_details) | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `letta-free` | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |### Detailed Test Results-#### letta-free {#letta_letta_free_details}+#### letta-free### Basic@@ -2128,65 +2128,65 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025| Model | Basic | Token Streaming | Multimodal | Context Window | Last Scanned ||---------------------------------------------------|:---:|:-------------:|:--------:|:------------:|:----------:|-| [`Qwen/Qwen2.5-72B-Instruct-Turbo`](#together_qwen_qwen2.5_72b_instruct_turbo_details) | ✅ | ✅ | ⚠️ | 131,072 | 2025-06-20 |-| [`arcee-ai/virtuoso-large`](#together_arcee_ai_virtuoso_large_details) | ⚠️ | ✅ | ✅ | 131,072 | 2025-06-20 |-| [`Qwen/QwQ-32B`](#together_qwen_qwq_32b_details) | ⚠️ | ✅ | ⚠️ | 131,072 | 2025-06-20 |-| [`Qwen/Qwen2.5-7B-Instruct-Turbo`](#together_qwen_qwen2.5_7b_instruct_turbo_details) | ⚠️ | ✅ | ⚠️ | 32,768 | 2025-06-20 |-| [`Qwen/Qwen2.5-Coder-32B-Instruct`](#together_qwen_qwen2.5_coder_32b_instruct_details) | ⚠️ | ✅ | ⚠️ | 16,384 | 2025-06-20 |-| [`arcee-ai/coder-large`](#together_arcee_ai_coder_large_details) | ⚠️ | ✅ | ⚠️ | 32,768 | 2025-06-20 |-| [`arcee_ai/arcee-spotlight`](#together_arcee_ai_arcee_spotlight_details) | ⚠️ | ✅ | ⚠️ | 131,072 | 2025-06-20 |-| [`meta-llama/Llama-3.2-3B-Instruct-Turbo`](#together_meta_llama_llama_3.2_3b_instruct_turbo_details) | ⚠️ | ✅ | ❌ | 131,072 | 2025-06-20 |-| [`meta-llama/Llama-3.3-70B-Instruct-Turbo`](#together_meta_llama_llama_3.3_70b_instruct_turbo_details) | ⚠️ | ✅ | ❌ | 131,072 | 2025-06-20 |-| [`meta-llama/Llama-3.3-70B-Instruct-Turbo-Free`](#together_meta_llama_llama_3.3_70b_instruct_turbo_free_details) | ⚠️ | ✅ | ❌ | 131,072 | 2025-06-20 |-| [`meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo`](#together_meta_llama_meta_llama_3.1_405b_instruct_turbo_details) | ⚠️ | ✅ | ❌ | 130,815 | 2025-06-20 |-| [`meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo`](#together_meta_llama_meta_llama_3.1_70b_instruct_turbo_details) | ⚠️ | ✅ | ❌ | 131,072 | 2025-06-20 |-| [`nvidia/Llama-3.1-Nemotron-70B-Instruct-HF`](#together_nvidia_llama_3.1_nemotron_70b_instruct_hf_details) | ⚠️ | ✅ | ❌ | 32,768 | 2025-06-20 |-| [`arcee-ai/virtuoso-medium-v2`](#together_arcee_ai_virtuoso_medium_v2_details) | ⚠️ | ⚠️ | ✅ | 131,072 | 2025-06-20 |-| [`meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8`](#together_meta_llama_llama_4_maverick_17b_128e_instruct_fp8_details) | ⚠️ | ❌ | ✅ | 1,048,576 | 2025-06-20 |-| [`Qwen/Qwen3-235B-A22B-fp8-tput`](#together_qwen_qwen3_235b_a22b_fp8_tput_details) | ⚠️ | ⚠️ | ❌ | 40,960 | 2025-06-20 |-| [`deepseek-ai/DeepSeek-V3`](#together_deepseek_ai_deepseek_v3_details) | ⚠️ | ⚠️ | ❌ | 131,072 | 2025-06-20 |-| [`meta-llama/Llama-4-Scout-17B-16E-Instruct`](#together_meta_llama_llama_4_scout_17b_16e_instruct_details) | ⚠️ | ⚠️ | ❌ | 1,048,576 | 2025-06-20 |-| [`meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo`](#together_meta_llama_meta_llama_3.1_8b_instruct_turbo_details) | ⚠️ | ⚠️ | ❌ | 131,072 | 2025-06-20 |-| [`mistralai/Mixtral-8x7B-Instruct-v0.1`](#together_mistralai_mixtral_8x7b_instruct_v0.1_details) | ⚠️ | ⚠️ | ❌ | 32,768 | 2025-06-20 |-| [`arcee-ai/caller`](#together_arcee_ai_caller_details) | ❌ | ⚠️ | ❌ | 32,768 | 2025-06-20 |-| [`mistralai/Mistral-Small-24B-Instruct-2501`](#together_mistralai_mistral_small_24b_instruct_2501_details) | ❌ | ⚠️ | ❌ | 32,768 | 2025-06-20 |-| [`NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO`](#together_nousresearch_nous_hermes_2_mixtral_8x7b_dpo_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`Qwen/Qwen2-72B-Instruct`](#together_qwen_qwen2_72b_instruct_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`Qwen/Qwen2-VL-72B-Instruct`](#together_qwen_qwen2_vl_72b_instruct_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`Qwen/Qwen2.5-VL-72B-Instruct`](#together_qwen_qwen2.5_vl_72b_instruct_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`arcee-ai/arcee-blitz`](#together_arcee_ai_arcee_blitz_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`arcee-ai/maestro-reasoning`](#together_arcee_ai_maestro_reasoning_details) | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |-| [`deepseek-ai/DeepSeek-R1`](#together_deepseek_ai_deepseek_r1_details) | ❌ | ❌ | ❌ | 163,840 | 2025-06-20 |-| [`deepseek-ai/DeepSeek-R1-Distill-Llama-70B`](#together_deepseek_ai_deepseek_r1_distill_llama_70b_details) | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |-| [`deepseek-ai/DeepSeek-R1-Distill-Llama-70B-free`](#together_deepseek_ai_deepseek_r1_distill_llama_70b_free_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B`](#together_deepseek_ai_deepseek_r1_distill_qwen_1.5b_details) | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |-| [`deepseek-ai/DeepSeek-R1-Distill-Qwen-14B`](#together_deepseek_ai_deepseek_r1_distill_qwen_14b_details) | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |-| [`deepseek-ai/DeepSeek-V3-p-dp`](#together_deepseek_ai_deepseek_v3_p_dp_details) | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |-| [`google/gemma-2-27b-it`](#together_google_gemma_2_27b_it_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`lgai/exaone-3-5-32b-instruct`](#together_lgai_exaone_3_5_32b_instruct_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`lgai/exaone-deep-32b`](#together_lgai_exaone_deep_32b_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`marin-community/marin-8b-instruct`](#together_marin_community_marin_8b_instruct_details) | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |-| [`meta-llama/Llama-3-70b-chat-hf`](#together_meta_llama_llama_3_70b_chat_hf_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`meta-llama/Llama-3-8b-chat-hf`](#together_meta_llama_llama_3_8b_chat_hf_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`meta-llama/Llama-3.2-11B-Vision-Instruct-Turbo`](#together_meta_llama_llama_3.2_11b_vision_instruct_turbo_details) | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |-| [`meta-llama/Llama-3.2-90B-Vision-Instruct-Turbo`](#together_meta_llama_llama_3.2_90b_vision_instruct_turbo_details) | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |-| [`meta-llama/Llama-Vision-Free`](#together_meta_llama_llama_vision_free_details) | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |-| [`meta-llama/Meta-Llama-3-70B-Instruct-Turbo`](#together_meta_llama_meta_llama_3_70b_instruct_turbo_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`meta-llama/Meta-Llama-3-8B-Instruct-Lite`](#together_meta_llama_meta_llama_3_8b_instruct_lite_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`mistralai/Mistral-7B-Instruct-v0.1`](#together_mistralai_mistral_7b_instruct_v0.1_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`mistralai/Mistral-7B-Instruct-v0.2`](#together_mistralai_mistral_7b_instruct_v0.2_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`mistralai/Mistral-7B-Instruct-v0.3`](#together_mistralai_mistral_7b_instruct_v0.3_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`perplexity-ai/r1-1776`](#together_perplexity_ai_r1_1776_details) | ❌ | ❌ | ❌ | 163,840 | 2025-06-20 |-| [`scb10x/scb10x-llama3-1-typhoon2-70b-instruct`](#together_scb10x_scb10x_llama3_1_typhoon2_70b_instruct_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`scb10x/scb10x-typhoon-2-1-gemma3-12b`](#together_scb10x_scb10x_typhoon_2_1_gemma3_12b_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`togethercomputer/MoA-1`](#together_togethercomputer_moa_1_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`togethercomputer/MoA-1-Turbo`](#together_togethercomputer_moa_1_turbo_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`togethercomputer/Refuel-Llm-V2`](#together_togethercomputer_refuel_llm_v2_details) | ❌ | ❌ | ❌ | 16,384 | 2025-06-20 |-| [`togethercomputer/Refuel-Llm-V2-Small`](#together_togethercomputer_refuel_llm_v2_small_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `Qwen/Qwen2.5-72B-Instruct-Turbo` | ✅ | ✅ | ⚠️ | 131,072 | 2025-06-20 |+| `arcee-ai/virtuoso-large` | ⚠️ | ✅ | ✅ | 131,072 | 2025-06-20 |+| `Qwen/QwQ-32B` | ⚠️ | ✅ | ⚠️ | 131,072 | 2025-06-20 |+| `Qwen/Qwen2.5-7B-Instruct-Turbo` | ⚠️ | ✅ | ⚠️ | 32,768 | 2025-06-20 |+| `Qwen/Qwen2.5-Coder-32B-Instruct` | ⚠️ | ✅ | ⚠️ | 16,384 | 2025-06-20 |+| `arcee-ai/coder-large` | ⚠️ | ✅ | ⚠️ | 32,768 | 2025-06-20 |+| `arcee_ai/arcee-spotlight` | ⚠️ | ✅ | ⚠️ | 131,072 | 2025-06-20 |+| `meta-llama/Llama-3.2-3B-Instruct-Turbo` | ⚠️ | ✅ | ❌ | 131,072 | 2025-06-20 |+| `meta-llama/Llama-3.3-70B-Instruct-Turbo` | ⚠️ | ✅ | ❌ | 131,072 | 2025-06-20 |+| `meta-llama/Llama-3.3-70B-Instruct-Turbo-Free` | ⚠️ | ✅ | ❌ | 131,072 | 2025-06-20 |+| `meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo` | ⚠️ | ✅ | ❌ | 130,815 | 2025-06-20 |+| `meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo` | ⚠️ | ✅ | ❌ | 131,072 | 2025-06-20 |+| `nvidia/Llama-3.1-Nemotron-70B-Instruct-HF` | ⚠️ | ✅ | ❌ | 32,768 | 2025-06-20 |+| `arcee-ai/virtuoso-medium-v2` | ⚠️ | ⚠️ | ✅ | 131,072 | 2025-06-20 |+| `meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8` | ⚠️ | ❌ | ✅ | 1,048,576 | 2025-06-20 |+| `Qwen/Qwen3-235B-A22B-fp8-tput` | ⚠️ | ⚠️ | ❌ | 40,960 | 2025-06-20 |+| `deepseek-ai/DeepSeek-V3` | ⚠️ | ⚠️ | ❌ | 131,072 | 2025-06-20 |+| `meta-llama/Llama-4-Scout-17B-16E-Instruct` | ⚠️ | ⚠️ | ❌ | 1,048,576 | 2025-06-20 |+| `meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo` | ⚠️ | ⚠️ | ❌ | 131,072 | 2025-06-20 |+| `mistralai/Mixtral-8x7B-Instruct-v0.1` | ⚠️ | ⚠️ | ❌ | 32,768 | 2025-06-20 |+| `arcee-ai/caller` | ❌ | ⚠️ | ❌ | 32,768 | 2025-06-20 |+| `mistralai/Mistral-Small-24B-Instruct-2501` | ❌ | ⚠️ | ❌ | 32,768 | 2025-06-20 |+| `NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO` | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |+| `Qwen/Qwen2-72B-Instruct` | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |+| `Qwen/Qwen2-VL-72B-Instruct` | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |+| `Qwen/Qwen2.5-VL-72B-Instruct` | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |+| `arcee-ai/arcee-blitz` | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |+| `arcee-ai/maestro-reasoning` | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |+| `deepseek-ai/DeepSeek-R1` | ❌ | ❌ | ❌ | 163,840 | 2025-06-20 |+| `deepseek-ai/DeepSeek-R1-Distill-Llama-70B` | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |+| `deepseek-ai/DeepSeek-R1-Distill-Llama-70B-free` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B` | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |+| `deepseek-ai/DeepSeek-R1-Distill-Qwen-14B` | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |+| `deepseek-ai/DeepSeek-V3-p-dp` | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |+| `google/gemma-2-27b-it` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `lgai/exaone-3-5-32b-instruct` | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |+| `lgai/exaone-deep-32b` | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |+| `marin-community/marin-8b-instruct` | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |+| `meta-llama/Llama-3-70b-chat-hf` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `meta-llama/Llama-3-8b-chat-hf` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `meta-llama/Llama-3.2-11B-Vision-Instruct-Turbo` | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |+| `meta-llama/Llama-3.2-90B-Vision-Instruct-Turbo` | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |+| `meta-llama/Llama-Vision-Free` | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |+| `meta-llama/Meta-Llama-3-70B-Instruct-Turbo` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `meta-llama/Meta-Llama-3-8B-Instruct-Lite` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `mistralai/Mistral-7B-Instruct-v0.1` | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |+| `mistralai/Mistral-7B-Instruct-v0.2` | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |+| `mistralai/Mistral-7B-Instruct-v0.3` | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |+| `perplexity-ai/r1-1776` | ❌ | ❌ | ❌ | 163,840 | 2025-06-20 |+| `scb10x/scb10x-llama3-1-typhoon2-70b-instruct` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `scb10x/scb10x-typhoon-2-1-gemma3-12b` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `togethercomputer/MoA-1` | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |+| `togethercomputer/MoA-1-Turbo` | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |+| `togethercomputer/Refuel-Llm-V2` | ❌ | ❌ | ❌ | 16,384 | 2025-06-20 |+| `togethercomputer/Refuel-Llm-V2-Small` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |### Detailed Test Results-#### Qwen/Qwen2.5-72B-Instruct-Turbo {#together_qwen_qwen2.5_72b_instruct_turbo_details}+#### Qwen/Qwen2.5-72B-Instruct-Turbo### Basic@@ -2213,7 +2213,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### arcee-ai/virtuoso-large {#together_arcee_ai_virtuoso_large_details}+#### arcee-ai/virtuoso-large### Basic@@ -2240,7 +2240,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### Qwen/QwQ-32B {#together_qwen_qwq_32b_details}+#### Qwen/QwQ-32B### Basic@@ -2267,7 +2267,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### Qwen/Qwen2.5-7B-Instruct-Turbo {#together_qwen_qwen2.5_7b_instruct_turbo_details}+#### Qwen/Qwen2.5-7B-Instruct-Turbo### Basic@@ -2294,7 +2294,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### Qwen/Qwen2.5-Coder-32B-Instruct {#together_qwen_qwen2.5_coder_32b_instruct_details}+#### Qwen/Qwen2.5-Coder-32B-Instruct### Basic@@ -2321,7 +2321,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### arcee-ai/coder-large {#together_arcee_ai_coder_large_details}+#### arcee-ai/coder-large### Basic@@ -2348,7 +2348,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### arcee_ai/arcee-spotlight {#together_arcee_ai_arcee_spotlight_details}+#### arcee_ai/arcee-spotlight### Basic@@ -2375,7 +2375,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### meta-llama/Llama-3.2-3B-Instruct-Turbo {#together_meta_llama_llama_3.2_3b_instruct_turbo_details}+#### meta-llama/Llama-3.2-3B-Instruct-Turbo### Basic@@ -2402,7 +2402,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### meta-llama/Llama-3.3-70B-Instruct-Turbo {#together_meta_llama_llama_3.3_70b_instruct_turbo_details}+#### meta-llama/Llama-3.3-70B-Instruct-Turbo### Basic@@ -2429,7 +2429,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### meta-llama/Llama-3.3-70B-Instruct-Turbo-Free {#together_meta_llama_llama_3.3_70b_instruct_turbo_free_details}+#### meta-llama/Llama-3.3-70B-Instruct-Turbo-Free### Basic@@ -2456,7 +2456,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo {#together_meta_llama_meta_llama_3.1_405b_instruct_turbo_details}+#### meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo### Basic@@ -2483,7 +2483,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo {#together_meta_llama_meta_llama_3.1_70b_instruct_turbo_details}+#### meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo### Basic@@ -2510,7 +2510,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### nvidia/Llama-3.1-Nemotron-70B-Instruct-HF {#together_nvidia_llama_3.1_nemotron_70b_instruct_hf_details}+#### nvidia/Llama-3.1-Nemotron-70B-Instruct-HF### Basic@@ -2537,7 +2537,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### arcee-ai/virtuoso-medium-v2 {#together_arcee_ai_virtuoso_medium_v2_details}+#### arcee-ai/virtuoso-medium-v2### Basic@@ -2564,7 +2564,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8 {#together_meta_llama_llama_4_maverick_17b_128e_instruct_fp8_details}+#### meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8### Basic@@ -2591,7 +2591,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ✅ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ✅ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### Qwen/Qwen3-235B-A22B-fp8-tput {#together_qwen_qwen3_235b_a22b_fp8_tput_details}+#### Qwen/Qwen3-235B-A22B-fp8-tput### Basic@@ -2618,7 +2618,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### deepseek-ai/DeepSeek-V3 {#together_deepseek_ai_deepseek_v3_details}+#### deepseek-ai/DeepSeek-V3### Basic@@ -2645,7 +2645,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### meta-llama/Llama-4-Scout-17B-16E-Instruct {#together_meta_llama_llama_4_scout_17b_16e_instruct_details}+#### meta-llama/Llama-4-Scout-17B-16E-Instruct### Basic@@ -2672,7 +2672,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo {#together_meta_llama_meta_llama_3.1_8b_instruct_turbo_details}+#### meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo### Basic@@ -2699,7 +2699,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### mistralai/Mixtral-8x7B-Instruct-v0.1 {#together_mistralai_mixtral_8x7b_instruct_v0.1_details}+#### mistralai/Mixtral-8x7B-Instruct-v0.1### Basic@@ -2726,7 +2726,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### arcee-ai/caller {#together_arcee_ai_caller_details}+#### arcee-ai/caller### Basic@@ -2753,7 +2753,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### mistralai/Mistral-Small-24B-Instruct-2501 {#together_mistralai_mistral_small_24b_instruct_2501_details}+#### mistralai/Mistral-Small-24B-Instruct-2501### Basic@@ -2780,7 +2780,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO {#together_nousresearch_nous_hermes_2_mixtral_8x7b_dpo_details}+#### NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO### Basic@@ -2807,7 +2807,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### Qwen/Qwen2-72B-Instruct {#together_qwen_qwen2_72b_instruct_details}+#### Qwen/Qwen2-72B-Instruct### Basic@@ -2834,7 +2834,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### Qwen/Qwen2-VL-72B-Instruct {#together_qwen_qwen2_vl_72b_instruct_details}+#### Qwen/Qwen2-VL-72B-Instruct### Basic@@ -2861,7 +2861,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### Qwen/Qwen2.5-VL-72B-Instruct {#together_qwen_qwen2.5_vl_72b_instruct_details}+#### Qwen/Qwen2.5-VL-72B-Instruct### Basic@@ -2888,7 +2888,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### arcee-ai/arcee-blitz {#together_arcee_ai_arcee_blitz_details}+#### arcee-ai/arcee-blitz### Basic@@ -2915,7 +2915,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### arcee-ai/maestro-reasoning {#together_arcee_ai_maestro_reasoning_details}+#### arcee-ai/maestro-reasoning### Basic@@ -2942,7 +2942,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### deepseek-ai/DeepSeek-R1 {#together_deepseek_ai_deepseek_r1_details}+#### deepseek-ai/DeepSeek-R1### Basic@@ -2969,7 +2969,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### deepseek-ai/DeepSeek-R1-Distill-Llama-70B {#together_deepseek_ai_deepseek_r1_distill_llama_70b_details}+#### deepseek-ai/DeepSeek-R1-Distill-Llama-70B### Basic@@ -2996,7 +2996,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### deepseek-ai/DeepSeek-R1-Distill-Llama-70B-free {#together_deepseek_ai_deepseek_r1_distill_llama_70b_free_details}+#### deepseek-ai/DeepSeek-R1-Distill-Llama-70B-free### Basic@@ -3023,7 +3023,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B {#together_deepseek_ai_deepseek_r1_distill_qwen_1.5b_details}+#### deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B### Basic@@ -3050,7 +3050,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### deepseek-ai/DeepSeek-R1-Distill-Qwen-14B {#together_deepseek_ai_deepseek_r1_distill_qwen_14b_details}+#### deepseek-ai/DeepSeek-R1-Distill-Qwen-14B### Basic@@ -3077,7 +3077,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### deepseek-ai/DeepSeek-V3-p-dp {#together_deepseek_ai_deepseek_v3_p_dp_details}+#### deepseek-ai/DeepSeek-V3-p-dp### Basic@@ -3104,7 +3104,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### google/gemma-2-27b-it {#together_google_gemma_2_27b_it_details}+#### google/gemma-2-27b-it### Basic@@ -3131,7 +3131,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### lgai/exaone-3-5-32b-instruct {#together_lgai_exaone_3_5_32b_instruct_details}+#### lgai/exaone-3-5-32b-instruct### Basic@@ -3158,7 +3158,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### lgai/exaone-deep-32b {#together_lgai_exaone_deep_32b_details}+#### lgai/exaone-deep-32b### Basic@@ -3185,7 +3185,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### marin-community/marin-8b-instruct {#together_marin_community_marin_8b_instruct_details}+#### marin-community/marin-8b-instruct### Basic@@ -3212,7 +3212,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### meta-llama/Llama-3-70b-chat-hf {#together_meta_llama_llama_3_70b_chat_hf_details}+#### meta-llama/Llama-3-70b-chat-hf### Basic@@ -3239,7 +3239,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### meta-llama/Llama-3-8b-chat-hf {#together_meta_llama_llama_3_8b_chat_hf_details}+#### meta-llama/Llama-3-8b-chat-hf### Basic@@ -3266,7 +3266,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### meta-llama/Llama-3.2-11B-Vision-Instruct-Turbo {#together_meta_llama_llama_3.2_11b_vision_instruct_turbo_details}+#### meta-llama/Llama-3.2-11B-Vision-Instruct-Turbo### Basic@@ -3293,7 +3293,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### meta-llama/Llama-3.2-90B-Vision-Instruct-Turbo {#together_meta_llama_llama_3.2_90b_vision_instruct_turbo_details}+#### meta-llama/Llama-3.2-90B-Vision-Instruct-Turbo### Basic@@ -3320,7 +3320,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### meta-llama/Llama-Vision-Free {#together_meta_llama_llama_vision_free_details}+#### meta-llama/Llama-Vision-Free### Basic@@ -3347,7 +3347,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### meta-llama/Meta-Llama-3-70B-Instruct-Turbo {#together_meta_llama_meta_llama_3_70b_instruct_turbo_details}+#### meta-llama/Meta-Llama-3-70B-Instruct-Turbo### Basic@@ -3374,7 +3374,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### meta-llama/Meta-Llama-3-8B-Instruct-Lite {#together_meta_llama_meta_llama_3_8b_instruct_lite_details}+#### meta-llama/Meta-Llama-3-8B-Instruct-Lite### Basic@@ -3401,7 +3401,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### mistralai/Mistral-7B-Instruct-v0.1 {#together_mistralai_mistral_7b_instruct_v0.1_details}+#### mistralai/Mistral-7B-Instruct-v0.1### Basic@@ -3428,7 +3428,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### mistralai/Mistral-7B-Instruct-v0.2 {#together_mistralai_mistral_7b_instruct_v0.2_details}+#### mistralai/Mistral-7B-Instruct-v0.2### Basic@@ -3455,7 +3455,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### mistralai/Mistral-7B-Instruct-v0.3 {#together_mistralai_mistral_7b_instruct_v0.3_details}+#### mistralai/Mistral-7B-Instruct-v0.3### Basic@@ -3482,7 +3482,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### perplexity-ai/r1-1776 {#together_perplexity_ai_r1_1776_details}+#### perplexity-ai/r1-1776### Basic@@ -3509,7 +3509,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### scb10x/scb10x-llama3-1-typhoon2-70b-instruct {#together_scb10x_scb10x_llama3_1_typhoon2_70b_instruct_details}+#### scb10x/scb10x-llama3-1-typhoon2-70b-instruct### Basic@@ -3536,7 +3536,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### scb10x/scb10x-typhoon-2-1-gemma3-12b {#together_scb10x_scb10x_typhoon_2_1_gemma3_12b_details}+#### scb10x/scb10x-typhoon-2-1-gemma3-12b### Basic@@ -3563,7 +3563,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### togethercomputer/MoA-1 {#together_togethercomputer_moa_1_details}+#### togethercomputer/MoA-1### Basic@@ -3590,7 +3590,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### togethercomputer/MoA-1-Turbo {#together_togethercomputer_moa_1_turbo_details}+#### togethercomputer/MoA-1-Turbo### Basic@@ -3617,7 +3617,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### togethercomputer/Refuel-Llm-V2 {#together_togethercomputer_refuel_llm_v2_details}+#### togethercomputer/Refuel-Llm-V2### Basic@@ -3644,7 +3644,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### togethercomputer/Refuel-Llm-V2-Small {#together_togethercomputer_refuel_llm_v2_small_details}+#### togethercomputer/Refuel-Llm-V2-Small### Basic@@ -3678,12 +3678,12 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025| Model | Basic | Token Streaming | Multimodal | Context Window | Last Scanned ||---------------------------------------------------|:---:|:-------------:|:--------:|:------------:|:----------:|-| [`deepseek-chat`](#deepseek_deepseek_chat_details) | ❌ | ❌ | ❌ | 64,000 | 2025-06-20 |-| [`deepseek-reasoner`](#deepseek_deepseek_reasoner_details) | ❌ | ❌ | ❌ | 64,000 | 2025-06-20 |+| `deepseek-chat` | ❌ | ❌ | ❌ | 64,000 | 2025-06-20 |+| `deepseek-reasoner` | ❌ | ❌ | ❌ | 64,000 | 2025-06-20 |### Detailed Test Results-#### deepseek-chat {#deepseek_deepseek_chat_details}+#### deepseek-chat### Basic@@ -3710,7 +3710,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### deepseek-reasoner {#deepseek_deepseek_reasoner_details}+#### deepseek-reasoner### Basic@@ -3744,33 +3744,33 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025| Model | Basic | Token Streaming | Multimodal | Context Window | Last Scanned ||---------------------------------------------------|:---:|:-------------:|:--------:|:------------:|:----------:|-| [`allam-2-7b`](#groq_allam_2_7b_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`compound-beta`](#groq_compound_beta_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`compound-beta-mini`](#groq_compound_beta_mini_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`deepseek-r1-distill-llama-70b`](#groq_deepseek_r1_distill_llama_70b_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`distil-whisper-large-v3-en`](#groq_distil_whisper_large_v3_en_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`gemma2-9b-it`](#groq_gemma2_9b_it_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`llama-3.1-8b-instant`](#groq_llama_3.1_8b_instant_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`llama-3.3-70b-versatile`](#groq_llama_3.3_70b_versatile_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`llama-guard-3-8b`](#groq_llama_guard_3_8b_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`llama3-70b-8192`](#groq_llama3_70b_8192_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`llama3-8b-8192`](#groq_llama3_8b_8192_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`meta-llama/llama-4-maverick-17b-128e-instruct`](#groq_meta_llama_llama_4_maverick_17b_128e_instruct_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`meta-llama/llama-4-scout-17b-16e-instruct`](#groq_meta_llama_llama_4_scout_17b_16e_instruct_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`meta-llama/llama-guard-4-12b`](#groq_meta_llama_llama_guard_4_12b_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`meta-llama/llama-prompt-guard-2-22m`](#groq_meta_llama_llama_prompt_guard_2_22m_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`meta-llama/llama-prompt-guard-2-86m`](#groq_meta_llama_llama_prompt_guard_2_86m_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`mistral-saba-24b`](#groq_mistral_saba_24b_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`playai-tts`](#groq_playai_tts_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`playai-tts-arabic`](#groq_playai_tts_arabic_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`qwen-qwq-32b`](#groq_qwen_qwq_32b_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`qwen/qwen3-32b`](#groq_qwen_qwen3_32b_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`whisper-large-v3`](#groq_whisper_large_v3_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`whisper-large-v3-turbo`](#groq_whisper_large_v3_turbo_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `allam-2-7b` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `compound-beta` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `compound-beta-mini` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `deepseek-r1-distill-llama-70b` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `distil-whisper-large-v3-en` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `gemma2-9b-it` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `llama-3.1-8b-instant` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `llama-3.3-70b-versatile` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `llama-guard-3-8b` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `llama3-70b-8192` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `llama3-8b-8192` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `meta-llama/llama-4-maverick-17b-128e-instruct` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `meta-llama/llama-4-scout-17b-16e-instruct` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `meta-llama/llama-guard-4-12b` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `meta-llama/llama-prompt-guard-2-22m` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `meta-llama/llama-prompt-guard-2-86m` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `mistral-saba-24b` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `playai-tts` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `playai-tts-arabic` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `qwen-qwq-32b` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `qwen/qwen3-32b` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `whisper-large-v3` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |+| `whisper-large-v3-turbo` | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |### Detailed Test Results-#### allam-2-7b {#groq_allam_2_7b_details}+#### allam-2-7b### Basic@@ -3797,7 +3797,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### compound-beta {#groq_compound_beta_details}+#### compound-beta### Basic@@ -3824,7 +3824,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### compound-beta-mini {#groq_compound_beta_mini_details}+#### compound-beta-mini### Basic@@ -3851,7 +3851,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### deepseek-r1-distill-llama-70b {#groq_deepseek_r1_distill_llama_70b_details}+#### deepseek-r1-distill-llama-70b### Basic@@ -3878,7 +3878,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### distil-whisper-large-v3-en {#groq_distil_whisper_large_v3_en_details}+#### distil-whisper-large-v3-en### Basic@@ -3905,7 +3905,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### gemma2-9b-it {#groq_gemma2_9b_it_details}+#### gemma2-9b-it### Basic@@ -3932,7 +3932,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### llama-3.1-8b-instant {#groq_llama_3.1_8b_instant_details}+#### llama-3.1-8b-instant### Basic@@ -3959,7 +3959,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### llama-3.3-70b-versatile {#groq_llama_3.3_70b_versatile_details}+#### llama-3.3-70b-versatile### Basic@@ -3986,7 +3986,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### llama-guard-3-8b {#groq_llama_guard_3_8b_details}+#### llama-guard-3-8b### Basic@@ -4013,7 +4013,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### llama3-70b-8192 {#groq_llama3_70b_8192_details}+#### llama3-70b-8192### Basic@@ -4040,7 +4040,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### llama3-8b-8192 {#groq_llama3_8b_8192_details}+#### llama3-8b-8192### Basic@@ -4067,7 +4067,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### meta-llama/llama-4-maverick-17b-128e-instruct {#groq_meta_llama_llama_4_maverick_17b_128e_instruct_details}+#### meta-llama/llama-4-maverick-17b-128e-instruct### Basic@@ -4094,7 +4094,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### meta-llama/llama-4-scout-17b-16e-instruct {#groq_meta_llama_llama_4_scout_17b_16e_instruct_details}+#### meta-llama/llama-4-scout-17b-16e-instruct### Basic@@ -4121,7 +4121,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### meta-llama/llama-guard-4-12b {#groq_meta_llama_llama_guard_4_12b_details}+#### meta-llama/llama-guard-4-12b### Basic@@ -4148,7 +4148,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### meta-llama/llama-prompt-guard-2-22m {#groq_meta_llama_llama_prompt_guard_2_22m_details}+#### meta-llama/llama-prompt-guard-2-22m### Basic@@ -4175,7 +4175,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### meta-llama/llama-prompt-guard-2-86m {#groq_meta_llama_llama_prompt_guard_2_86m_details}+#### meta-llama/llama-prompt-guard-2-86m### Basic@@ -4202,7 +4202,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### mistral-saba-24b {#groq_mistral_saba_24b_details}+#### mistral-saba-24b### Basic@@ -4229,7 +4229,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### playai-tts {#groq_playai_tts_details}+#### playai-tts### Basic@@ -4256,7 +4256,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### playai-tts-arabic {#groq_playai_tts_arabic_details}+#### playai-tts-arabic### Basic@@ -4283,7 +4283,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### qwen-qwq-32b {#groq_qwen_qwq_32b_details}+#### qwen-qwq-32b### Basic@@ -4310,7 +4310,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### qwen/qwen3-32b {#groq_qwen_qwen3_32b_details}+#### qwen/qwen3-32b### Basic@@ -4337,7 +4337,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### whisper-large-v3 {#groq_whisper_large_v3_details}+#### whisper-large-v3### Basic@@ -4364,7 +4364,7 @@ Ran 2464 tests against 154 models across 7 providers on June 20th, 2025- ❌ [`test_base64_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L440)- ❌ [`test_url_image_input`](https://github.com/letta-ai/letta/blob/kian/add_model_sweep/.github/scripts/model-sweep/model_sweep.py#L417)-#### whisper-large-v3-turbo {#groq_whisper_large_v3_turbo_details}+#### whisper-large-v3-turbo### Basicdiff --git a/.github/scripts/model-sweep/test_markdown.mdx b/.github/scripts/model-sweep/test_markdown.mdxdeleted file mode 100644index 5a25816dc..000000000--- a/.github/scripts/model-sweep/test_markdown.mdx+++ /dev/null@@ -1,216 +0,0 @@-----title: Support Models-generated: 2025-06-16T17:37:43.167565------# Supported Models--## Overview--Letta routinely runs automated scans against available providers and models. These are the results of the latest scan.--Ran 2464 tests against 154 models across 7 providers on June 16th, 2025---## anthropic--| Model | Basic | Token Streaming | Multimodal | Context Window | Last Scanned |-|---------------------------------------------------|:---:|:-------------:|:--------:|:------------:|:----------:|-| `claude-3-5-haiku-20241022` | ✅ | ✅ | ✅ | 200,000 | 2025-06-16 |-| `claude-3-5-sonnet-20241022` | ✅ | ✅ | ✅ | 200,000 | 2025-06-16 |-| `claude-3-7-sonnet-20250219` | ✅ | ✅ | ✅ | 200,000 | 2025-06-16 |-| `claude-sonnet-4-20250514` | ✅ | ✅ | ✅ | 200,000 | 2025-06-16 |-| `claude-opus-4-20250514` | ✅ | ✅ | ⚠️ | 200,000 | 2025-06-16 |-| `claude-3-5-sonnet-20240620` | ⚠️ | ❌ | ✅ | 200,000 | 2025-06-16 |-| `claude-3-haiku-20240307` | ⚠️ | ❌ | ✅ | 200,000 | 2025-06-16 |-| `claude-3-opus-20240229` | ⚠️ | ❌ | ✅ | 200,000 | 2025-06-16 |-| `claude-3-sonnet-20240229` | ❌ | ❌ | ❌ | 200,000 | 2025-06-16 |-------## openai--| Model | Basic | Token Streaming | Multimodal | Context Window | Last Scanned |-|---------------------------------------------------|:---:|:-------------:|:--------:|:------------:|:----------:|-| `gpt-4.1` | ✅ | ✅ | ✅ | 1,047,576 | 2025-06-16 |-| `gpt-4.1-2025-04-14` | ✅ | ✅ | ✅ | 1,047,576 | 2025-06-16 |-| `gpt-4.1-nano-2025-04-14` | ✅ | ✅ | ✅ | 1,047,576 | 2025-06-16 |-| `gpt-4o` | ✅ | ✅ | ✅ | 128,000 | 2025-06-16 |-| `gpt-4o-2024-05-13` | ✅ | ✅ | ✅ | 128,000 | 2025-06-16 |-| `gpt-4-turbo` | ✅ | ✅ | ⚠️ | 8,192 | 2025-06-16 |-| `gpt-4.1-mini` | ✅ | ✅ | ⚠️ | 1,047,576 | 2025-06-16 |-| `gpt-4.5-preview` | ✅ | ✅ | ⚠️ | 128,000 | 2025-06-16 |-| `gpt-4.5-preview-2025-02-27` | ✅ | ✅ | ⚠️ | 128,000 | 2025-06-16 |-| `gpt-4o-2024-08-06` | ✅ | ✅ | ⚠️ | 128,000 | 2025-06-16 |-| `gpt-4-0613` | ✅ | ✅ | ❌ | 8,192 | 2025-06-16 |-| `gpt-4-1106-preview` | ✅ | ✅ | ❌ | 128,000 | 2025-06-16 |-| `gpt-4-turbo-2024-04-09` | ✅ | ⚠️ | ✅ | 128,000 | 2025-06-16 |-| `gpt-4.1-mini-2025-04-14` | ⚠️ | ✅ | ✅ | 1,047,576 | 2025-06-16 |-| `gpt-4.1-nano` | ⚠️ | ✅ | ✅ | 1,047,576 | 2025-06-16 |-| `gpt-4o-2024-11-20` | ⚠️ | ✅ | ✅ | 8,192 | 2025-06-16 |-| `gpt-4-turbo-preview` | ✅ | ⚠️ | ❌ | 128,000 | 2025-06-16 |-| `gpt-4-0125-preview` | ⚠️ | ✅ | ❌ | 128,000 | 2025-06-16 |-| `gpt-4o-mini` | ⚠️ | ⚠️ | ⚠️ | 128,000 | 2025-06-16 |-| `gpt-4o-mini-2024-07-18` | ⚠️ | ⚠️ | ❌ | 128,000 | 2025-06-16 |-| `gpt-4` | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `o1` | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `o1-2024-12-17` | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `o3` | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `o3-2025-04-16` | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `o3-mini` | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `o3-mini-2025-01-31` | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `o3-pro` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `o3-pro-2025-06-10` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-------## google_ai--| Model | Basic | Token Streaming | Multimodal | Context Window | Last Scanned |-|---------------------------------------------------|:---:|:-------------:|:--------:|:------------:|:----------:|-| `gemini-1.5-pro` | ✅ | ✅ | ✅ | 2,000,000 | 2025-06-16 |-| `gemini-1.5-pro-002` | ✅ | ✅ | ✅ | 2,000,000 | 2025-06-16 |-| `gemini-1.5-pro-latest` | ✅ | ✅ | ✅ | 2,000,000 | 2025-06-16 |-| `gemini-2.5-flash-preview-04-17-thinking` | ✅ | ✅ | ✅ | 1,048,576 | 2025-06-16 |-| `gemini-2.5-pro-preview-03-25` | ✅ | ✅ | ✅ | 1,048,576 | 2025-06-16 |-| `gemini-2.5-pro-preview-05-06` | ✅ | ✅ | ✅ | 1,048,576 | 2025-06-16 |-| `gemini-2.5-flash-preview-05-20` | ✅ | ⚠️ | ✅ | 1,048,576 | 2025-06-16 |-| `gemini-2.0-flash-thinking-exp` | ⚠️ | ✅ | ✅ | 1,048,576 | 2025-06-16 |-| `gemini-2.0-flash-thinking-exp-1219` | ⚠️ | ✅ | ✅ | 1,048,576 | 2025-06-16 |-| `gemini-2.0-flash-thinking-exp-01-21` | ⚠️ | ✅ | ⚠️ | 1,048,576 | 2025-06-16 |-| `gemini-2.5-flash-preview-04-17` | ⚠️ | ✅ | ⚠️ | 1,048,576 | 2025-06-16 |-| `gemini-2.5-pro-preview-06-05` | ⚠️ | ✅ | ⚠️ | 1,048,576 | 2025-06-16 |-| `gemini-1.0-pro-vision-latest` | ❌ | ❌ | ❌ | 12,288 | 2025-06-16 |-| `gemini-1.5-flash` | ❌ | ❌ | ❌ | 1,000,000 | 2025-06-16 |-| `gemini-1.5-flash-002` | ❌ | ❌ | ❌ | 1,000,000 | 2025-06-16 |-| `gemini-1.5-flash-8b` | ❌ | ❌ | ❌ | 1,000,000 | 2025-06-16 |-| `gemini-1.5-flash-8b-001` | ❌ | ❌ | ❌ | 1,000,000 | 2025-06-16 |-| `gemini-1.5-flash-8b-latest` | ❌ | ❌ | ❌ | 1,000,000 | 2025-06-16 |-| `gemini-1.5-flash-latest` | ❌ | ❌ | ❌ | 1,000,000 | 2025-06-16 |-| `gemini-2.0-flash` | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-16 |-| `gemini-2.0-flash-001` | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-16 |-| `gemini-2.0-flash-exp` | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-16 |-| `gemini-2.0-flash-exp-image-generation` | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-16 |-| `gemini-2.0-flash-lite` | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-16 |-| `gemini-2.0-flash-lite-001` | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-16 |-| `gemini-2.0-flash-lite-preview` | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-16 |-| `gemini-2.0-flash-lite-preview-02-05` | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-16 |-| `gemini-2.0-flash-preview-image-generation` | ❌ | ❌ | ❌ | 32,768 | 2025-06-16 |-| `gemini-2.0-pro-exp` | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-16 |-| `gemini-2.0-pro-exp-02-05` | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-16 |-| `gemini-2.5-flash-preview-tts` | ❌ | ❌ | ❌ | 32,768 | 2025-06-16 |-| `gemini-2.5-pro-exp-03-25` | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-16 |-| `gemini-2.5-pro-preview-tts` | ❌ | ❌ | ❌ | 65,536 | 2025-06-16 |-| `gemini-exp-1206` | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-16 |-| `gemini-pro-vision` | ❌ | ❌ | ❌ | 12,288 | 2025-06-16 |-------## letta--| Model | Basic | Token Streaming | Multimodal | Context Window | Last Scanned |-|---------------------------------------------------|:---:|:-------------:|:--------:|:------------:|:----------:|-| `letta-free` | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-16 |-------## together--| Model | Basic | Token Streaming | Multimodal | Context Window | Last Scanned |-|---------------------------------------------------|:---:|:-------------:|:--------:|:------------:|:----------:|-| `Qwen/Qwen2.5-72B-Instruct-Turbo` | ✅ | ✅ | ⚠️ | 131,072 | 2025-06-16 |-| `arcee-ai/virtuoso-large` | ⚠️ | ✅ | ✅ | 131,072 | 2025-06-16 |-| `Qwen/QwQ-32B` | ⚠️ | ✅ | ⚠️ | 131,072 | 2025-06-16 |-| `Qwen/Qwen2.5-7B-Instruct-Turbo` | ⚠️ | ✅ | ⚠️ | 32,768 | 2025-06-16 |-| `Qwen/Qwen2.5-Coder-32B-Instruct` | ⚠️ | ✅ | ⚠️ | 16,384 | 2025-06-16 |-| `arcee-ai/coder-large` | ⚠️ | ✅ | ⚠️ | 32,768 | 2025-06-16 |-| `arcee_ai/arcee-spotlight` | ⚠️ | ✅ | ⚠️ | 131,072 | 2025-06-16 |-| `meta-llama/Llama-3.2-3B-Instruct-Turbo` | ⚠️ | ✅ | ❌ | 131,072 | 2025-06-16 |-| `meta-llama/Llama-3.3-70B-Instruct-Turbo` | ⚠️ | ✅ | ❌ | 131,072 | 2025-06-16 |-| `meta-llama/Llama-3.3-70B-Instruct-Turbo-Free` | ⚠️ | ✅ | ❌ | 131,072 | 2025-06-16 |-| `meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo` | ⚠️ | ✅ | ❌ | 130,815 | 2025-06-16 |-| `meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo` | ⚠️ | ✅ | ❌ | 131,072 | 2025-06-16 |-| `nvidia/Llama-3.1-Nemotron-70B-Instruct-HF` | ⚠️ | ✅ | ❌ | 32,768 | 2025-06-16 |-| `arcee-ai/virtuoso-medium-v2` | ⚠️ | ⚠️ | ✅ | 131,072 | 2025-06-16 |-| `meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8` | ⚠️ | ❌ | ✅ | 1,048,576 | 2025-06-16 |-| `Qwen/Qwen3-235B-A22B-fp8-tput` | ⚠️ | ⚠️ | ❌ | 40,960 | 2025-06-16 |-| `deepseek-ai/DeepSeek-V3` | ⚠️ | ⚠️ | ❌ | 131,072 | 2025-06-16 |-| `meta-llama/Llama-4-Scout-17B-16E-Instruct` | ⚠️ | ⚠️ | ❌ | 1,048,576 | 2025-06-16 |-| `meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo` | ⚠️ | ⚠️ | ❌ | 131,072 | 2025-06-16 |-| `mistralai/Mixtral-8x7B-Instruct-v0.1` | ⚠️ | ⚠️ | ❌ | 32,768 | 2025-06-16 |-| `arcee-ai/caller` | ❌ | ⚠️ | ❌ | 32,768 | 2025-06-16 |-| `mistralai/Mistral-Small-24B-Instruct-2501` | ❌ | ⚠️ | ❌ | 32,768 | 2025-06-16 |-| `NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO` | ❌ | ❌ | ❌ | 32,768 | 2025-06-16 |-| `Qwen/Qwen2-72B-Instruct` | ❌ | ❌ | ❌ | 32,768 | 2025-06-16 |-| `Qwen/Qwen2-VL-72B-Instruct` | ❌ | ❌ | ❌ | 32,768 | 2025-06-16 |-| `Qwen/Qwen2.5-VL-72B-Instruct` | ❌ | ❌ | ❌ | 32,768 | 2025-06-16 |-| `arcee-ai/arcee-blitz` | ❌ | ❌ | ❌ | 32,768 | 2025-06-16 |-| `arcee-ai/maestro-reasoning` | ❌ | ❌ | ❌ | 131,072 | 2025-06-16 |-| `deepseek-ai/DeepSeek-R1` | ❌ | ❌ | ❌ | 163,840 | 2025-06-16 |-| `deepseek-ai/DeepSeek-R1-Distill-Llama-70B` | ❌ | ❌ | ❌ | 131,072 | 2025-06-16 |-| `deepseek-ai/DeepSeek-R1-Distill-Llama-70B-free` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B` | ❌ | ❌ | ❌ | 131,072 | 2025-06-16 |-| `deepseek-ai/DeepSeek-R1-Distill-Qwen-14B` | ❌ | ❌ | ❌ | 131,072 | 2025-06-16 |-| `deepseek-ai/DeepSeek-V3-p-dp` | ❌ | ❌ | ❌ | 131,072 | 2025-06-16 |-| `google/gemma-2-27b-it` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `lgai/exaone-3-5-32b-instruct` | ❌ | ❌ | ❌ | 32,768 | 2025-06-16 |-| `lgai/exaone-deep-32b` | ❌ | ❌ | ❌ | 32,768 | 2025-06-16 |-| `marin-community/marin-8b-instruct` | ❌ | ❌ | ❌ | 131,072 | 2025-06-16 |-| `meta-llama/Llama-3-70b-chat-hf` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `meta-llama/Llama-3-8b-chat-hf` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `meta-llama/Llama-3.2-11B-Vision-Instruct-Turbo` | ❌ | ❌ | ❌ | 131,072 | 2025-06-16 |-| `meta-llama/Llama-3.2-90B-Vision-Instruct-Turbo` | ❌ | ❌ | ❌ | 131,072 | 2025-06-16 |-| `meta-llama/Llama-Vision-Free` | ❌ | ❌ | ❌ | 131,072 | 2025-06-16 |-| `meta-llama/Meta-Llama-3-70B-Instruct-Turbo` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `meta-llama/Meta-Llama-3-8B-Instruct-Lite` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `mistralai/Mistral-7B-Instruct-v0.1` | ❌ | ❌ | ❌ | 32,768 | 2025-06-16 |-| `mistralai/Mistral-7B-Instruct-v0.2` | ❌ | ❌ | ❌ | 32,768 | 2025-06-16 |-| `mistralai/Mistral-7B-Instruct-v0.3` | ❌ | ❌ | ❌ | 32,768 | 2025-06-16 |-| `perplexity-ai/r1-1776` | ❌ | ❌ | ❌ | 163,840 | 2025-06-16 |-| `scb10x/scb10x-llama3-1-typhoon2-70b-instruct` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `scb10x/scb10x-typhoon-2-1-gemma3-12b` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `togethercomputer/MoA-1` | ❌ | ❌ | ❌ | 32,768 | 2025-06-16 |-| `togethercomputer/MoA-1-Turbo` | ❌ | ❌ | ❌ | 32,768 | 2025-06-16 |-| `togethercomputer/Refuel-Llm-V2` | ❌ | ❌ | ❌ | 16,384 | 2025-06-16 |-| `togethercomputer/Refuel-Llm-V2-Small` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-------## deepseek--| Model | Basic | Token Streaming | Multimodal | Context Window | Last Scanned |-|---------------------------------------------------|:---:|:-------------:|:--------:|:------------:|:----------:|-| `deepseek-chat` | ❌ | ❌ | ❌ | 64,000 | 2025-06-16 |-| `deepseek-reasoner` | ❌ | ❌ | ❌ | 64,000 | 2025-06-16 |-------## groq--| Model | Basic | Token Streaming | Multimodal | Context Window | Last Scanned |-|---------------------------------------------------|:---:|:-------------:|:--------:|:------------:|:----------:|-| `allam-2-7b` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `compound-beta` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `compound-beta-mini` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `deepseek-r1-distill-llama-70b` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `distil-whisper-large-v3-en` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `gemma2-9b-it` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `llama-3.1-8b-instant` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `llama-3.3-70b-versatile` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `llama-guard-3-8b` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `llama3-70b-8192` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `llama3-8b-8192` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `meta-llama/llama-4-maverick-17b-128e-instruct` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `meta-llama/llama-4-scout-17b-16e-instruct` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `meta-llama/llama-guard-4-12b` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `meta-llama/llama-prompt-guard-2-22m` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `meta-llama/llama-prompt-guard-2-86m` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `mistral-saba-24b` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `playai-tts` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `playai-tts-arabic` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `qwen-qwq-32b` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `qwen/qwen3-32b` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `whisper-large-v3` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-| `whisper-large-v3-turbo` | ❌ | ❌ | ❌ | 8,192 | 2025-06-16 |-----diff --git a/.github/scripts/model-sweep/test_output.mdx b/.github/scripts/model-sweep/test_output.mdxdeleted file mode 100644index c5fcd41e1..000000000--- a/.github/scripts/model-sweep/test_output.mdx+++ /dev/null@@ -1,4395 +0,0 @@-----title: Support Models-generated: 2025-06-20T15:57:32.626938------# Supported Models--## Overview--Letta routinely runs automated scans against available providers and models. These are the results of the latest scan.--Ran 2464 tests against 154 models across 7 providers on June 20th, 2025---## anthropic--| Model | Basic | Token Streaming | Multimodal | Context Window | Last Scanned |-|---------------------------------------------------|:---:|:-------------:|:--------:|:------------:|:----------:|-| [`claude-3-5-haiku-20241022`](#anthropic_claude_3_5_haiku_20241022_details) | ✅ | ✅ | ✅ | 200,000 | 2025-06-20 |-| [`claude-3-5-sonnet-20241022`](#anthropic_claude_3_5_sonnet_20241022_details) | ✅ | ✅ | ✅ | 200,000 | 2025-06-20 |-| [`claude-3-7-sonnet-20250219`](#anthropic_claude_3_7_sonnet_20250219_details) | ✅ | ✅ | ✅ | 200,000 | 2025-06-20 |-| [`claude-sonnet-4-20250514`](#anthropic_claude_sonnet_4_20250514_details) | ✅ | ✅ | ✅ | 200,000 | 2025-06-20 |-| [`claude-opus-4-20250514`](#anthropic_claude_opus_4_20250514_details) | ✅ | ✅ | ⚠️ | 200,000 | 2025-06-20 |-| [`claude-3-5-sonnet-20240620`](#anthropic_claude_3_5_sonnet_20240620_details) | ⚠️ | ❌ | ✅ | 200,000 | 2025-06-20 |-| [`claude-3-haiku-20240307`](#anthropic_claude_3_haiku_20240307_details) | ⚠️ | ❌ | ✅ | 200,000 | 2025-06-20 |-| [`claude-3-opus-20240229`](#anthropic_claude_3_opus_20240229_details) | ⚠️ | ❌ | ✅ | 200,000 | 2025-06-20 |-| [`claude-3-sonnet-20240229`](#anthropic_claude_3_sonnet_20240229_details) | ❌ | ❌ | ❌ | 200,000 | 2025-06-20 |--### Detailed Test Results--#### claude-3-5-haiku-20241022 {#anthropic_claude_3_5_haiku_20241022_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### claude-3-5-sonnet-20241022 {#anthropic_claude_3_5_sonnet_20241022_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### claude-3-7-sonnet-20250219 {#anthropic_claude_3_7_sonnet_20250219_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### claude-sonnet-4-20250514 {#anthropic_claude_sonnet_4_20250514_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### claude-opus-4-20250514 {#anthropic_claude_opus_4_20250514_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ✅ `test_url_image_input`--#### claude-3-5-sonnet-20240620 {#anthropic_claude_3_5_sonnet_20240620_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### claude-3-haiku-20240307 {#anthropic_claude_3_haiku_20240307_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### claude-3-opus-20240229 {#anthropic_claude_3_opus_20240229_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### claude-3-sonnet-20240229 {#anthropic_claude_3_sonnet_20240229_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--------## openai--| Model | Basic | Token Streaming | Multimodal | Context Window | Last Scanned |-|---------------------------------------------------|:---:|:-------------:|:--------:|:------------:|:----------:|-| [`gpt-4.1`](#openai_gpt_4.1_details) | ✅ | ✅ | ✅ | 1,047,576 | 2025-06-20 |-| [`gpt-4.1-2025-04-14`](#openai_gpt_4.1_2025_04_14_details) | ✅ | ✅ | ✅ | 1,047,576 | 2025-06-20 |-| [`gpt-4.1-nano-2025-04-14`](#openai_gpt_4.1_nano_2025_04_14_details) | ✅ | ✅ | ✅ | 1,047,576 | 2025-06-20 |-| [`gpt-4o`](#openai_gpt_4o_details) | ✅ | ✅ | ✅ | 128,000 | 2025-06-20 |-| [`gpt-4o-2024-05-13`](#openai_gpt_4o_2024_05_13_details) | ✅ | ✅ | ✅ | 128,000 | 2025-06-20 |-| [`gpt-4-turbo`](#openai_gpt_4_turbo_details) | ✅ | ✅ | ⚠️ | 8,192 | 2025-06-20 |-| [`gpt-4.1-mini`](#openai_gpt_4.1_mini_details) | ✅ | ✅ | ⚠️ | 1,047,576 | 2025-06-20 |-| [`gpt-4.5-preview`](#openai_gpt_4.5_preview_details) | ✅ | ✅ | ⚠️ | 128,000 | 2025-06-20 |-| [`gpt-4.5-preview-2025-02-27`](#openai_gpt_4.5_preview_2025_02_27_details) | ✅ | ✅ | ⚠️ | 128,000 | 2025-06-20 |-| [`gpt-4o-2024-08-06`](#openai_gpt_4o_2024_08_06_details) | ✅ | ✅ | ⚠️ | 128,000 | 2025-06-20 |-| [`gpt-4-0613`](#openai_gpt_4_0613_details) | ✅ | ✅ | ❌ | 8,192 | 2025-06-20 |-| [`gpt-4-1106-preview`](#openai_gpt_4_1106_preview_details) | ✅ | ✅ | ❌ | 128,000 | 2025-06-20 |-| [`gpt-4-turbo-2024-04-09`](#openai_gpt_4_turbo_2024_04_09_details) | ✅ | ⚠️ | ✅ | 128,000 | 2025-06-20 |-| [`gpt-4.1-mini-2025-04-14`](#openai_gpt_4.1_mini_2025_04_14_details) | ⚠️ | ✅ | ✅ | 1,047,576 | 2025-06-20 |-| [`gpt-4.1-nano`](#openai_gpt_4.1_nano_details) | ⚠️ | ✅ | ✅ | 1,047,576 | 2025-06-20 |-| [`gpt-4o-2024-11-20`](#openai_gpt_4o_2024_11_20_details) | ⚠️ | ✅ | ✅ | 8,192 | 2025-06-20 |-| [`gpt-4-turbo-preview`](#openai_gpt_4_turbo_preview_details) | ✅ | ⚠️ | ❌ | 128,000 | 2025-06-20 |-| [`gpt-4-0125-preview`](#openai_gpt_4_0125_preview_details) | ⚠️ | ✅ | ❌ | 128,000 | 2025-06-20 |-| [`gpt-4o-mini`](#openai_gpt_4o_mini_details) | ⚠️ | ⚠️ | ⚠️ | 128,000 | 2025-06-20 |-| [`gpt-4o-mini-2024-07-18`](#openai_gpt_4o_mini_2024_07_18_details) | ⚠️ | ⚠️ | ❌ | 128,000 | 2025-06-20 |-| [`gpt-4`](#openai_gpt_4_details) | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`o1`](#openai_o1_details) | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`o1-2024-12-17`](#openai_o1_2024_12_17_details) | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`o3`](#openai_o3_details) | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`o3-2025-04-16`](#openai_o3_2025_04_16_details) | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`o3-mini`](#openai_o3_mini_details) | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`o3-mini-2025-01-31`](#openai_o3_mini_2025_01_31_details) | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`o3-pro`](#openai_o3_pro_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`o3-pro-2025-06-10`](#openai_o3_pro_2025_06_10_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |--### Detailed Test Results--#### gpt-4.1 {#openai_gpt_4.1_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gpt-4.1-2025-04-14 {#openai_gpt_4.1_2025_04_14_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gpt-4.1-nano-2025-04-14 {#openai_gpt_4.1_nano_2025_04_14_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gpt-4o {#openai_gpt_4o_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gpt-4o-2024-05-13 {#openai_gpt_4o_2024_05_13_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gpt-4-turbo {#openai_gpt_4_turbo_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gpt-4.1-mini {#openai_gpt_4.1_mini_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gpt-4.5-preview {#openai_gpt_4.5_preview_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gpt-4.5-preview-2025-02-27 {#openai_gpt_4.5_preview_2025_02_27_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gpt-4o-2024-08-06 {#openai_gpt_4o_2024_08_06_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gpt-4-0613 {#openai_gpt_4_0613_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gpt-4-1106-preview {#openai_gpt_4_1106_preview_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gpt-4-turbo-2024-04-09 {#openai_gpt_4_turbo_2024_04_09_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gpt-4.1-mini-2025-04-14 {#openai_gpt_4.1_mini_2025_04_14_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gpt-4.1-nano {#openai_gpt_4.1_nano_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gpt-4o-2024-11-20 {#openai_gpt_4o_2024_11_20_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gpt-4-turbo-preview {#openai_gpt_4_turbo_preview_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gpt-4-0125-preview {#openai_gpt_4_0125_preview_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gpt-4o-mini {#openai_gpt_4o_mini_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gpt-4o-mini-2024-07-18 {#openai_gpt_4o_mini_2024_07_18_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gpt-4 {#openai_gpt_4_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### o1 {#openai_o1_details}--### Basic--- ❌ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### o1-2024-12-17 {#openai_o1_2024_12_17_details}--### Basic--- ❌ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### o3 {#openai_o3_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### o3-2025-04-16 {#openai_o3_2025_04_16_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### o3-mini {#openai_o3_mini_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### o3-mini-2025-01-31 {#openai_o3_mini_2025_01_31_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### o3-pro {#openai_o3_pro_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### o3-pro-2025-06-10 {#openai_o3_pro_2025_06_10_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--------## google_ai--| Model | Basic | Token Streaming | Multimodal | Context Window | Last Scanned |-|---------------------------------------------------|:---:|:-------------:|:--------:|:------------:|:----------:|-| [`gemini-1.5-pro`](#google_ai_gemini_1.5_pro_details) | ✅ | ✅ | ✅ | 2,000,000 | 2025-06-20 |-| [`gemini-1.5-pro-002`](#google_ai_gemini_1.5_pro_002_details) | ✅ | ✅ | ✅ | 2,000,000 | 2025-06-20 |-| [`gemini-1.5-pro-latest`](#google_ai_gemini_1.5_pro_latest_details) | ✅ | ✅ | ✅ | 2,000,000 | 2025-06-20 |-| [`gemini-2.5-flash-preview-04-17-thinking`](#google_ai_gemini_2.5_flash_preview_04_17_thinking_details) | ✅ | ✅ | ✅ | 1,048,576 | 2025-06-20 |-| [`gemini-2.5-pro-preview-03-25`](#google_ai_gemini_2.5_pro_preview_03_25_details) | ✅ | ✅ | ✅ | 1,048,576 | 2025-06-20 |-| [`gemini-2.5-pro-preview-05-06`](#google_ai_gemini_2.5_pro_preview_05_06_details) | ✅ | ✅ | ✅ | 1,048,576 | 2025-06-20 |-| [`gemini-2.5-flash-preview-05-20`](#google_ai_gemini_2.5_flash_preview_05_20_details) | ✅ | ⚠️ | ✅ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-thinking-exp`](#google_ai_gemini_2.0_flash_thinking_exp_details) | ⚠️ | ✅ | ✅ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-thinking-exp-1219`](#google_ai_gemini_2.0_flash_thinking_exp_1219_details) | ⚠️ | ✅ | ✅ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-thinking-exp-01-21`](#google_ai_gemini_2.0_flash_thinking_exp_01_21_details) | ⚠️ | ✅ | ⚠️ | 1,048,576 | 2025-06-20 |-| [`gemini-2.5-flash-preview-04-17`](#google_ai_gemini_2.5_flash_preview_04_17_details) | ⚠️ | ✅ | ⚠️ | 1,048,576 | 2025-06-20 |-| [`gemini-2.5-pro-preview-06-05`](#google_ai_gemini_2.5_pro_preview_06_05_details) | ⚠️ | ✅ | ⚠️ | 1,048,576 | 2025-06-20 |-| [`gemini-1.0-pro-vision-latest`](#google_ai_gemini_1.0_pro_vision_latest_details) | ❌ | ❌ | ❌ | 12,288 | 2025-06-20 |-| [`gemini-1.5-flash`](#google_ai_gemini_1.5_flash_details) | ❌ | ❌ | ❌ | 1,000,000 | 2025-06-20 |-| [`gemini-1.5-flash-002`](#google_ai_gemini_1.5_flash_002_details) | ❌ | ❌ | ❌ | 1,000,000 | 2025-06-20 |-| [`gemini-1.5-flash-8b`](#google_ai_gemini_1.5_flash_8b_details) | ❌ | ❌ | ❌ | 1,000,000 | 2025-06-20 |-| [`gemini-1.5-flash-8b-001`](#google_ai_gemini_1.5_flash_8b_001_details) | ❌ | ❌ | ❌ | 1,000,000 | 2025-06-20 |-| [`gemini-1.5-flash-8b-latest`](#google_ai_gemini_1.5_flash_8b_latest_details) | ❌ | ❌ | ❌ | 1,000,000 | 2025-06-20 |-| [`gemini-1.5-flash-latest`](#google_ai_gemini_1.5_flash_latest_details) | ❌ | ❌ | ❌ | 1,000,000 | 2025-06-20 |-| [`gemini-2.0-flash`](#google_ai_gemini_2.0_flash_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-001`](#google_ai_gemini_2.0_flash_001_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-exp`](#google_ai_gemini_2.0_flash_exp_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-exp-image-generation`](#google_ai_gemini_2.0_flash_exp_image_generation_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-lite`](#google_ai_gemini_2.0_flash_lite_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-lite-001`](#google_ai_gemini_2.0_flash_lite_001_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-lite-preview`](#google_ai_gemini_2.0_flash_lite_preview_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-lite-preview-02-05`](#google_ai_gemini_2.0_flash_lite_preview_02_05_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-preview-image-generation`](#google_ai_gemini_2.0_flash_preview_image_generation_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`gemini-2.0-pro-exp`](#google_ai_gemini_2.0_pro_exp_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-pro-exp-02-05`](#google_ai_gemini_2.0_pro_exp_02_05_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.5-flash-preview-tts`](#google_ai_gemini_2.5_flash_preview_tts_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`gemini-2.5-pro-exp-03-25`](#google_ai_gemini_2.5_pro_exp_03_25_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.5-pro-preview-tts`](#google_ai_gemini_2.5_pro_preview_tts_details) | ❌ | ❌ | ❌ | 65,536 | 2025-06-20 |-| [`gemini-exp-1206`](#google_ai_gemini_exp_1206_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-pro-vision`](#google_ai_gemini_pro_vision_details) | ❌ | ❌ | ❌ | 12,288 | 2025-06-20 |--### Detailed Test Results--#### gemini-1.5-pro {#google_ai_gemini_1.5_pro_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gemini-1.5-pro-002 {#google_ai_gemini_1.5_pro_002_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gemini-1.5-pro-latest {#google_ai_gemini_1.5_pro_latest_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gemini-2.5-flash-preview-04-17-thinking {#google_ai_gemini_2.5_flash_preview_04_17_thinking_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gemini-2.5-pro-preview-03-25 {#google_ai_gemini_2.5_pro_preview_03_25_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gemini-2.5-pro-preview-05-06 {#google_ai_gemini_2.5_pro_preview_05_06_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gemini-2.5-flash-preview-05-20 {#google_ai_gemini_2.5_flash_preview_05_20_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gemini-2.0-flash-thinking-exp {#google_ai_gemini_2.0_flash_thinking_exp_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gemini-2.0-flash-thinking-exp-1219 {#google_ai_gemini_2.0_flash_thinking_exp_1219_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gemini-2.0-flash-thinking-exp-01-21 {#google_ai_gemini_2.0_flash_thinking_exp_01_21_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.5-flash-preview-04-17 {#google_ai_gemini_2.5_flash_preview_04_17_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.5-pro-preview-06-05 {#google_ai_gemini_2.5_pro_preview_06_05_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gemini-1.0-pro-vision-latest {#google_ai_gemini_1.0_pro_vision_latest_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-1.5-flash {#google_ai_gemini_1.5_flash_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-1.5-flash-002 {#google_ai_gemini_1.5_flash_002_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-1.5-flash-8b {#google_ai_gemini_1.5_flash_8b_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-1.5-flash-8b-001 {#google_ai_gemini_1.5_flash_8b_001_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-1.5-flash-8b-latest {#google_ai_gemini_1.5_flash_8b_latest_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-1.5-flash-latest {#google_ai_gemini_1.5_flash_latest_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.0-flash {#google_ai_gemini_2.0_flash_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.0-flash-001 {#google_ai_gemini_2.0_flash_001_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.0-flash-exp {#google_ai_gemini_2.0_flash_exp_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.0-flash-exp-image-generation {#google_ai_gemini_2.0_flash_exp_image_generation_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.0-flash-lite {#google_ai_gemini_2.0_flash_lite_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.0-flash-lite-001 {#google_ai_gemini_2.0_flash_lite_001_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.0-flash-lite-preview {#google_ai_gemini_2.0_flash_lite_preview_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.0-flash-lite-preview-02-05 {#google_ai_gemini_2.0_flash_lite_preview_02_05_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.0-flash-preview-image-generation {#google_ai_gemini_2.0_flash_preview_image_generation_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.0-pro-exp {#google_ai_gemini_2.0_pro_exp_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.0-pro-exp-02-05 {#google_ai_gemini_2.0_pro_exp_02_05_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.5-flash-preview-tts {#google_ai_gemini_2.5_flash_preview_tts_details}--### Basic--- ❌ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.5-pro-exp-03-25 {#google_ai_gemini_2.5_pro_exp_03_25_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.5-pro-preview-tts {#google_ai_gemini_2.5_pro_preview_tts_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-exp-1206 {#google_ai_gemini_exp_1206_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-pro-vision {#google_ai_gemini_pro_vision_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--------## letta--| Model | Basic | Token Streaming | Multimodal | Context Window | Last Scanned |-|---------------------------------------------------|:---:|:-------------:|:--------:|:------------:|:----------:|-| [`letta-free`](#letta_letta_free_details) | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |--### Detailed Test Results--#### letta-free {#letta_letta_free_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--------## together--| Model | Basic | Token Streaming | Multimodal | Context Window | Last Scanned |-|---------------------------------------------------|:---:|:-------------:|:--------:|:------------:|:----------:|-| [`Qwen/Qwen2.5-72B-Instruct-Turbo`](#together_qwen_qwen2.5_72b_instruct_turbo_details) | ✅ | ✅ | ⚠️ | 131,072 | 2025-06-20 |-| [`arcee-ai/virtuoso-large`](#together_arcee_ai_virtuoso_large_details) | ⚠️ | ✅ | ✅ | 131,072 | 2025-06-20 |-| [`Qwen/QwQ-32B`](#together_qwen_qwq_32b_details) | ⚠️ | ✅ | ⚠️ | 131,072 | 2025-06-20 |-| [`Qwen/Qwen2.5-7B-Instruct-Turbo`](#together_qwen_qwen2.5_7b_instruct_turbo_details) | ⚠️ | ✅ | ⚠️ | 32,768 | 2025-06-20 |-| [`Qwen/Qwen2.5-Coder-32B-Instruct`](#together_qwen_qwen2.5_coder_32b_instruct_details) | ⚠️ | ✅ | ⚠️ | 16,384 | 2025-06-20 |-| [`arcee-ai/coder-large`](#together_arcee_ai_coder_large_details) | ⚠️ | ✅ | ⚠️ | 32,768 | 2025-06-20 |-| [`arcee_ai/arcee-spotlight`](#together_arcee_ai_arcee_spotlight_details) | ⚠️ | ✅ | ⚠️ | 131,072 | 2025-06-20 |-| [`meta-llama/Llama-3.2-3B-Instruct-Turbo`](#together_meta_llama_llama_3.2_3b_instruct_turbo_details) | ⚠️ | ✅ | ❌ | 131,072 | 2025-06-20 |-| [`meta-llama/Llama-3.3-70B-Instruct-Turbo`](#together_meta_llama_llama_3.3_70b_instruct_turbo_details) | ⚠️ | ✅ | ❌ | 131,072 | 2025-06-20 |-| [`meta-llama/Llama-3.3-70B-Instruct-Turbo-Free`](#together_meta_llama_llama_3.3_70b_instruct_turbo_free_details) | ⚠️ | ✅ | ❌ | 131,072 | 2025-06-20 |-| [`meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo`](#together_meta_llama_meta_llama_3.1_405b_instruct_turbo_details) | ⚠️ | ✅ | ❌ | 130,815 | 2025-06-20 |-| [`meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo`](#together_meta_llama_meta_llama_3.1_70b_instruct_turbo_details) | ⚠️ | ✅ | ❌ | 131,072 | 2025-06-20 |-| [`nvidia/Llama-3.1-Nemotron-70B-Instruct-HF`](#together_nvidia_llama_3.1_nemotron_70b_instruct_hf_details) | ⚠️ | ✅ | ❌ | 32,768 | 2025-06-20 |-| [`arcee-ai/virtuoso-medium-v2`](#together_arcee_ai_virtuoso_medium_v2_details) | ⚠️ | ⚠️ | ✅ | 131,072 | 2025-06-20 |-| [`meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8`](#together_meta_llama_llama_4_maverick_17b_128e_instruct_fp8_details) | ⚠️ | ❌ | ✅ | 1,048,576 | 2025-06-20 |-| [`Qwen/Qwen3-235B-A22B-fp8-tput`](#together_qwen_qwen3_235b_a22b_fp8_tput_details) | ⚠️ | ⚠️ | ❌ | 40,960 | 2025-06-20 |-| [`deepseek-ai/DeepSeek-V3`](#together_deepseek_ai_deepseek_v3_details) | ⚠️ | ⚠️ | ❌ | 131,072 | 2025-06-20 |-| [`meta-llama/Llama-4-Scout-17B-16E-Instruct`](#together_meta_llama_llama_4_scout_17b_16e_instruct_details) | ⚠️ | ⚠️ | ❌ | 1,048,576 | 2025-06-20 |-| [`meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo`](#together_meta_llama_meta_llama_3.1_8b_instruct_turbo_details) | ⚠️ | ⚠️ | ❌ | 131,072 | 2025-06-20 |-| [`mistralai/Mixtral-8x7B-Instruct-v0.1`](#together_mistralai_mixtral_8x7b_instruct_v0.1_details) | ⚠️ | ⚠️ | ❌ | 32,768 | 2025-06-20 |-| [`arcee-ai/caller`](#together_arcee_ai_caller_details) | ❌ | ⚠️ | ❌ | 32,768 | 2025-06-20 |-| [`mistralai/Mistral-Small-24B-Instruct-2501`](#together_mistralai_mistral_small_24b_instruct_2501_details) | ❌ | ⚠️ | ❌ | 32,768 | 2025-06-20 |-| [`NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO`](#together_nousresearch_nous_hermes_2_mixtral_8x7b_dpo_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`Qwen/Qwen2-72B-Instruct`](#together_qwen_qwen2_72b_instruct_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`Qwen/Qwen2-VL-72B-Instruct`](#together_qwen_qwen2_vl_72b_instruct_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`Qwen/Qwen2.5-VL-72B-Instruct`](#together_qwen_qwen2.5_vl_72b_instruct_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`arcee-ai/arcee-blitz`](#together_arcee_ai_arcee_blitz_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`arcee-ai/maestro-reasoning`](#together_arcee_ai_maestro_reasoning_details) | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |-| [`deepseek-ai/DeepSeek-R1`](#together_deepseek_ai_deepseek_r1_details) | ❌ | ❌ | ❌ | 163,840 | 2025-06-20 |-| [`deepseek-ai/DeepSeek-R1-Distill-Llama-70B`](#together_deepseek_ai_deepseek_r1_distill_llama_70b_details) | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |-| [`deepseek-ai/DeepSeek-R1-Distill-Llama-70B-free`](#together_deepseek_ai_deepseek_r1_distill_llama_70b_free_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B`](#together_deepseek_ai_deepseek_r1_distill_qwen_1.5b_details) | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |-| [`deepseek-ai/DeepSeek-R1-Distill-Qwen-14B`](#together_deepseek_ai_deepseek_r1_distill_qwen_14b_details) | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |-| [`deepseek-ai/DeepSeek-V3-p-dp`](#together_deepseek_ai_deepseek_v3_p_dp_details) | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |-| [`google/gemma-2-27b-it`](#together_google_gemma_2_27b_it_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`lgai/exaone-3-5-32b-instruct`](#together_lgai_exaone_3_5_32b_instruct_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`lgai/exaone-deep-32b`](#together_lgai_exaone_deep_32b_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`marin-community/marin-8b-instruct`](#together_marin_community_marin_8b_instruct_details) | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |-| [`meta-llama/Llama-3-70b-chat-hf`](#together_meta_llama_llama_3_70b_chat_hf_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`meta-llama/Llama-3-8b-chat-hf`](#together_meta_llama_llama_3_8b_chat_hf_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`meta-llama/Llama-3.2-11B-Vision-Instruct-Turbo`](#together_meta_llama_llama_3.2_11b_vision_instruct_turbo_details) | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |-| [`meta-llama/Llama-3.2-90B-Vision-Instruct-Turbo`](#together_meta_llama_llama_3.2_90b_vision_instruct_turbo_details) | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |-| [`meta-llama/Llama-Vision-Free`](#together_meta_llama_llama_vision_free_details) | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |-| [`meta-llama/Meta-Llama-3-70B-Instruct-Turbo`](#together_meta_llama_meta_llama_3_70b_instruct_turbo_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`meta-llama/Meta-Llama-3-8B-Instruct-Lite`](#together_meta_llama_meta_llama_3_8b_instruct_lite_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`mistralai/Mistral-7B-Instruct-v0.1`](#together_mistralai_mistral_7b_instruct_v0.1_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`mistralai/Mistral-7B-Instruct-v0.2`](#together_mistralai_mistral_7b_instruct_v0.2_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`mistralai/Mistral-7B-Instruct-v0.3`](#together_mistralai_mistral_7b_instruct_v0.3_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`perplexity-ai/r1-1776`](#together_perplexity_ai_r1_1776_details) | ❌ | ❌ | ❌ | 163,840 | 2025-06-20 |-| [`scb10x/scb10x-llama3-1-typhoon2-70b-instruct`](#together_scb10x_scb10x_llama3_1_typhoon2_70b_instruct_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`scb10x/scb10x-typhoon-2-1-gemma3-12b`](#together_scb10x_scb10x_typhoon_2_1_gemma3_12b_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`togethercomputer/MoA-1`](#together_togethercomputer_moa_1_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`togethercomputer/MoA-1-Turbo`](#together_togethercomputer_moa_1_turbo_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`togethercomputer/Refuel-Llm-V2`](#together_togethercomputer_refuel_llm_v2_details) | ❌ | ❌ | ❌ | 16,384 | 2025-06-20 |-| [`togethercomputer/Refuel-Llm-V2-Small`](#together_togethercomputer_refuel_llm_v2_small_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |--### Detailed Test Results--#### Qwen/Qwen2.5-72B-Instruct-Turbo {#together_qwen_qwen2.5_72b_instruct_turbo_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ❌ `test_url_image_input`--#### arcee-ai/virtuoso-large {#together_arcee_ai_virtuoso_large_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### Qwen/QwQ-32B {#together_qwen_qwq_32b_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ✅ `test_url_image_input`--#### Qwen/Qwen2.5-7B-Instruct-Turbo {#together_qwen_qwen2.5_7b_instruct_turbo_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ❌ `test_url_image_input`--#### Qwen/Qwen2.5-Coder-32B-Instruct {#together_qwen_qwen2.5_coder_32b_instruct_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ❌ `test_url_image_input`--#### arcee-ai/coder-large {#together_arcee_ai_coder_large_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ✅ `test_url_image_input`--#### arcee_ai/arcee-spotlight {#together_arcee_ai_arcee_spotlight_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/Llama-3.2-3B-Instruct-Turbo {#together_meta_llama_llama_3.2_3b_instruct_turbo_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/Llama-3.3-70B-Instruct-Turbo {#together_meta_llama_llama_3.3_70b_instruct_turbo_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/Llama-3.3-70B-Instruct-Turbo-Free {#together_meta_llama_llama_3.3_70b_instruct_turbo_free_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo {#together_meta_llama_meta_llama_3.1_405b_instruct_turbo_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo {#together_meta_llama_meta_llama_3.1_70b_instruct_turbo_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### nvidia/Llama-3.1-Nemotron-70B-Instruct-HF {#together_nvidia_llama_3.1_nemotron_70b_instruct_hf_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### arcee-ai/virtuoso-medium-v2 {#together_arcee_ai_virtuoso_medium_v2_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8 {#together_meta_llama_llama_4_maverick_17b_128e_instruct_fp8_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### Qwen/Qwen3-235B-A22B-fp8-tput {#together_qwen_qwen3_235b_a22b_fp8_tput_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### deepseek-ai/DeepSeek-V3 {#together_deepseek_ai_deepseek_v3_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/Llama-4-Scout-17B-16E-Instruct {#together_meta_llama_llama_4_scout_17b_16e_instruct_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo {#together_meta_llama_meta_llama_3.1_8b_instruct_turbo_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### mistralai/Mixtral-8x7B-Instruct-v0.1 {#together_mistralai_mixtral_8x7b_instruct_v0.1_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### arcee-ai/caller {#together_arcee_ai_caller_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### mistralai/Mistral-Small-24B-Instruct-2501 {#together_mistralai_mistral_small_24b_instruct_2501_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO {#together_nousresearch_nous_hermes_2_mixtral_8x7b_dpo_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### Qwen/Qwen2-72B-Instruct {#together_qwen_qwen2_72b_instruct_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### Qwen/Qwen2-VL-72B-Instruct {#together_qwen_qwen2_vl_72b_instruct_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### Qwen/Qwen2.5-VL-72B-Instruct {#together_qwen_qwen2.5_vl_72b_instruct_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### arcee-ai/arcee-blitz {#together_arcee_ai_arcee_blitz_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### arcee-ai/maestro-reasoning {#together_arcee_ai_maestro_reasoning_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### deepseek-ai/DeepSeek-R1 {#together_deepseek_ai_deepseek_r1_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### deepseek-ai/DeepSeek-R1-Distill-Llama-70B {#together_deepseek_ai_deepseek_r1_distill_llama_70b_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### deepseek-ai/DeepSeek-R1-Distill-Llama-70B-free {#together_deepseek_ai_deepseek_r1_distill_llama_70b_free_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B {#together_deepseek_ai_deepseek_r1_distill_qwen_1.5b_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### deepseek-ai/DeepSeek-R1-Distill-Qwen-14B {#together_deepseek_ai_deepseek_r1_distill_qwen_14b_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### deepseek-ai/DeepSeek-V3-p-dp {#together_deepseek_ai_deepseek_v3_p_dp_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### google/gemma-2-27b-it {#together_google_gemma_2_27b_it_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### lgai/exaone-3-5-32b-instruct {#together_lgai_exaone_3_5_32b_instruct_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### lgai/exaone-deep-32b {#together_lgai_exaone_deep_32b_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### marin-community/marin-8b-instruct {#together_marin_community_marin_8b_instruct_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/Llama-3-70b-chat-hf {#together_meta_llama_llama_3_70b_chat_hf_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/Llama-3-8b-chat-hf {#together_meta_llama_llama_3_8b_chat_hf_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/Llama-3.2-11B-Vision-Instruct-Turbo {#together_meta_llama_llama_3.2_11b_vision_instruct_turbo_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/Llama-3.2-90B-Vision-Instruct-Turbo {#together_meta_llama_llama_3.2_90b_vision_instruct_turbo_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/Llama-Vision-Free {#together_meta_llama_llama_vision_free_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/Meta-Llama-3-70B-Instruct-Turbo {#together_meta_llama_meta_llama_3_70b_instruct_turbo_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/Meta-Llama-3-8B-Instruct-Lite {#together_meta_llama_meta_llama_3_8b_instruct_lite_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### mistralai/Mistral-7B-Instruct-v0.1 {#together_mistralai_mistral_7b_instruct_v0.1_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### mistralai/Mistral-7B-Instruct-v0.2 {#together_mistralai_mistral_7b_instruct_v0.2_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### mistralai/Mistral-7B-Instruct-v0.3 {#together_mistralai_mistral_7b_instruct_v0.3_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### perplexity-ai/r1-1776 {#together_perplexity_ai_r1_1776_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### scb10x/scb10x-llama3-1-typhoon2-70b-instruct {#together_scb10x_scb10x_llama3_1_typhoon2_70b_instruct_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### scb10x/scb10x-typhoon-2-1-gemma3-12b {#together_scb10x_scb10x_typhoon_2_1_gemma3_12b_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### togethercomputer/MoA-1 {#together_togethercomputer_moa_1_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### togethercomputer/MoA-1-Turbo {#together_togethercomputer_moa_1_turbo_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### togethercomputer/Refuel-Llm-V2 {#together_togethercomputer_refuel_llm_v2_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### togethercomputer/Refuel-Llm-V2-Small {#together_togethercomputer_refuel_llm_v2_small_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--------## deepseek--| Model | Basic | Token Streaming | Multimodal | Context Window | Last Scanned |-|---------------------------------------------------|:---:|:-------------:|:--------:|:------------:|:----------:|-| [`deepseek-chat`](#deepseek_deepseek_chat_details) | ❌ | ❌ | ❌ | 64,000 | 2025-06-20 |-| [`deepseek-reasoner`](#deepseek_deepseek_reasoner_details) | ❌ | ❌ | ❌ | 64,000 | 2025-06-20 |--### Detailed Test Results--#### deepseek-chat {#deepseek_deepseek_chat_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### deepseek-reasoner {#deepseek_deepseek_reasoner_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--------## groq--| Model | Basic | Token Streaming | Multimodal | Context Window | Last Scanned |-|---------------------------------------------------|:---:|:-------------:|:--------:|:------------:|:----------:|-| [`allam-2-7b`](#groq_allam_2_7b_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`compound-beta`](#groq_compound_beta_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`compound-beta-mini`](#groq_compound_beta_mini_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`deepseek-r1-distill-llama-70b`](#groq_deepseek_r1_distill_llama_70b_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`distil-whisper-large-v3-en`](#groq_distil_whisper_large_v3_en_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`gemma2-9b-it`](#groq_gemma2_9b_it_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`llama-3.1-8b-instant`](#groq_llama_3.1_8b_instant_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`llama-3.3-70b-versatile`](#groq_llama_3.3_70b_versatile_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`llama-guard-3-8b`](#groq_llama_guard_3_8b_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`llama3-70b-8192`](#groq_llama3_70b_8192_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`llama3-8b-8192`](#groq_llama3_8b_8192_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`meta-llama/llama-4-maverick-17b-128e-instruct`](#groq_meta_llama_llama_4_maverick_17b_128e_instruct_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`meta-llama/llama-4-scout-17b-16e-instruct`](#groq_meta_llama_llama_4_scout_17b_16e_instruct_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`meta-llama/llama-guard-4-12b`](#groq_meta_llama_llama_guard_4_12b_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`meta-llama/llama-prompt-guard-2-22m`](#groq_meta_llama_llama_prompt_guard_2_22m_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`meta-llama/llama-prompt-guard-2-86m`](#groq_meta_llama_llama_prompt_guard_2_86m_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`mistral-saba-24b`](#groq_mistral_saba_24b_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`playai-tts`](#groq_playai_tts_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`playai-tts-arabic`](#groq_playai_tts_arabic_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`qwen-qwq-32b`](#groq_qwen_qwq_32b_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`qwen/qwen3-32b`](#groq_qwen_qwen3_32b_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`whisper-large-v3`](#groq_whisper_large_v3_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`whisper-large-v3-turbo`](#groq_whisper_large_v3_turbo_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |--### Detailed Test Results--#### allam-2-7b {#groq_allam_2_7b_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### compound-beta {#groq_compound_beta_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### compound-beta-mini {#groq_compound_beta_mini_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### deepseek-r1-distill-llama-70b {#groq_deepseek_r1_distill_llama_70b_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### distil-whisper-large-v3-en {#groq_distil_whisper_large_v3_en_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemma2-9b-it {#groq_gemma2_9b_it_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### llama-3.1-8b-instant {#groq_llama_3.1_8b_instant_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### llama-3.3-70b-versatile {#groq_llama_3.3_70b_versatile_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### llama-guard-3-8b {#groq_llama_guard_3_8b_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### llama3-70b-8192 {#groq_llama3_70b_8192_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### llama3-8b-8192 {#groq_llama3_8b_8192_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/llama-4-maverick-17b-128e-instruct {#groq_meta_llama_llama_4_maverick_17b_128e_instruct_details}--### Basic--- ❌ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/llama-4-scout-17b-16e-instruct {#groq_meta_llama_llama_4_scout_17b_16e_instruct_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/llama-guard-4-12b {#groq_meta_llama_llama_guard_4_12b_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/llama-prompt-guard-2-22m {#groq_meta_llama_llama_prompt_guard_2_22m_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/llama-prompt-guard-2-86m {#groq_meta_llama_llama_prompt_guard_2_86m_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### mistral-saba-24b {#groq_mistral_saba_24b_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### playai-tts {#groq_playai_tts_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### playai-tts-arabic {#groq_playai_tts_arabic_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### qwen-qwq-32b {#groq_qwen_qwq_32b_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### qwen/qwen3-32b {#groq_qwen_qwen3_32b_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### whisper-large-v3 {#groq_whisper_large_v3_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### whisper-large-v3-turbo {#groq_whisper_large_v3_turbo_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`------diff --git a/.github/scripts/model-sweep/test_output_with_links.mdx b/.github/scripts/model-sweep/test_output_with_links.mdxdeleted file mode 100644index 64a14ea7d..000000000--- a/.github/scripts/model-sweep/test_output_with_links.mdx+++ /dev/null@@ -1,4395 +0,0 @@-----title: Support Models-generated: 2025-06-20T16:06:14.319292------# Supported Models--## Overview--Letta routinely runs automated scans against available providers and models. These are the results of the latest scan.--Ran 2464 tests against 154 models across 7 providers on June 20th, 2025---## anthropic--| Model | Basic | Token Streaming | Multimodal | Context Window | Last Scanned |-|---------------------------------------------------|:---:|:-------------:|:--------:|:------------:|:----------:|-| [`claude-3-5-haiku-20241022`](#anthropic_claude_3_5_haiku_20241022_details) | ✅ | ✅ | ✅ | 200,000 | 2025-06-20 |-| [`claude-3-5-sonnet-20241022`](#anthropic_claude_3_5_sonnet_20241022_details) | ✅ | ✅ | ✅ | 200,000 | 2025-06-20 |-| [`claude-3-7-sonnet-20250219`](#anthropic_claude_3_7_sonnet_20250219_details) | ✅ | ✅ | ✅ | 200,000 | 2025-06-20 |-| [`claude-sonnet-4-20250514`](#anthropic_claude_sonnet_4_20250514_details) | ✅ | ✅ | ✅ | 200,000 | 2025-06-20 |-| [`claude-opus-4-20250514`](#anthropic_claude_opus_4_20250514_details) | ✅ | ✅ | ⚠️ | 200,000 | 2025-06-20 |-| [`claude-3-5-sonnet-20240620`](#anthropic_claude_3_5_sonnet_20240620_details) | ⚠️ | ❌ | ✅ | 200,000 | 2025-06-20 |-| [`claude-3-haiku-20240307`](#anthropic_claude_3_haiku_20240307_details) | ⚠️ | ❌ | ✅ | 200,000 | 2025-06-20 |-| [`claude-3-opus-20240229`](#anthropic_claude_3_opus_20240229_details) | ⚠️ | ❌ | ✅ | 200,000 | 2025-06-20 |-| [`claude-3-sonnet-20240229`](#anthropic_claude_3_sonnet_20240229_details) | ❌ | ❌ | ❌ | 200,000 | 2025-06-20 |--### Detailed Test Results--#### claude-3-5-haiku-20241022 {#anthropic_claude_3_5_haiku_20241022_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### claude-3-5-sonnet-20241022 {#anthropic_claude_3_5_sonnet_20241022_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### claude-3-7-sonnet-20250219 {#anthropic_claude_3_7_sonnet_20250219_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### claude-sonnet-4-20250514 {#anthropic_claude_sonnet_4_20250514_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### claude-opus-4-20250514 {#anthropic_claude_opus_4_20250514_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ✅ `test_url_image_input`--#### claude-3-5-sonnet-20240620 {#anthropic_claude_3_5_sonnet_20240620_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### claude-3-haiku-20240307 {#anthropic_claude_3_haiku_20240307_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### claude-3-opus-20240229 {#anthropic_claude_3_opus_20240229_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### claude-3-sonnet-20240229 {#anthropic_claude_3_sonnet_20240229_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--------## openai--| Model | Basic | Token Streaming | Multimodal | Context Window | Last Scanned |-|---------------------------------------------------|:---:|:-------------:|:--------:|:------------:|:----------:|-| [`gpt-4.1`](#openai_gpt_4.1_details) | ✅ | ✅ | ✅ | 1,047,576 | 2025-06-20 |-| [`gpt-4.1-2025-04-14`](#openai_gpt_4.1_2025_04_14_details) | ✅ | ✅ | ✅ | 1,047,576 | 2025-06-20 |-| [`gpt-4.1-nano-2025-04-14`](#openai_gpt_4.1_nano_2025_04_14_details) | ✅ | ✅ | ✅ | 1,047,576 | 2025-06-20 |-| [`gpt-4o`](#openai_gpt_4o_details) | ✅ | ✅ | ✅ | 128,000 | 2025-06-20 |-| [`gpt-4o-2024-05-13`](#openai_gpt_4o_2024_05_13_details) | ✅ | ✅ | ✅ | 128,000 | 2025-06-20 |-| [`gpt-4-turbo`](#openai_gpt_4_turbo_details) | ✅ | ✅ | ⚠️ | 8,192 | 2025-06-20 |-| [`gpt-4.1-mini`](#openai_gpt_4.1_mini_details) | ✅ | ✅ | ⚠️ | 1,047,576 | 2025-06-20 |-| [`gpt-4.5-preview`](#openai_gpt_4.5_preview_details) | ✅ | ✅ | ⚠️ | 128,000 | 2025-06-20 |-| [`gpt-4.5-preview-2025-02-27`](#openai_gpt_4.5_preview_2025_02_27_details) | ✅ | ✅ | ⚠️ | 128,000 | 2025-06-20 |-| [`gpt-4o-2024-08-06`](#openai_gpt_4o_2024_08_06_details) | ✅ | ✅ | ⚠️ | 128,000 | 2025-06-20 |-| [`gpt-4-0613`](#openai_gpt_4_0613_details) | ✅ | ✅ | ❌ | 8,192 | 2025-06-20 |-| [`gpt-4-1106-preview`](#openai_gpt_4_1106_preview_details) | ✅ | ✅ | ❌ | 128,000 | 2025-06-20 |-| [`gpt-4-turbo-2024-04-09`](#openai_gpt_4_turbo_2024_04_09_details) | ✅ | ⚠️ | ✅ | 128,000 | 2025-06-20 |-| [`gpt-4.1-mini-2025-04-14`](#openai_gpt_4.1_mini_2025_04_14_details) | ⚠️ | ✅ | ✅ | 1,047,576 | 2025-06-20 |-| [`gpt-4.1-nano`](#openai_gpt_4.1_nano_details) | ⚠️ | ✅ | ✅ | 1,047,576 | 2025-06-20 |-| [`gpt-4o-2024-11-20`](#openai_gpt_4o_2024_11_20_details) | ⚠️ | ✅ | ✅ | 8,192 | 2025-06-20 |-| [`gpt-4-turbo-preview`](#openai_gpt_4_turbo_preview_details) | ✅ | ⚠️ | ❌ | 128,000 | 2025-06-20 |-| [`gpt-4-0125-preview`](#openai_gpt_4_0125_preview_details) | ⚠️ | ✅ | ❌ | 128,000 | 2025-06-20 |-| [`gpt-4o-mini`](#openai_gpt_4o_mini_details) | ⚠️ | ⚠️ | ⚠️ | 128,000 | 2025-06-20 |-| [`gpt-4o-mini-2024-07-18`](#openai_gpt_4o_mini_2024_07_18_details) | ⚠️ | ⚠️ | ❌ | 128,000 | 2025-06-20 |-| [`gpt-4`](#openai_gpt_4_details) | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`o1`](#openai_o1_details) | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`o1-2024-12-17`](#openai_o1_2024_12_17_details) | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`o3`](#openai_o3_details) | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`o3-2025-04-16`](#openai_o3_2025_04_16_details) | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`o3-mini`](#openai_o3_mini_details) | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`o3-mini-2025-01-31`](#openai_o3_mini_2025_01_31_details) | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`o3-pro`](#openai_o3_pro_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`o3-pro-2025-06-10`](#openai_o3_pro_2025_06_10_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |--### Detailed Test Results--#### gpt-4.1 {#openai_gpt_4.1_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gpt-4.1-2025-04-14 {#openai_gpt_4.1_2025_04_14_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gpt-4.1-nano-2025-04-14 {#openai_gpt_4.1_nano_2025_04_14_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gpt-4o {#openai_gpt_4o_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gpt-4o-2024-05-13 {#openai_gpt_4o_2024_05_13_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gpt-4-turbo {#openai_gpt_4_turbo_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gpt-4.1-mini {#openai_gpt_4.1_mini_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gpt-4.5-preview {#openai_gpt_4.5_preview_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gpt-4.5-preview-2025-02-27 {#openai_gpt_4.5_preview_2025_02_27_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gpt-4o-2024-08-06 {#openai_gpt_4o_2024_08_06_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gpt-4-0613 {#openai_gpt_4_0613_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gpt-4-1106-preview {#openai_gpt_4_1106_preview_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gpt-4-turbo-2024-04-09 {#openai_gpt_4_turbo_2024_04_09_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gpt-4.1-mini-2025-04-14 {#openai_gpt_4.1_mini_2025_04_14_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gpt-4.1-nano {#openai_gpt_4.1_nano_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gpt-4o-2024-11-20 {#openai_gpt_4o_2024_11_20_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gpt-4-turbo-preview {#openai_gpt_4_turbo_preview_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gpt-4-0125-preview {#openai_gpt_4_0125_preview_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gpt-4o-mini {#openai_gpt_4o_mini_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gpt-4o-mini-2024-07-18 {#openai_gpt_4o_mini_2024_07_18_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gpt-4 {#openai_gpt_4_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### o1 {#openai_o1_details}--### Basic--- ❌ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### o1-2024-12-17 {#openai_o1_2024_12_17_details}--### Basic--- ❌ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### o3 {#openai_o3_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### o3-2025-04-16 {#openai_o3_2025_04_16_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### o3-mini {#openai_o3_mini_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### o3-mini-2025-01-31 {#openai_o3_mini_2025_01_31_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### o3-pro {#openai_o3_pro_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### o3-pro-2025-06-10 {#openai_o3_pro_2025_06_10_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--------## google_ai--| Model | Basic | Token Streaming | Multimodal | Context Window | Last Scanned |-|---------------------------------------------------|:---:|:-------------:|:--------:|:------------:|:----------:|-| [`gemini-1.5-pro`](#google_ai_gemini_1.5_pro_details) | ✅ | ✅ | ✅ | 2,000,000 | 2025-06-20 |-| [`gemini-1.5-pro-002`](#google_ai_gemini_1.5_pro_002_details) | ✅ | ✅ | ✅ | 2,000,000 | 2025-06-20 |-| [`gemini-1.5-pro-latest`](#google_ai_gemini_1.5_pro_latest_details) | ✅ | ✅ | ✅ | 2,000,000 | 2025-06-20 |-| [`gemini-2.5-flash-preview-04-17-thinking`](#google_ai_gemini_2.5_flash_preview_04_17_thinking_details) | ✅ | ✅ | ✅ | 1,048,576 | 2025-06-20 |-| [`gemini-2.5-pro-preview-03-25`](#google_ai_gemini_2.5_pro_preview_03_25_details) | ✅ | ✅ | ✅ | 1,048,576 | 2025-06-20 |-| [`gemini-2.5-pro-preview-05-06`](#google_ai_gemini_2.5_pro_preview_05_06_details) | ✅ | ✅ | ✅ | 1,048,576 | 2025-06-20 |-| [`gemini-2.5-flash-preview-05-20`](#google_ai_gemini_2.5_flash_preview_05_20_details) | ✅ | ⚠️ | ✅ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-thinking-exp`](#google_ai_gemini_2.0_flash_thinking_exp_details) | ⚠️ | ✅ | ✅ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-thinking-exp-1219`](#google_ai_gemini_2.0_flash_thinking_exp_1219_details) | ⚠️ | ✅ | ✅ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-thinking-exp-01-21`](#google_ai_gemini_2.0_flash_thinking_exp_01_21_details) | ⚠️ | ✅ | ⚠️ | 1,048,576 | 2025-06-20 |-| [`gemini-2.5-flash-preview-04-17`](#google_ai_gemini_2.5_flash_preview_04_17_details) | ⚠️ | ✅ | ⚠️ | 1,048,576 | 2025-06-20 |-| [`gemini-2.5-pro-preview-06-05`](#google_ai_gemini_2.5_pro_preview_06_05_details) | ⚠️ | ✅ | ⚠️ | 1,048,576 | 2025-06-20 |-| [`gemini-1.0-pro-vision-latest`](#google_ai_gemini_1.0_pro_vision_latest_details) | ❌ | ❌ | ❌ | 12,288 | 2025-06-20 |-| [`gemini-1.5-flash`](#google_ai_gemini_1.5_flash_details) | ❌ | ❌ | ❌ | 1,000,000 | 2025-06-20 |-| [`gemini-1.5-flash-002`](#google_ai_gemini_1.5_flash_002_details) | ❌ | ❌ | ❌ | 1,000,000 | 2025-06-20 |-| [`gemini-1.5-flash-8b`](#google_ai_gemini_1.5_flash_8b_details) | ❌ | ❌ | ❌ | 1,000,000 | 2025-06-20 |-| [`gemini-1.5-flash-8b-001`](#google_ai_gemini_1.5_flash_8b_001_details) | ❌ | ❌ | ❌ | 1,000,000 | 2025-06-20 |-| [`gemini-1.5-flash-8b-latest`](#google_ai_gemini_1.5_flash_8b_latest_details) | ❌ | ❌ | ❌ | 1,000,000 | 2025-06-20 |-| [`gemini-1.5-flash-latest`](#google_ai_gemini_1.5_flash_latest_details) | ❌ | ❌ | ❌ | 1,000,000 | 2025-06-20 |-| [`gemini-2.0-flash`](#google_ai_gemini_2.0_flash_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-001`](#google_ai_gemini_2.0_flash_001_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-exp`](#google_ai_gemini_2.0_flash_exp_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-exp-image-generation`](#google_ai_gemini_2.0_flash_exp_image_generation_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-lite`](#google_ai_gemini_2.0_flash_lite_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-lite-001`](#google_ai_gemini_2.0_flash_lite_001_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-lite-preview`](#google_ai_gemini_2.0_flash_lite_preview_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-lite-preview-02-05`](#google_ai_gemini_2.0_flash_lite_preview_02_05_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-flash-preview-image-generation`](#google_ai_gemini_2.0_flash_preview_image_generation_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`gemini-2.0-pro-exp`](#google_ai_gemini_2.0_pro_exp_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.0-pro-exp-02-05`](#google_ai_gemini_2.0_pro_exp_02_05_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.5-flash-preview-tts`](#google_ai_gemini_2.5_flash_preview_tts_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`gemini-2.5-pro-exp-03-25`](#google_ai_gemini_2.5_pro_exp_03_25_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-2.5-pro-preview-tts`](#google_ai_gemini_2.5_pro_preview_tts_details) | ❌ | ❌ | ❌ | 65,536 | 2025-06-20 |-| [`gemini-exp-1206`](#google_ai_gemini_exp_1206_details) | ❌ | ❌ | ❌ | 1,048,576 | 2025-06-20 |-| [`gemini-pro-vision`](#google_ai_gemini_pro_vision_details) | ❌ | ❌ | ❌ | 12,288 | 2025-06-20 |--### Detailed Test Results--#### gemini-1.5-pro {#google_ai_gemini_1.5_pro_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gemini-1.5-pro-002 {#google_ai_gemini_1.5_pro_002_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gemini-1.5-pro-latest {#google_ai_gemini_1.5_pro_latest_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gemini-2.5-flash-preview-04-17-thinking {#google_ai_gemini_2.5_flash_preview_04_17_thinking_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gemini-2.5-pro-preview-03-25 {#google_ai_gemini_2.5_pro_preview_03_25_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gemini-2.5-pro-preview-05-06 {#google_ai_gemini_2.5_pro_preview_05_06_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gemini-2.5-flash-preview-05-20 {#google_ai_gemini_2.5_flash_preview_05_20_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gemini-2.0-flash-thinking-exp {#google_ai_gemini_2.0_flash_thinking_exp_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gemini-2.0-flash-thinking-exp-1219 {#google_ai_gemini_2.0_flash_thinking_exp_1219_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gemini-2.0-flash-thinking-exp-01-21 {#google_ai_gemini_2.0_flash_thinking_exp_01_21_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.5-flash-preview-04-17 {#google_ai_gemini_2.5_flash_preview_04_17_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.5-pro-preview-06-05 {#google_ai_gemini_2.5_pro_preview_06_05_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ✅ `test_url_image_input`--#### gemini-1.0-pro-vision-latest {#google_ai_gemini_1.0_pro_vision_latest_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-1.5-flash {#google_ai_gemini_1.5_flash_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-1.5-flash-002 {#google_ai_gemini_1.5_flash_002_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-1.5-flash-8b {#google_ai_gemini_1.5_flash_8b_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-1.5-flash-8b-001 {#google_ai_gemini_1.5_flash_8b_001_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-1.5-flash-8b-latest {#google_ai_gemini_1.5_flash_8b_latest_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-1.5-flash-latest {#google_ai_gemini_1.5_flash_latest_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.0-flash {#google_ai_gemini_2.0_flash_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.0-flash-001 {#google_ai_gemini_2.0_flash_001_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.0-flash-exp {#google_ai_gemini_2.0_flash_exp_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.0-flash-exp-image-generation {#google_ai_gemini_2.0_flash_exp_image_generation_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.0-flash-lite {#google_ai_gemini_2.0_flash_lite_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.0-flash-lite-001 {#google_ai_gemini_2.0_flash_lite_001_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.0-flash-lite-preview {#google_ai_gemini_2.0_flash_lite_preview_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.0-flash-lite-preview-02-05 {#google_ai_gemini_2.0_flash_lite_preview_02_05_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.0-flash-preview-image-generation {#google_ai_gemini_2.0_flash_preview_image_generation_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.0-pro-exp {#google_ai_gemini_2.0_pro_exp_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.0-pro-exp-02-05 {#google_ai_gemini_2.0_pro_exp_02_05_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.5-flash-preview-tts {#google_ai_gemini_2.5_flash_preview_tts_details}--### Basic--- ❌ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.5-pro-exp-03-25 {#google_ai_gemini_2.5_pro_exp_03_25_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-2.5-pro-preview-tts {#google_ai_gemini_2.5_pro_preview_tts_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-exp-1206 {#google_ai_gemini_exp_1206_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemini-pro-vision {#google_ai_gemini_pro_vision_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--------## letta--| Model | Basic | Token Streaming | Multimodal | Context Window | Last Scanned |-|---------------------------------------------------|:---:|:-------------:|:--------:|:------------:|:----------:|-| [`letta-free`](#letta_letta_free_details) | ⚠️ | ❌ | ❌ | 8,192 | 2025-06-20 |--### Detailed Test Results--#### letta-free {#letta_letta_free_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--------## together--| Model | Basic | Token Streaming | Multimodal | Context Window | Last Scanned |-|---------------------------------------------------|:---:|:-------------:|:--------:|:------------:|:----------:|-| [`Qwen/Qwen2.5-72B-Instruct-Turbo`](#together_qwen_qwen2.5_72b_instruct_turbo_details) | ✅ | ✅ | ⚠️ | 131,072 | 2025-06-20 |-| [`arcee-ai/virtuoso-large`](#together_arcee_ai_virtuoso_large_details) | ⚠️ | ✅ | ✅ | 131,072 | 2025-06-20 |-| [`Qwen/QwQ-32B`](#together_qwen_qwq_32b_details) | ⚠️ | ✅ | ⚠️ | 131,072 | 2025-06-20 |-| [`Qwen/Qwen2.5-7B-Instruct-Turbo`](#together_qwen_qwen2.5_7b_instruct_turbo_details) | ⚠️ | ✅ | ⚠️ | 32,768 | 2025-06-20 |-| [`Qwen/Qwen2.5-Coder-32B-Instruct`](#together_qwen_qwen2.5_coder_32b_instruct_details) | ⚠️ | ✅ | ⚠️ | 16,384 | 2025-06-20 |-| [`arcee-ai/coder-large`](#together_arcee_ai_coder_large_details) | ⚠️ | ✅ | ⚠️ | 32,768 | 2025-06-20 |-| [`arcee_ai/arcee-spotlight`](#together_arcee_ai_arcee_spotlight_details) | ⚠️ | ✅ | ⚠️ | 131,072 | 2025-06-20 |-| [`meta-llama/Llama-3.2-3B-Instruct-Turbo`](#together_meta_llama_llama_3.2_3b_instruct_turbo_details) | ⚠️ | ✅ | ❌ | 131,072 | 2025-06-20 |-| [`meta-llama/Llama-3.3-70B-Instruct-Turbo`](#together_meta_llama_llama_3.3_70b_instruct_turbo_details) | ⚠️ | ✅ | ❌ | 131,072 | 2025-06-20 |-| [`meta-llama/Llama-3.3-70B-Instruct-Turbo-Free`](#together_meta_llama_llama_3.3_70b_instruct_turbo_free_details) | ⚠️ | ✅ | ❌ | 131,072 | 2025-06-20 |-| [`meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo`](#together_meta_llama_meta_llama_3.1_405b_instruct_turbo_details) | ⚠️ | ✅ | ❌ | 130,815 | 2025-06-20 |-| [`meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo`](#together_meta_llama_meta_llama_3.1_70b_instruct_turbo_details) | ⚠️ | ✅ | ❌ | 131,072 | 2025-06-20 |-| [`nvidia/Llama-3.1-Nemotron-70B-Instruct-HF`](#together_nvidia_llama_3.1_nemotron_70b_instruct_hf_details) | ⚠️ | ✅ | ❌ | 32,768 | 2025-06-20 |-| [`arcee-ai/virtuoso-medium-v2`](#together_arcee_ai_virtuoso_medium_v2_details) | ⚠️ | ⚠️ | ✅ | 131,072 | 2025-06-20 |-| [`meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8`](#together_meta_llama_llama_4_maverick_17b_128e_instruct_fp8_details) | ⚠️ | ❌ | ✅ | 1,048,576 | 2025-06-20 |-| [`Qwen/Qwen3-235B-A22B-fp8-tput`](#together_qwen_qwen3_235b_a22b_fp8_tput_details) | ⚠️ | ⚠️ | ❌ | 40,960 | 2025-06-20 |-| [`deepseek-ai/DeepSeek-V3`](#together_deepseek_ai_deepseek_v3_details) | ⚠️ | ⚠️ | ❌ | 131,072 | 2025-06-20 |-| [`meta-llama/Llama-4-Scout-17B-16E-Instruct`](#together_meta_llama_llama_4_scout_17b_16e_instruct_details) | ⚠️ | ⚠️ | ❌ | 1,048,576 | 2025-06-20 |-| [`meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo`](#together_meta_llama_meta_llama_3.1_8b_instruct_turbo_details) | ⚠️ | ⚠️ | ❌ | 131,072 | 2025-06-20 |-| [`mistralai/Mixtral-8x7B-Instruct-v0.1`](#together_mistralai_mixtral_8x7b_instruct_v0.1_details) | ⚠️ | ⚠️ | ❌ | 32,768 | 2025-06-20 |-| [`arcee-ai/caller`](#together_arcee_ai_caller_details) | ❌ | ⚠️ | ❌ | 32,768 | 2025-06-20 |-| [`mistralai/Mistral-Small-24B-Instruct-2501`](#together_mistralai_mistral_small_24b_instruct_2501_details) | ❌ | ⚠️ | ❌ | 32,768 | 2025-06-20 |-| [`NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO`](#together_nousresearch_nous_hermes_2_mixtral_8x7b_dpo_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`Qwen/Qwen2-72B-Instruct`](#together_qwen_qwen2_72b_instruct_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`Qwen/Qwen2-VL-72B-Instruct`](#together_qwen_qwen2_vl_72b_instruct_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`Qwen/Qwen2.5-VL-72B-Instruct`](#together_qwen_qwen2.5_vl_72b_instruct_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`arcee-ai/arcee-blitz`](#together_arcee_ai_arcee_blitz_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`arcee-ai/maestro-reasoning`](#together_arcee_ai_maestro_reasoning_details) | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |-| [`deepseek-ai/DeepSeek-R1`](#together_deepseek_ai_deepseek_r1_details) | ❌ | ❌ | ❌ | 163,840 | 2025-06-20 |-| [`deepseek-ai/DeepSeek-R1-Distill-Llama-70B`](#together_deepseek_ai_deepseek_r1_distill_llama_70b_details) | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |-| [`deepseek-ai/DeepSeek-R1-Distill-Llama-70B-free`](#together_deepseek_ai_deepseek_r1_distill_llama_70b_free_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B`](#together_deepseek_ai_deepseek_r1_distill_qwen_1.5b_details) | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |-| [`deepseek-ai/DeepSeek-R1-Distill-Qwen-14B`](#together_deepseek_ai_deepseek_r1_distill_qwen_14b_details) | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |-| [`deepseek-ai/DeepSeek-V3-p-dp`](#together_deepseek_ai_deepseek_v3_p_dp_details) | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |-| [`google/gemma-2-27b-it`](#together_google_gemma_2_27b_it_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`lgai/exaone-3-5-32b-instruct`](#together_lgai_exaone_3_5_32b_instruct_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`lgai/exaone-deep-32b`](#together_lgai_exaone_deep_32b_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`marin-community/marin-8b-instruct`](#together_marin_community_marin_8b_instruct_details) | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |-| [`meta-llama/Llama-3-70b-chat-hf`](#together_meta_llama_llama_3_70b_chat_hf_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`meta-llama/Llama-3-8b-chat-hf`](#together_meta_llama_llama_3_8b_chat_hf_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`meta-llama/Llama-3.2-11B-Vision-Instruct-Turbo`](#together_meta_llama_llama_3.2_11b_vision_instruct_turbo_details) | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |-| [`meta-llama/Llama-3.2-90B-Vision-Instruct-Turbo`](#together_meta_llama_llama_3.2_90b_vision_instruct_turbo_details) | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |-| [`meta-llama/Llama-Vision-Free`](#together_meta_llama_llama_vision_free_details) | ❌ | ❌ | ❌ | 131,072 | 2025-06-20 |-| [`meta-llama/Meta-Llama-3-70B-Instruct-Turbo`](#together_meta_llama_meta_llama_3_70b_instruct_turbo_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`meta-llama/Meta-Llama-3-8B-Instruct-Lite`](#together_meta_llama_meta_llama_3_8b_instruct_lite_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`mistralai/Mistral-7B-Instruct-v0.1`](#together_mistralai_mistral_7b_instruct_v0.1_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`mistralai/Mistral-7B-Instruct-v0.2`](#together_mistralai_mistral_7b_instruct_v0.2_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`mistralai/Mistral-7B-Instruct-v0.3`](#together_mistralai_mistral_7b_instruct_v0.3_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`perplexity-ai/r1-1776`](#together_perplexity_ai_r1_1776_details) | ❌ | ❌ | ❌ | 163,840 | 2025-06-20 |-| [`scb10x/scb10x-llama3-1-typhoon2-70b-instruct`](#together_scb10x_scb10x_llama3_1_typhoon2_70b_instruct_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`scb10x/scb10x-typhoon-2-1-gemma3-12b`](#together_scb10x_scb10x_typhoon_2_1_gemma3_12b_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`togethercomputer/MoA-1`](#together_togethercomputer_moa_1_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`togethercomputer/MoA-1-Turbo`](#together_togethercomputer_moa_1_turbo_details) | ❌ | ❌ | ❌ | 32,768 | 2025-06-20 |-| [`togethercomputer/Refuel-Llm-V2`](#together_togethercomputer_refuel_llm_v2_details) | ❌ | ❌ | ❌ | 16,384 | 2025-06-20 |-| [`togethercomputer/Refuel-Llm-V2-Small`](#together_togethercomputer_refuel_llm_v2_small_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |--### Detailed Test Results--#### Qwen/Qwen2.5-72B-Instruct-Turbo {#together_qwen_qwen2.5_72b_instruct_turbo_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ❌ `test_url_image_input`--#### arcee-ai/virtuoso-large {#together_arcee_ai_virtuoso_large_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### Qwen/QwQ-32B {#together_qwen_qwq_32b_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ✅ `test_url_image_input`--#### Qwen/Qwen2.5-7B-Instruct-Turbo {#together_qwen_qwen2.5_7b_instruct_turbo_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ❌ `test_url_image_input`--#### Qwen/Qwen2.5-Coder-32B-Instruct {#together_qwen_qwen2.5_coder_32b_instruct_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ❌ `test_url_image_input`--#### arcee-ai/coder-large {#together_arcee_ai_coder_large_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ✅ `test_url_image_input`--#### arcee_ai/arcee-spotlight {#together_arcee_ai_arcee_spotlight_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/Llama-3.2-3B-Instruct-Turbo {#together_meta_llama_llama_3.2_3b_instruct_turbo_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/Llama-3.3-70B-Instruct-Turbo {#together_meta_llama_llama_3.3_70b_instruct_turbo_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/Llama-3.3-70B-Instruct-Turbo-Free {#together_meta_llama_llama_3.3_70b_instruct_turbo_free_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo {#together_meta_llama_meta_llama_3.1_405b_instruct_turbo_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo {#together_meta_llama_meta_llama_3.1_70b_instruct_turbo_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### nvidia/Llama-3.1-Nemotron-70B-Instruct-HF {#together_nvidia_llama_3.1_nemotron_70b_instruct_hf_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ✅ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### arcee-ai/virtuoso-medium-v2 {#together_arcee_ai_virtuoso_medium_v2_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8 {#together_meta_llama_llama_4_maverick_17b_128e_instruct_fp8_details}--### Basic--- ✅ `test_agent_loop_error`-- ✅ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ✅ `test_base64_image_input`-- ✅ `test_url_image_input`--#### Qwen/Qwen3-235B-A22B-fp8-tput {#together_qwen_qwen3_235b_a22b_fp8_tput_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### deepseek-ai/DeepSeek-V3 {#together_deepseek_ai_deepseek_v3_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/Llama-4-Scout-17B-16E-Instruct {#together_meta_llama_llama_4_scout_17b_16e_instruct_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo {#together_meta_llama_meta_llama_3.1_8b_instruct_turbo_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ✅ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### mistralai/Mixtral-8x7B-Instruct-v0.1 {#together_mistralai_mixtral_8x7b_instruct_v0.1_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ✅ `test_auto_summarize`-- ✅ `test_greeting_with_assistant_message`-- ✅ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ✅ `test_step_streaming_greeting_with_assistant_message`-- ✅ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ✅ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### arcee-ai/caller {#together_arcee_ai_caller_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ✅ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### mistralai/Mistral-Small-24B-Instruct-2501 {#together_mistralai_mistral_small_24b_instruct_2501_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ✅ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO {#together_nousresearch_nous_hermes_2_mixtral_8x7b_dpo_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### Qwen/Qwen2-72B-Instruct {#together_qwen_qwen2_72b_instruct_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### Qwen/Qwen2-VL-72B-Instruct {#together_qwen_qwen2_vl_72b_instruct_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### Qwen/Qwen2.5-VL-72B-Instruct {#together_qwen_qwen2.5_vl_72b_instruct_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### arcee-ai/arcee-blitz {#together_arcee_ai_arcee_blitz_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### arcee-ai/maestro-reasoning {#together_arcee_ai_maestro_reasoning_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### deepseek-ai/DeepSeek-R1 {#together_deepseek_ai_deepseek_r1_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### deepseek-ai/DeepSeek-R1-Distill-Llama-70B {#together_deepseek_ai_deepseek_r1_distill_llama_70b_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### deepseek-ai/DeepSeek-R1-Distill-Llama-70B-free {#together_deepseek_ai_deepseek_r1_distill_llama_70b_free_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B {#together_deepseek_ai_deepseek_r1_distill_qwen_1.5b_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### deepseek-ai/DeepSeek-R1-Distill-Qwen-14B {#together_deepseek_ai_deepseek_r1_distill_qwen_14b_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### deepseek-ai/DeepSeek-V3-p-dp {#together_deepseek_ai_deepseek_v3_p_dp_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### google/gemma-2-27b-it {#together_google_gemma_2_27b_it_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### lgai/exaone-3-5-32b-instruct {#together_lgai_exaone_3_5_32b_instruct_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### lgai/exaone-deep-32b {#together_lgai_exaone_deep_32b_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### marin-community/marin-8b-instruct {#together_marin_community_marin_8b_instruct_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/Llama-3-70b-chat-hf {#together_meta_llama_llama_3_70b_chat_hf_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/Llama-3-8b-chat-hf {#together_meta_llama_llama_3_8b_chat_hf_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/Llama-3.2-11B-Vision-Instruct-Turbo {#together_meta_llama_llama_3.2_11b_vision_instruct_turbo_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/Llama-3.2-90B-Vision-Instruct-Turbo {#together_meta_llama_llama_3.2_90b_vision_instruct_turbo_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/Llama-Vision-Free {#together_meta_llama_llama_vision_free_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/Meta-Llama-3-70B-Instruct-Turbo {#together_meta_llama_meta_llama_3_70b_instruct_turbo_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/Meta-Llama-3-8B-Instruct-Lite {#together_meta_llama_meta_llama_3_8b_instruct_lite_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### mistralai/Mistral-7B-Instruct-v0.1 {#together_mistralai_mistral_7b_instruct_v0.1_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### mistralai/Mistral-7B-Instruct-v0.2 {#together_mistralai_mistral_7b_instruct_v0.2_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### mistralai/Mistral-7B-Instruct-v0.3 {#together_mistralai_mistral_7b_instruct_v0.3_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### perplexity-ai/r1-1776 {#together_perplexity_ai_r1_1776_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### scb10x/scb10x-llama3-1-typhoon2-70b-instruct {#together_scb10x_scb10x_llama3_1_typhoon2_70b_instruct_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### scb10x/scb10x-typhoon-2-1-gemma3-12b {#together_scb10x_scb10x_typhoon_2_1_gemma3_12b_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### togethercomputer/MoA-1 {#together_togethercomputer_moa_1_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### togethercomputer/MoA-1-Turbo {#together_togethercomputer_moa_1_turbo_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### togethercomputer/Refuel-Llm-V2 {#together_togethercomputer_refuel_llm_v2_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### togethercomputer/Refuel-Llm-V2-Small {#together_togethercomputer_refuel_llm_v2_small_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--------## deepseek--| Model | Basic | Token Streaming | Multimodal | Context Window | Last Scanned |-|---------------------------------------------------|:---:|:-------------:|:--------:|:------------:|:----------:|-| [`deepseek-chat`](#deepseek_deepseek_chat_details) | ❌ | ❌ | ❌ | 64,000 | 2025-06-20 |-| [`deepseek-reasoner`](#deepseek_deepseek_reasoner_details) | ❌ | ❌ | ❌ | 64,000 | 2025-06-20 |--### Detailed Test Results--#### deepseek-chat {#deepseek_deepseek_chat_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### deepseek-reasoner {#deepseek_deepseek_reasoner_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--------## groq--| Model | Basic | Token Streaming | Multimodal | Context Window | Last Scanned |-|---------------------------------------------------|:---:|:-------------:|:--------:|:------------:|:----------:|-| [`allam-2-7b`](#groq_allam_2_7b_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`compound-beta`](#groq_compound_beta_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`compound-beta-mini`](#groq_compound_beta_mini_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`deepseek-r1-distill-llama-70b`](#groq_deepseek_r1_distill_llama_70b_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`distil-whisper-large-v3-en`](#groq_distil_whisper_large_v3_en_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`gemma2-9b-it`](#groq_gemma2_9b_it_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`llama-3.1-8b-instant`](#groq_llama_3.1_8b_instant_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`llama-3.3-70b-versatile`](#groq_llama_3.3_70b_versatile_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`llama-guard-3-8b`](#groq_llama_guard_3_8b_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`llama3-70b-8192`](#groq_llama3_70b_8192_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`llama3-8b-8192`](#groq_llama3_8b_8192_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`meta-llama/llama-4-maverick-17b-128e-instruct`](#groq_meta_llama_llama_4_maverick_17b_128e_instruct_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`meta-llama/llama-4-scout-17b-16e-instruct`](#groq_meta_llama_llama_4_scout_17b_16e_instruct_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`meta-llama/llama-guard-4-12b`](#groq_meta_llama_llama_guard_4_12b_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`meta-llama/llama-prompt-guard-2-22m`](#groq_meta_llama_llama_prompt_guard_2_22m_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`meta-llama/llama-prompt-guard-2-86m`](#groq_meta_llama_llama_prompt_guard_2_86m_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`mistral-saba-24b`](#groq_mistral_saba_24b_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`playai-tts`](#groq_playai_tts_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`playai-tts-arabic`](#groq_playai_tts_arabic_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`qwen-qwq-32b`](#groq_qwen_qwq_32b_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`qwen/qwen3-32b`](#groq_qwen_qwen3_32b_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`whisper-large-v3`](#groq_whisper_large_v3_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |-| [`whisper-large-v3-turbo`](#groq_whisper_large_v3_turbo_details) | ❌ | ❌ | ❌ | 8,192 | 2025-06-20 |--### Detailed Test Results--#### allam-2-7b {#groq_allam_2_7b_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### compound-beta {#groq_compound_beta_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### compound-beta-mini {#groq_compound_beta_mini_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### deepseek-r1-distill-llama-70b {#groq_deepseek_r1_distill_llama_70b_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### distil-whisper-large-v3-en {#groq_distil_whisper_large_v3_en_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### gemma2-9b-it {#groq_gemma2_9b_it_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### llama-3.1-8b-instant {#groq_llama_3.1_8b_instant_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### llama-3.3-70b-versatile {#groq_llama_3.3_70b_versatile_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### llama-guard-3-8b {#groq_llama_guard_3_8b_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### llama3-70b-8192 {#groq_llama3_70b_8192_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### llama3-8b-8192 {#groq_llama3_8b_8192_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/llama-4-maverick-17b-128e-instruct {#groq_meta_llama_llama_4_maverick_17b_128e_instruct_details}--### Basic--- ❌ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/llama-4-scout-17b-16e-instruct {#groq_meta_llama_llama_4_scout_17b_16e_instruct_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/llama-guard-4-12b {#groq_meta_llama_llama_guard_4_12b_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/llama-prompt-guard-2-22m {#groq_meta_llama_llama_prompt_guard_2_22m_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### meta-llama/llama-prompt-guard-2-86m {#groq_meta_llama_llama_prompt_guard_2_86m_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### mistral-saba-24b {#groq_mistral_saba_24b_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### playai-tts {#groq_playai_tts_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### playai-tts-arabic {#groq_playai_tts_arabic_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### qwen-qwq-32b {#groq_qwen_qwq_32b_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### qwen/qwen3-32b {#groq_qwen_qwen3_32b_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### whisper-large-v3 {#groq_whisper_large_v3_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`--#### whisper-large-v3-turbo {#groq_whisper_large_v3_turbo_details}--### Basic--- ✅ `test_agent_loop_error`-- ❌ `test_async_greeting_with_assistant_message`-- ❌ `test_auto_summarize`-- ❌ `test_greeting_with_assistant_message`-- ❌ `test_greeting_without_assistant_message`-- ✅ `test_step_stream_agent_loop_error`-- ❌ `test_step_streaming_greeting_with_assistant_message`-- ❌ `test_step_streaming_greeting_without_assistant_message`-- ❌ `test_step_streaming_tool_call`-- ❌ `test_tool_call`--### Token Streaming--- ✅ `test_token_streaming_agent_loop_error`-- ❌ `test_token_streaming_greeting_with_assistant_message`-- ❌ `test_token_streaming_greeting_without_assistant_message`-- ❌ `test_token_streaming_tool_call`--### Multimodal--- ❌ `test_base64_image_input`-- ❌ `test_url_image_input`------