Skip to content

armgit / zed

Public Imported from GitHub
0

Code at the speed of thought – Zed is a high-performance, multiplayer code editor from the creators of Atom and Tree-sitter.

language_models: Clear speed for OpenAI-compatible providers that don't support fast mode (#59496)

Anant Goelc4e30593b16f
diff --git a/crates/language_models/src/provider/open_ai_compatible.rs b/crates/language_models/src/provider/open_ai_compatible.rs
index c1f9f70a154..da165a77b5f 100644
--- a/crates/language_models/src/provider/open_ai_compatible.rs
+++ b/crates/language_models/src/provider/open_ai_compatible.rs
@@ -322,7 +322,7 @@ impl LanguageModel for OpenAiCompatibleLanguageModel {
fn stream_completion(
&self,
- request: LanguageModelRequest,
+ mut request: LanguageModelRequest,
cx: &AsyncApp,
) -> BoxFuture<
'static,
@@ -334,6 +334,12 @@ impl LanguageModel for OpenAiCompatibleLanguageModel {
LanguageModelCompletionError,
>,
> {
+ // `speed` can leak in from a parent thread's model; this provider never
+ // supports fast mode, and arbitrary compatible endpoints reject `service_tier`.
+ if !self.supports_fast_mode() {
+ request.speed = None;
+ }
+
if self.model.capabilities.chat_completions {
let request = into_open_ai(
request,