Skip to content

πŸ™Œ OpenHands: AI-Driven Development

Add ArcticInference doc (#9492)

Graham Neubige05e6279578e
diff --git a/docs/usage/llms/local-llms.mdx b/docs/usage/llms/local-llms.mdx
index 4ca70f528a..06d390f1ed 100644
--- a/docs/usage/llms/local-llms.mdx
+++ b/docs/usage/llms/local-llms.mdx
@@ -175,6 +175,10 @@ vllm serve mistralai/Devstral-Small-2505 \
--enable-prefix-caching
```
+If you are interested in further improved inference speed, you can also try Snowflake's version
+of vLLM, [ArcticInference](https://www.snowflake.com/en/engineering-blog/fast-speculative-decoding-vllm-arctic/),
+which can achieve up to 2x speedup in some cases.
+
### Run OpenHands (Alternative Backends)
#### Using Docker