π OpenHands: AI-Driven Development
Add ArcticInference doc (#9492)
Graham Neubige05e6279578e
diff --git a/docs/usage/llms/local-llms.mdx b/docs/usage/llms/local-llms.mdxindex 4ca70f528a..06d390f1ed 100644--- a/docs/usage/llms/local-llms.mdx+++ b/docs/usage/llms/local-llms.mdx@@ -175,6 +175,10 @@ vllm serve mistralai/Devstral-Small-2505 \--enable-prefix-caching```+If you are interested in further improved inference speed, you can also try Snowflake's version+of vLLM, [ArcticInference](https://www.snowflake.com/en/engineering-blog/fast-speculative-decoding-vllm-arctic/),+which can achieve up to 2x speedup in some cases.+### Run OpenHands (Alternative Backends)#### Using Docker