Skip to content

SWE-agent takes a GitHub issue and tries to automatically fix it, using your LM of choice. It can also be employed for offensive cybersecurity or competitive coding challenges. [NeurIPS 2024]

Doc: Improve documentation of local models

Kilian Lierete439081eff5e
diff --git a/docs/faq.md b/docs/faq.md
index f0cd2638..eb1d8f70 100644
--- a/docs/faq.md
+++ b/docs/faq.md
@@ -6,8 +6,8 @@ Probably all of them! There's even a few for testing. See [models](installation/
> What can I do if my model doesn't support function calling?
-You can configure how to parse the model's response by choosing your `parse_function`.
-The default now is `function_calling`, but you can choose to `thought_action`.
+You can configure how to parse the model's response by choosing your `agent.tools.parse_function`.
+The default now is `function_calling`, but you can change it to `thought_action`.
More information in the [reference](reference/parsers.md).
There are also some config example in our [config folder](https://github.com/SWE-agent/SWE-agent/tree/main/config).
diff --git a/docs/installation/keys.md b/docs/installation/keys.md
index fde8cd0a..f102726f 100644
--- a/docs/installation/keys.md
+++ b/docs/installation/keys.md
@@ -22,7 +22,7 @@ Here's an example
See the following links for tutorials on obtaining [Anthropic](https://docs.anthropic.com/en/api/getting-started), [OpenAI](https://platform.openai.com/docs/quickstart/step-2-set-up-your-api-key), and [Github](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) tokens.
-## Supported models
+## Supported API models
We support all models supported by [litellm](https://github.com/BerriAI/litellm), see their list [here](https://docs.litellm.ai/docs/providers).
@@ -34,19 +34,43 @@ Here are a few options for `--agent.model.name`:
| `gpt-4o` | `OPENAI_API_KEY` | |
| `o1-preview` | `OPENAI_API_KEY` | You might need to set temperature and sampling to the supported values. |
-!!! hint "Parsers"
+!!! warning "Function calling and more: Setting the correct parser"
- The default config uses function calling to retrieve actions from the model response.
- If your model doesn't support function calling, you can use the `thought_action` parser.
- See [our API docs](../reference/parsers.md) for more details.
- Remember to document the tools in your prompt as the model will not be able to see the function signature
- like with function calling.
+ The default config uses function calling to retrieve actions from the model response, i.e.,
+ the model directly provides the action as a JSON object.
+ If your model doesn't support function calling, you can use the `thought_action` parser by setting
+ `agent.tools.parse_function` to `thought_action`.
+ Then, we extract the last triple-backticks block from the model response as the action.
+ See [our API docs](../reference/parsers.md) for more details on parsers.
+ Remember to document the tools in your prompt as the model will not be able to see the function signature
+ like with function calling.
-## Model options
+## Using local models
-See [our API docs](../reference/model_config.md) for more details.
+We currently support all models that serve to an endpoint with an OpenAI-compatible API.
-### Models for testing
+For example, to use llama, you can folloow the [litellm instructions](https://docs.litellm.ai/docs/providers/ollama) and set
+
+```
+agent:
+ model:
+ name: ollama/llama2
+ api_base: http://localhost:11434
+```
+
+Please see the above note about using a config that uses the `thought_action` parser instead of the function calling parser.
+
+!!! warning "Exit conditions"
+
+ If your local model does not have a cost assigned to, you can use `agent.model.per_instance_call_limit` to limit the runtime per issue.
+
+## Complete model options
+
+!!! hint "Complete model options"
+
+ See [our API docs](../reference/model_config.md) for all available options.
+
+## Models for testing
We also provide models for testing SWE-agent without spending any credits
@@ -60,4 +84,6 @@ We also provide models for testing SWE-agent without spending any credits
whether you set `OPENAI_API_BASE_URL` correctly (if you're not using it, the
line should be deleted or commented out).
Also see [this issue](https://github.com/SWE-agent/SWE-agent/issues/467)
- for reference.
\ No newline at end of file
+ for reference.
+
+% include-markdown "../_footer.md" %}
\ No newline at end of file
diff --git a/docs/reference/model_config.md b/docs/reference/model_config.md
index ab554f79..16315e27 100644
--- a/docs/reference/model_config.md
+++ b/docs/reference/model_config.md
@@ -1,8 +1,14 @@
# Models
+!!! hint "Tutorial"
+
+ Please see the [model section in the installation guide](../installation/keys.md) for an overview of the different models and how to configure them.
+
This page documents the configuration objects used to specify the behavior of a language model (LM).
-## Normal LMs
+In most cases, you will want to use the `GenericAPIModelConfig` object.
+
+## API LMs
::: sweagent.agent.models.GenericAPIModelConfig
options: