Skip to content

armgit / fara

Public Imported from GitHub
0

Fara1.5 – A family of frontier computer use agent models

self-hosted scenario fix (#13)

Alexey Taymanov2606c8e1c469
diff --git a/README.md b/README.md
index 6ed16c1..d46f1a9 100644
--- a/README.md
+++ b/README.md
@@ -173,6 +173,8 @@ The script will download the model to `model_checkpoints/fara-7b/` by default. Y
Second, host the model locally on a GPU machine:
```bash
+cd src/fara/vllm
+python -m pip install -r requirements.txt
python az_vllm.py --model_url /path/to/model_checkpoints/ --device_id 0,1
```
@@ -186,7 +188,7 @@ Run the test script to see Fara in action:
python test_fara_agent.py --task "how many pages does wikipedia have" --start_page "https://www.bing.com" --endpoint_config endpoint_configs/azure_foundry_config.json [--headful] [--downloads_folder "/path/to/downloads"] [--save_screenshots] [--max_rounds 100] [--browserbase]
```
-The `client_config` points to `"base_url": "http://localhost:5000/v1"` from the VLLM server above.
+In self-hosting scenario the `endpoint_config` points to `endpoint_configs/vllm_config.json` from the VLLM server above.
If you set `--browserbase`, export environment variables for the API key and project ID.
diff --git a/endpoint_configs/vllm_config.json b/endpoint_configs/vllm_config.json
new file mode 100755
index 0000000..7a727b4
--- /dev/null
+++ b/endpoint_configs/vllm_config.json
@@ -0,0 +1,5 @@
+{
+ "model": "gpt-4o-mini-2024-07-18",
+ "base_url": "http://localhost:5000/v1",
+ "api_key": "<YOUR API KEY>"
+}
\ No newline at end of file
diff --git a/src/fara/vllm/az_vllm.py b/src/fara/vllm/az_vllm.py
index 709e176..198f1fe 100644
--- a/src/fara/vllm/az_vllm.py
+++ b/src/fara/vllm/az_vllm.py
@@ -9,7 +9,7 @@ import subprocess
from pathlib import Path
from typing import Optional
-from .vllm_facade import VLLM, Status
+from vllm_facade import VLLM, Status
try:
from aztool.azcp import AzFolder, LocalFolder
diff --git a/src/fara/vllm/requirements.txt b/src/fara/vllm/requirements.txt
new file mode 100644
index 0000000..f78d6a8
--- /dev/null
+++ b/src/fara/vllm/requirements.txt
@@ -0,0 +1,4 @@
+uvicorn
+fastapi
+vllm==0.10.0
+transformers==4.55.0
\ No newline at end of file