Skip to content

armgit / uzu

Public Imported from GitHub
0

A high-performance inference engine for AI models

Update README.md

Aleksei Savincf881656d5d0
diff --git a/README.md b/README.md
index 1ab6d006c..25f258202 100644
--- a/README.md
+++ b/README.md
@@ -16,7 +16,7 @@
A high-performance inference engine for AI models on Apple Silicon. Key features:
- Simple, high-level API
-- [Hybrid architecture](https://docs.trymirai.com/overview/uzu#before-we-start), where layers can be computed as GPU kernels or via MPSGraph (a low-level API beneath CoreML with [ANE](https://trymirai.com/blog/iphone-hardware) access)
+- [Hybrid architecture](https://docs.trymirai.com/overview/uzu#before-we-start), where layers can be computed as GPU kernels or via MPSGraph (a low-level API beneath CoreML)
- Unified model configurations, making it easy to add support for new models
- Traceable computations to ensure correctness against the source-of-truth implementation
- Utilizes unified memory on Apple devices
@@ -27,7 +27,15 @@ For a detailed explanation of the architecture, please refer to the [documentati
### [Models](https://trymirai.com/models)
-`uzu` uses its own model format. To export a specific model, use [lalamo](https://github.com/trymirai/lalamo). First, get the list of supported models:
+`uzu` uses its own model format. To export a specific model, use [lalamo](https://github.com/trymirai/lalamo):
+
+```bash
+git clone https://github.com/trymirai/lalamo.git
+cd lalamo
+git checkout v0.5.2
+```
+
+After that, you can retrieve the list of supported models:
```bash
uv run lalamo list-models
@@ -39,12 +47,23 @@ Then, export the specific one:
uv run lalamo convert meta-llama/Llama-3.2-1B-Instruct
```
-Alternatively, you can download a prepared model using the sample script:
+Alternatively, you can download a test model using the sample script:
```bash
./scripts/download_test_model.sh
```
+Or you can download any supported model that has already been converted using:
+
+```bash
+cd ./scripts/tools/
+uv sync # install dependencies
+uv run main.py list-models # show the list of supported models
+uv run main.py download-model {REPO_ID} # download a specific model using repo_id
+```
+
+After that, you can find the downloaded model at `./models/{VERSION}/`.
+
### Bindings
- [uzu-swift](https://github.com/trymirai/uzu-swift) - a prebuilt Swift framework, ready to use with SPM