Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

Terminal shows the app's version (0.2.0-beta) with the build number beside it; the archive README no longer contradicts itself, ships the launch doc it links and gives free-memory guidance for the system it is for; launch doc updated

Niko122164af23dafcb7
diff --git a/docs/TERMINAL_LAUNCH.md b/docs/TERMINAL_LAUNCH.md
index ffb22d40c..b950d80df 100644
--- a/docs/TERMINAL_LAUNCH.md
+++ b/docs/TERMINAL_LAUNCH.md
@@ -23,27 +23,32 @@ to your graphics card) comes with the first model: `strata launch` fetches the o
2. **Model**: the catalog of the app and `setup.py` (one source), with download size, what it needs (graphics memory + RAM) and "fits / tight / does not fit this computer".
A warning, never a block. The n-gram table stays on your SSD and is not counted. `o` = your own files (a folder or a .gguf; checked with the fingerprint table).
*(Cards: only with two or more usable graphics cards.)*
-3. **Folder**: where model files are stored, free space shown.
+3. **Folder**: where model files are stored, free space shown. For a model that is already installed (also one the desktop app installed) it starts where the model is, says
+ "Already downloaded" and lists only what is really missing with its size (the draft layer, about 5.2 GB; the picture reader, 0.9 GB). It never says "download the model" when only
+ a helper file is missing.
4. **Context** (8K to 512K, with the extra memory each costs) and the memory format (`m`: 8-bit or 4-bit).
-5. **Pictures**: only for models that can read them; says what it adds (about 0.9 GB: the picture-reader file, downloaded with the model through
- setup's Control mode, the same path the app's "Read pictures" uses). On a card under 24 GB and on AMD the picture reader runs on the CPU (nothing taken
- from the graphics card); on a bigger NVIDIA card it runs on the GPU. It needs `strata-vision` beside the engine in the engine package; the screen
- warns in plain words when the installed engine has none, and the run says "Pictures could not be switched on" instead of silently answering text only.
+5. **Pictures**: only for models that can read them. They are **on by default** (Yes is preselected; `--vision off` or "No" turns them off). It says what it adds (about 0.9 GB:
+ the picture-reader file, downloaded with the model through setup's Control mode, the same path the app's "Read pictures" uses). The reader runs on the graphics card
+ for the NVIDIA (CUDA) and AMD (HIP) builds (it keeps about 1.4 GB free) and on the CPU for the others. It needs `strata-vision` beside the engine in the engine package; with an
+ engine that has none, pictures default to off and the screen says so in plain words. A model that cannot read pictures stays off with no question.
6. **Parallel requests**: "fit to your card" (12 GB: 1, 20: 2, 24: 3, 32+: 4) or 1 to 4; a warning if it will not fit.
-7. **Tuning**: quick (about 20 minutes) / thorough (up to 2 hours) / skip (`strata tune auto`; the "answers may change very slightly" tick is off).
+7. **Tuning**: quick (about 20 minutes) / thorough (up to 2 hours) / skip (`strata tune auto`; the "answers may change very slightly" tick is off). The screens suggest quick the first
+ time (it is done once and kept); the command line skips tuning unless you add `--tune`.
8. **Advanced** (optional): every engine setting with default, tuned value and your value; edit, reset, add a setting by name (checked against the engine's own list; a warning, not a refusal);
- the exact command and the matching `strata launch` flags.
+ the exact command (it includes the context, memory format and picture choice you made above; it has its own scroll area: Tab, then the arrows) and the matching `strata launch` flags.
9. **Access**: this computer only (127.0.0.1), your network (LAN address, key), Cloudflare quick tunnel (needs `cloudflared`), Tailscale (needs it installed and connected); port; API key
(made for you beyond this computer: the server will not start on a network without one).
-10. **Save as preset** and launch. Then downloads (with progress), engine start, and the monitor.
+10. **Save as preset** and launch. Then downloads (with progress), engine start, and the monitor. In a small window the summary shrinks so the three choices stay on screen,
+ and the key line says what Enter will do. No web page opens by itself; only `--open` opens one.
Keys: arrows choose, Enter goes on, Esc goes back, `?` explains the screen, `q` leaves. Every screen says in plain words what it means and what to do next.
## After the start: the monitor
The Strata Monitor with the new **Overview** (key 1: model state bar, speed, GPU load, VRAM, temperature, power, PCIe, CPU, disk, context fill, experts in VRAM, system RAM, recent requests; read from the
-server's `/metrics`, the same data as the browser Monitor), the other five views as before, and **7 Connect**: the address, the key (hidden until `k`), and the setup for each app
-(`c` copy, `w` write it for you after a backup, `h` check it: one prompt and one tool call sent the way that app sends them). `q` asks: keep serving in the background, or stop
+server's `/metrics`, the same data as the browser Monitor), the other five views as before, and **7 Connect** (the seventh view: the arrow keys reach it like the others, and every view's hint says 1-7): the address, the key (hidden until `k`), and the setup for each app
+(`c` copy, `w` write it for you after a backup, `h` check it: one prompt and one tool call sent the way that app sends them). The list keeps two things apart: **Vendor tested** (the app
+version its maker tested the setup with: history) and **This server** (Not checked / Passed / Failed for your server, after `h`). `q` asks: keep serving in the background, or stop
(stopping unloads the model and frees the graphics memory).
In the classic Windows console (the Windows 10 default window) the monitor first switches that window's font to Cascadia Mono when it is installed (the console font API: no admin, nothing saved, gone with the window) and draws the full look; otherwise it draws a solid-block look: █ bars with a thin ▄ track, ▄ / █ sparklines, square boxes and the 16 palette colours (`•` instead of `●` when the font is not Consolas). It always shows one line: "For the full look, use Windows Terminal (built into Windows 11)". Windows Terminal, Linux and macOS are never touched. `STRATA_GLYPHS=unicode|ascii|solid` forces a look. With two or more graphics cards the Overview tiles show the busiest card (load), all cards (memory), the hottest card and the total power, a "Cards" strip lists each card (up to 4, then "press 5" for the Hardware view), and the CPU tile says "2 CPUs · 64 cores" from the server's new `sockets` figure.
@@ -57,6 +62,8 @@ strata launch --model iq3_xxs --context 128k --parallel 4 --vision off --tune qu
strata launch --model-path /data/models/IQ3_XXS --no-tui # files you already have
strata launch --preset Office --set prefill=65536 --set STRATA_PIN_BY_NEED=1 --no-tui
strata launch --plan --model q2_0 # what would happen; nothing changes
+strata launch --model q2_0 --yes --no-tui # no questions (for one: download on a computer with no engine yet)
+strata launch --model q2_0 --open # also open the server's page in your browser once it serves
strata launch --list-presets | --delete-preset NAME | --status | --stop [--preset NAME] | --show-key
```
@@ -76,6 +83,26 @@ words (for example: Claude Code needs `CLAUDE_CODE_MAX_CONTEXT_TOKENS`; OpenCode
`setup.py` sets up NVIDIA (CUDA) and AMD (HIP) cards. For an Intel Arc card its Linux path is a container build that the Terminal does not run: if the model's files and their prepared pack are
already on the disk, `strata launch` writes the run settings itself and says so; otherwise it says which file is missing. Nothing is downloaded or compiled in that case.
+## Models you already have
+
+`strata models list` looks in the same places the screens and `strata launch` look (one function, `strata_control/discovery.py`): the folder you name with `--install-dir` or
+STRATA_INSTALL_DIR (then only that one), the Terminal's own folder, the folders earlier installs remembered in settings.json, and the desktop app's install folder, which the Terminal
+shares (Windows `%LOCALAPPDATA%\Strata\install`; Linux and macOS `$XDG_DATA_HOME/strata/install`, by default `~/.local/share/strata/install`). The data folder is the one you set, else the
+app's (`app-settings.json` beside its install folder, else a `Documents/Strata` that holds models), else Strata's default. `strata models import PATH --plan` knows the Gyro-S and Gyro-M files
+(TQK6/TQK7 and TQ2_T experts) and every file set setup installs, also under another name when the header matches, and says which run config, prepared pack, draft layer and picture reader
+already exist for them. Nothing is copied or moved.
+
+## When there is no engine for this computer
+
+`strata launch` does not refuse. It says, in plain words, "Strata couldn't find an engine for this computer (the reason). The model will download, but it may not run until you install the right
+engine (next step: ...)", asks whether to go on (Enter means yes; `--yes` skips the question; nothing to download means no question), downloads the model, and ends with a message that names the
+engine problem and the next step. The exit code is 0 and nothing is started.
+
+## Cards of different makes
+
+One engine drives the cards of one make (CUDA: NVIDIA, HIP: AMD). Only those cards are offered for running and splitting; a model is never split across makes, and two models at the same
+time, one on each make, is not supported yet. When the other make has the bigger card you get one line: "This computer also has an AMD R9700 (32 GB). The AMD build of Strata can use it - it fits bigger models."
+
## Limits
* The model download uses `setup.py` in its control mode (no pip, no engine fetch, no compiler), as the desktop app does; it needs internet and room for the files.
diff --git a/packaging/package_terminal.py b/packaging/package_terminal.py
index 98842291a..9d0111c22 100644
--- a/packaging/package_terminal.py
+++ b/packaging/package_terminal.py
@@ -97,6 +97,7 @@ PAYLOAD = (
"requirements.txt",
"requirements-monitor.txt",
"docs/MANAGEMENT_PROTOCOL.md",
+ "docs/TERMINAL_LAUNCH.md", # the README below points at it (`strata launch`); it used to be missing from the archive
"data/gguf_fingerprints.json", # `strata models import --plan` names a known file set from this table
)
@@ -438,16 +439,30 @@ Put `bin/` on your PATH to get the `strata` command everywhere:
echo "export PATH=\\"$PWD/bin:$PATH\\"" >> ~/.profile
"""
+ ram = (" Task Manager > Performance > Memory > Available, or in PowerShell:\n"
+ " [math]::Round((Get-CimInstance Win32_OperatingSystem).FreePhysicalMemory / 1MB, 1) # GB free" if win else
+ " free -g" if not target.startswith("darwin") else
+ " Activity Monitor > Memory, or: vm_stat (free + inactive pages x 16 KB)")
return f"""Strata Terminal {version} ({target})
{'=' * (len(version) + len(target) + 18)}
This is the terminal distribution. It carries its own Python {PYTHON_VERSION} and its own packages, so you do not
-need Python, pip, Node, a compiler or a development checkout. Starting and serving never install or download
-anything; if something is missing, the error names the command that installs it.
+need Python, pip, Node, a compiler or a development checkout. Serving and watching (`strata serve`, `strata runtime`,
+`strata monitor`) never install or download anything; if something is missing, the error names the command that
+installs it. Only two commands fetch files, and only when you run them: `strata launch` (the engine and the model you
+choose, after it tells you the size) and `strata update` (the engine and the Strata components).
Install
-------
{install}
+Before you start a big model
+----------------------------
+A model of tens of GB needs free memory. Check how much is free first:
+
+{ram}
+
+Close other programs (browsers use a lot) if it is less than the model needs; `strata launch` shows what each model needs.
+
What it does NOT include
------------------------
The **engine** is a separate, independently versioned component and is not bundled: the right engine package
diff --git a/strata_cli/__init__.py b/strata_cli/__init__.py
index 038fe7315..8928a3366 100644
--- a/strata_cli/__init__.py
+++ b/strata_cli/__init__.py
@@ -5,6 +5,10 @@ from pathlib import Path
SCHEMA_VERSION = 1
+#: The version people are told: the Strata app's own (apps/desktop/src-tauri/Cargo.toml `version`; tests/terminal/test_tuiqa_version.py keeps the
+#: two equal). `version()` below is the files' build number, which the engine-compatibility checks compare, so it stays numeric.
+PRODUCT_VERSION = "0.2.0-beta"
+
def version() -> str:
"""The terminal's version: the VERSION file of an archive, else the source tree's CMake project version."""
diff --git a/strata_cli/cli.py b/strata_cli/cli.py
index 28b1f1891..933e0a3ca 100644
--- a/strata_cli/cli.py
+++ b/strata_cli/cli.py
@@ -33,7 +33,7 @@ class _VersionAction(argparse.Action):
def __call__(self, parser, namespace, values, option_string=None):
import strata_cli
- sys.stdout.write(f"strata {strata_cli.version()} (python {sys.version.split()[0]} at {sys.executable})\n")
+ sys.stdout.write(f"strata {strata_cli.PRODUCT_VERSION} (build {strata_cli.version()}; python {sys.version.split()[0]} at {sys.executable})\n")
parser.exit()
diff --git a/tests/terminal/test_tuiqa_version.py b/tests/terminal/test_tuiqa_version.py
new file mode 100644
index 000000000..dfa0c1e0b
--- /dev/null
+++ b/tests/terminal/test_tuiqa_version.py
@@ -0,0 +1,61 @@
+"""The Terminal tells people the app's version (0.2.0-beta), and its README no longer contradicts itself."""
+
+from __future__ import annotations
+
+import re
+import subprocess
+import sys
+import unittest
+from pathlib import Path
+
+import strata_cli
+
+ROOT = Path(__file__).resolve().parents[2]
+sys.path.insert(0, str(ROOT / "packaging"))
+import package_terminal # noqa: E402
+
+
+class ProductVersion(unittest.TestCase):
+ def test_it_equals_the_apps_own_version(self):
+ cargo = ROOT / "apps" / "desktop" / "src-tauri" / "Cargo.toml"
+ if not cargo.is_file():
+ self.skipTest("no app source in this tree")
+ m = re.search(r'^version\s*=\s*"([^"]+)"', cargo.read_text(encoding="utf-8"), re.MULTILINE)
+ self.assertEqual(strata_cli.PRODUCT_VERSION, m.group(1))
+
+ def test_dash_dash_version_leads_with_it_and_keeps_the_build_number(self):
+ out = subprocess.run([sys.executable, str(ROOT / "strata.py"), "--version"], capture_output=True, text=True, check=True).stdout
+ self.assertTrue(out.startswith(f"strata {strata_cli.PRODUCT_VERSION} (build {strata_cli.version()};"), out)
+ self.assertIn(sys.executable, out) # the packaging smoke test reads the interpreter back from this line
+
+ def test_the_build_number_stays_numeric_for_the_engine_compatibility_checks(self):
+ self.assertRegex(strata_cli.version(), r"^[0-9]+(\.[0-9]+)+$")
+
+
+class Readme(unittest.TestCase):
+ def readme(self, target: str) -> str:
+ return package_terminal._readme("0.2.0-beta", target)
+
+ def test_it_does_not_say_nothing_downloads_and_then_that_launch_fetches(self):
+ text = " ".join(self.readme("windows-x86_64").split())
+ self.assertNotIn("Starting and serving never install or download anything", text)
+ self.assertIn("Serving and watching", text)
+ self.assertIn("Only two commands fetch files", text)
+ self.assertIn("`strata launch`", text)
+
+ def test_the_launch_doc_it_points_at_is_in_the_archive(self):
+ self.assertIn("docs/TERMINAL_LAUNCH.md", self.readme("linux-x86_64"))
+ self.assertIn("docs/TERMINAL_LAUNCH.md", package_terminal.PAYLOAD)
+ self.assertTrue((ROOT / "docs" / "TERMINAL_LAUNCH.md").is_file())
+
+ def test_free_memory_guidance_fits_the_system(self):
+ win, lin, mac = (self.readme(t) for t in ("windows-x86_64", "linux-x86_64", "darwin-arm64"))
+ self.assertNotIn("free -g", win)
+ self.assertIn("FreePhysicalMemory", win)
+ self.assertIn("Task Manager", win)
+ self.assertIn("free -g", lin)
+ self.assertNotIn("free -g", mac)
+
+
+if __name__ == "__main__":
+ unittest.main()