Make your computer a home for your AI partner. Avibe is the local-first Agent OS: Vibey, its built-in AI partner, works out of the box, and Claude Code, Codex & OpenCode run unchanged beside it — reach them from your browser, phone, or any chat app.
Regression Testing
回归测试 is the manual regression workflow for this repository. It runs Avibe
inside Incus so the environment behaves like a real long-running Linux machine:
systemd service, real home directory, persistent state, source sync, service
restart, and Show Runtime preparation.
It complements automated E2E tests instead of replacing them:
E2E testingkeeps using scripts and pytest for automatable scenarios.- capability scenario metadata lives under
tests/scenarios/ - multi-step auth/setup journeys should add or update
tests/scenarios/auth_setup/catalog.yamlandtests/scenarios/auth_setup/test_auth_setup_scenarios.py docs/regression/is a human-facing entry layer, not the canonical source of truth for scenario metadataRegression testingis for human-triggered checks on real IM platforms.
Scenario Metadata Navigation
Start here only if you are doing manual regression or need the human-readable index.
For deterministic scenario metadata, read:
tests/scenarios/INDEX.yamltests/scenarios/<capability>/catalog.yamltests/scenarios/<capability>/observations.yamltests/scenarios/<capability>/test_*.py
Runtime Model
The regression runner manages two local Incus environment types:
master: a long-running persistent regression environment.worktree: a temporary isolated environment for the current git worktree.
The master environment keeps product state across normal updates:
- platform credentials,
- Avibe Cloud remote-access pairing,
- agent CLI homes,
- Harness/session state,
- Show Page workspaces,
- Show Runtime cache where safe.
Worktree environments get their own Incus project/instance and host port. Their
mapping is recorded under .runtime/incus-regression/worktrees.json in the
primary checkout.
On macOS, run the Incus daemon in a local Linux VM and use the local machine as the operator/client. Development regression is local Incus only; do not use remote Incus hosts, remote tenant instances, demos, or customer/user environments for project testing.
Setup
Source synchronization requires rsync on the operator machine and in the
container. The regression base image already includes it; on a Linux operator
machine, install the distribution's rsync package if needed. The macOS system
rsync is supported. Transfers use Incus exec as the existing service user, not
an SSH daemon or host ownership metadata.
Configure the local Incus host.
python3 scripts/incus_regression.py doctorIf you are initializing a fresh Linux host directly:
python3 scripts/incus_regression.py init-host --minimalBuild or provide the reusable base image.
python3 scripts/incus_regression.py build-baseThe base image contains slow-changing dependencies such as Python, Node, build tools, and agent CLIs. Normal code updates do not rebuild this image. It is built and published in the Incus
defaultproject, where target projects read images from, whatever project the Incus client has selected.Copy the local env template:
cp .env.regression.example .env.regressionWhen the runner is invoked from a task worktree, it looks for
.env.regressionin that current worktree first, then falls back to the primary checkout. Keep the worktree copy intentional because it shadows the primary checkout's configuration.--env-file <path>names the file explicitly.The loaded file is authoritative for every key it defines, including keys it leaves empty: a value exported in the operator's shell under the same name is replaced, so an agent session's own
ANTHROPIC_*orOPENAI_*cannot reach the instance in place of the declared one. Keys the file does not mention keep their shell value. The runner prints the file it loaded and the names of the shell values it replaced, andupprints the name and source (env file, shell, or runner) of every key it writes into the instance. It never prints values.Fill in
.env.regressionwith:
- shared LLM credentials:
ANTHROPIC_API_KEY,OPENAI_API_KEY - optional API base URLs:
ANTHROPIC_BASE_URL,OPENAI_BASE_URL,OPENAI_API_BASE - optional UI proxy bind host:
REGRESSION_PORT_BIND_HOST - platform-specific bot credentials for Slack, Discord, Feishu, and WeChat
- the target regression channel for each platform, if you want channel routing preseeded at startup
- the backend that each platform's channel should pin to by default
Channel IDs are optional. If you leave them empty, the environment still starts and you can configure channels later from the Web UI.
- Keep these local-only files out of git:
.env.regression.runtime/incus-regression/
Usage
The compatibility entry point now uses Incus by default:
./scripts/run_regression.sh
Normal updates synchronize source incrementally using rsync's size/mtime quick
check. Unchanged files are not rewritten; stale source is deleted. Excluded
dependency/cache/runtime directories are pruned and preserved. --no-build-ui
includes local ui/dist in synchronization, so stale assets are deleted there
too. --clean remains an explicit full source reset, including generated files;
it does not reset product state. If a tool deliberately changes source while
preserving both its size and timestamp, use --clean to force replacement.
Environment files are different from reusable caches: host .env and .env.*
entries are never sent, and stale entries in the receiver's source directories
are deleted during normal synchronization. Excluded runtime/dependency trees
remain outside this cleanup boundary.
The same sender-only policy covers .DS_Store at any depth and these
repository-root host artifacts: .bot.pid, .gstack/, .tmp/,
desktop/target (including a symlink to an external build directory),
vibe/_version.py, and vibe/show_runtime_manifest.json. Their stale copies
are removed only from the disposable synced source, never from the operator's
checkout or persistent product home.
This is not a blanket .gitignore filter: worktree regression still accepts
uncommitted source files, and similarly named files inside test fixtures remain
source inputs. When checking a master deployment against its Git tree, compare
the filtered files and symlinks; empty directories carry no versioned content.
UI assets are built in the instance. Editable installation generates its own
Python version file when installation is needed; otherwise the source checkout
supports its build-less version fallback, and /api/version identifies the
deployed commit from the source receipt. Show Runtime uses the regression
archive prepared below, not a host's generated release-packaging manifest.
Show Runtime resolves the upstream commit on each update, but builds only when that commit, the target Node/platform/npm versions, the build recipe, or the archive checksum changes. The archive and its build receipt are kept together in the regression cache. Network resolution failures remain errors, not silent permission to deploy a stale build. Runtime preparation and validation still run on every update.
Updates and base-image builds from worktrees of the same primary checkout share one heavy-I/O slot per daemon. An update waits for that slot before creating an instance or stopping its existing service, and holds it through recovery if the update fails. Other environments keep serving while it waits. Old runner versions and independently cloned repositories do not participate in this shared lock.
Direct runner commands:
python3 scripts/incus_regression.py up --target master
python3 scripts/incus_regression.py status --target master
python3 scripts/incus_regression.py logs --target master
python3 scripts/incus_regression.py shell --target master
python3 scripts/incus_regression.py down --target master
Temporary worktree environment:
python3 scripts/incus_regression.py up --target worktree
python3 scripts/incus_regression.py status --target worktree
python3 scripts/incus_regression.py delete --target worktree --yes
python3 scripts/incus_regression.py reconcile
python3 scripts/incus_regression.py reconcile --yes
Protect a worktree environment while long-running work uses it, then release it when that work finishes:
python3 scripts/incus_regression.py hold --target worktree --slug <slug> --owner <session-id-or-name> --reason "Long-running evaluation"
python3 scripts/incus_regression.py status --target worktree --slug <slug>
python3 scripts/incus_regression.py release --target worktree --slug <slug>
The instance itself stores user.avibe-regression.hold.owner, .hold.reason,
and .hold.since (UTC), so another checkout sees the same hold. hold sets or
replaces it; release clears only these fields. status and reconcile show
the owner, reason, and time, including for instances without runner metadata.
delete refuses a held environment before removing anything, even with
--yes; only an explicit --ignore-hold overrides it. Deletion previews
(--dry-run) also check the live hold and require a reachable Incus daemon
unless explicitly overridden. Reconcile never includes that flag in its
printed commands. Holds do not expire automatically or stop
up/down, and old runners or raw Incus commands do not enforce them.
Fresh-install worktree environment, for testing the first-run experience:
python3 scripts/incus_regression.py up --target worktree --slug <slug> --seed none
--seed none builds the environment the way a new user's machine starts: an
empty Avibe home, no agent CLI logins, no platform configuration, and an
/etc/avibe-regression.env that holds only the runner's own settings, never the
LLM credentials or REGRESSION_* values from the env file. No platform tokens
are required. The service's first start writes the default configuration with
setup not completed, so the Web UI opens the setup wizard; remote-access pairing
happens in the product.
The mode is recorded on the instance (user.avibe-regression.seed), so later
updates of that slug stay unseeded without repeating the flag. An environment
changes mode only together with --reset-mode all, which empties its Avibe home
and agent CLI homes: --seed none --reset-mode all turns an existing worktree
environment back into a fresh install, and --seed regression --reset-mode all
seeds it again. Until that reset has rebuilt the home, the instance records
switching; if the switch is interrupted, every later up refuses until the
same --seed <mode> --reset-mode all is run again, because the home then
matches neither mode. --reset-mode config on a fresh environment removes the Avibe
config and state to rerun the wizard while keeping CLI logins made in it. The
master environment is always seeded.
Delete worktree environments promptly after the worktree is merged, abandoned,
or removed. Before updating, restarting, or deleting any regression
environment, check its hold with status or reconcile; if held, tell the user
who holds it and why, and do not proceed or offer a one-click delete.
Long-running work must hold its environment for as long as it runs, including
on master (hold --target master). The persistent master environment
should stay running and preserve its product state across normal source
updates. Other Sessions run long checks on it, so deploy only the merged
default branch there and verify an unmerged branch on a worktree target.
reconcile answers "what is actually still here?". It enumerates worktree
environments from Incus rather than from the runner's metadata, so an
environment created outside the runner shows up too — otherwise it is invisible
to every command that reads .runtime/incus-regression/worktrees.json, and can
only be removed by hand. For each one it prints what Incus was observed to hold
— its project and its instance, reported separately, because an environment can
be half gone — plus whatever provenance the metadata holds and the delete
command that removes it. Every instance Incus reported under that name is listed,
because Incus scopes instance names per project and one name can be several
instances. An instance living in a project other than the one its slug implies is
listed with that project and warned about by hand: delete --slug derives the
project from the slug, so it would report success without touching the instance.
An environment holding both kinds gets both the command and the warning — one
statement covers what the convention reaches, the other what it does not.
Those names are the ones the daemon reported, never derived a second time from
the slug. A slug here is an observed name with a known prefix removed, so it is
bounded by what Incus accepts rather than by what the runner would choose — and
--slug is stricter than Incus. A discovered name the runner would not have
minted therefore gets no command at all: the report names its project and
instance for a manual reclamation instead of printing a delete --slug that
would exit on its own argument.
Deletion stays a separate, explicit call. Nothing recorded about an environment
can prove it is no longer wanted: the recorded path is the checkout the runner
was invoked from, shared by every environment created there and still present
long after that worktree is gone; the slug is chosen by the caller; and an
environment may sit on a detached HEAD with no branch whose merge status could
be checked. reconcile --yes therefore changes exactly one thing — it drops
metadata rows for environments Incus no longer has, releasing their reserved
host ports. Without --yes it only reports, and --dry-run withholds the write
even with it.
Reporting is what the command is for, so having something to report is not a
failure: reconcile exits 0 whether or not any row was stale. A non-zero exit
for the case the command exists to show cannot be told apart from a command that
broke, by a && chain, a CI step, or anyone reading $? — while the report it
just printed says the run went fine.
"No longer has" is read strictly, because releasing a port that is still in use is worse than keeping a row nobody needs:
The daemon must have completed a listing that held neither the environment's project nor its instance, and every entry in that listing must have been readable. A daemon that could not be reached, or an entry the runner could not identify, is an unanswered question rather than an absence, and aborts instead.
A row whose slug a live
upis still holding is left alone.uprecords the slug and its port before it creates the project and the instance, so this is what a concurrentuplooks like from the outside. Who holds a slug is asked of the kernel, not of the file: anuptakes that environment's update lock — named by the daemon and the project, so a--remoterun is never read as the local environment of the same name — before it writes the row and holds it until the environment is built, soreconcileanswers the question by trying to take the same lock. Every command that changes what a slug names holds it, not justup: adeletetakes it across removing the objects and forgetting the row, and stops rather than waiting when somebody else has it. Those two halves are one change, and a delete that split them around a liveupwould remove nothing — the objects do not exist yet — drop that run's reservation anyway, and leave the finished environment untracked with its host port free for the next slug. Nothing a run writes could answer it. A record over-reports — a run killed outright leaves its own behind — and under-reports, since a run from an older checkout writes no field a newerreconcilewould recognise; a lock does neither, because the kernel drops it however the run ends and every version of the runner that builds a worktree environment takes the same one. So an abandoned reservation is reported as an environment the daemon no longer has and--yesprunes it, while a live run's row is kept without any operator having to tell the two apart. Not knowing counts as held: a platform withoutflock, or a lock file that cannot be opened, keeps the row like every other unanswered question here. Two things the rule does not cover, and both are in older checkouts rather than in the rule. Theirupwrites the row and then takes the lock, so areconcile --yeslanding between the two prunes a reservation whose build is about to start. What that exposes is those two operations and not the build: between releasing the mapping lock and acquiring the update lock they run onegit rev-parseand the two lock calls, and contention shortens the window rather than widening it, since a slug whose lock somebody is already holding reads as in flight. The older run then repairs the row itself, because itsuprewrites the mapping when the build finishes. What is left of it needs a thirdupto start inside that same window and take the freed port, and that ends loudly: anupcreating a local environment re-checks the host port inside its own update lock before it builds. Theirupalso keys the lock on the project alone, so a releasedup --remotetakes the local environment's lock and reads as a local run — which errs towards in flight, and keeps a row. Closing either from this side would mean deciding on the row's stamp again, written by a clock this command cannot check and no evidence about whether a run is live, or never pruning a row without a claim, which is every row any release has written.Giving a row back at the end of a failed
upis still worth doing — it frees the port now instead of at the nextreconcile --yes— and both of its conditions are read at that moment rather than remembered from an earlier one. The daemon must report no project for the slug, since a project it holds may bind that port. The row must also still be the one that run wrote, which is what the opaque claim in it is for:upmerges over whatever row it finds, so a later run can take the row over, and the first one failing afterwards must not free a port the second is building on.reserved_atandupdated_atare provenance for whoever reads the file, and nothing decides on them: a stamp written by another machine's clock, or by this one before it was corrected, is no evidence about whether anupis running right now. For the same reason a reservation records no branch or commit until it completes — the run that built the environment is the one that can say what it built — so a slug reserved again reports the branch it is still running, not the one being installed over it.worktrees.jsonis reached only through an accessor bound to the daemon it describes. The file reserves host ports on this machine and records what this machine's daemon holds, so every read of it and every write to it is a claim about exactly one authority, and a--remotecommand has no name for it: it neither reads nor writes a byte.reconcile --remotereports the remote inventory and says once that runner metadata is not shown;delete --remoteremoves the remote environment and says it kept the local row;up --remoterequires--host-port, because allocating from this machine's reservations is no evidence about which of another daemon's ports are free. Thedeletecommandsreconcile --remoteprints carry--remote, or they would name the same slug on the wrong daemon. The lock over that file belongs to the same accessor, for the same reason: a--remotecommand holds nothing of this file's inside its listings, so it does not keep local runs out of the file while it waits on another daemon.An empty
--remotenames this machine's daemon, normalized once where the flag is defined rather than at each reader. An unexpanded--remote "$INCUS_REMOTE"would otherwise be two authorities at once: the environment created here, while the accessor bound to some other daemon and recorded nothing about it — leaving the host port allocated with no row naming it.
Useful flags:
--host-port <port>: set the host-side Web UI proxy port.--slug <slug>: set the worktree environment slug.--reset-mode config: re-seed config/state/runtime.--reset-mode all: wipe and re-seed the environment state.--seed none: build a worktree environment as a fresh install; see above.--env-file <path>: load this env file instead of discovering.env.regression.--clean: compatibility flag; normal syncs already remove stale source files.--force-deps: force Python dependency refresh.--no-build-ui: skip UI asset build.--dry-run: print the planned Incus commands without changing the host.
The wrapper maps common legacy flags:
./scripts/run_regression.sh --status
./scripts/run_regression.sh --logs
./scripts/run_regression.sh --worktree
./scripts/run_regression.sh --reset-config
./scripts/run_regression.sh --dry-run
What You Get
On success, the runner prints one local UI URL:
Incus regression environment is ready:
URL: http://127.0.0.1:15130
Target: master
Project: avr-master
Instance: avibe-master
Show Runtime source: archive
Default names:
- master project:
avr-master - master instance:
avibe-master - master URL:
http://127.0.0.1:15130 - worktree project:
avr-wt-<slug> - worktree instance:
avibe-wt-<slug> - worktree ports: allocated from
15200-15399unless overridden
Architecture
The Incus runner separates slow-changing dependencies from fast-changing source:
- Base image: Ubuntu plus Python, Node, build tools, systemd unit helpers, and agent CLI prerequisites.
- Source sync: current worktree source is streamed into
/opt/avibe/source, excluding.git,.runtime, dependency directories, and generated assets. - Service: Avibe runs under
avibe-regression.serviceas useravibe. - Home:
/home/avibe/.avibeis the active product state home;/home/avibe/.vibe_remoteis a compatibility symlink. - Build identity: the version badge and
/api/versionreport the commit recorded by the latest source sync separately from install-time package metadata. Source targets do not use that package metadata for update prompts. The recorded branch, commit, and dirty flag are read before the sync and checked again after it; a checkout that moves in between fails the update rather than attributing the synced tree to either revision. - Show Runtime: every successful update runs
vibe runtime prepare --strictand then verifiesvibe runtime status --json.
The runner fingerprints dependency inputs:
- Python dependencies:
pyproject.toml,uv.lock - UI dependencies:
ui/package.json,ui/package-lock.json - UI source:
ui/src,ui/public,ui/index.html, Vite config, and TypeScript config - Show Runtime provider/ref
If fingerprints are unchanged, the runner skips unnecessary Python dependency
installation. Source syncs replace the source tree, so UI dependencies and UI
assets are rebuilt for each update to avoid serving stale or missing ui/dist
content.
Secret Safety
- Never commit
.env.regression. - Never commit generated files under
.runtime/. - Runtime secrets are written into the Incus instance through stdin to
/etc/avibe-regression.env; they should not appear in command-line logs. - A
--seed noneenvironment receives no secrets: neither the runtime env file nor the home contains credentials from the operator machine. - Share
.env.regression.exampleif you only need to show the structure.