One self-hosted app for your shows, movies and music.
title: Testing description: Run the checks CI runs, then the end-to-end tests for the area you changed. level: Setup sidebar: { order: 5 }
Check your change on your machine before you send it.
Before you start
Finish Setup. The Go tests serve the built web app, so build it first.
Install the test browser once:
npx --prefix app/frontend playwright install chromium
Run the everyday checks
Build and vet the Go code:
go build ./... go vet ./...Run the Go tests:
go test ./...Check and test the web app:
npm --prefix app/frontend run check npm --prefix app/frontend run lint npm --prefix app/frontend run testRun the layer and file-layout rules:
go test -tags structure ./app/backend/tools/structurecheck
You should see no failure. CI runs the same commands, the Go ones on amd64 and arm64, plus one run with the race detector (go test -race ./...).
:::note
The full go test ./... runs long. Use make test, which sets a 25 min timeout. Go's default is 10 min.
:::
Run the end-to-end tests
End-to-end tests drive a real browser against a real program. Run the project for the area you touched:
npm --prefix app/frontend run e2e -- --project=movies
The run starts its own HolyArrs on a fresh config folder. The setup project makes the owner first, and most other projects wait for it. Run the full suite with no --project before a release.
Choices
Which projects to run
| You changed | Run |
|---|---|
| Movies | movies |
| Shows | shows |
| Requests | requests, lists, reports |
| Downloads | sources, downloaders, downloads |
| Sign-in and users | accounts, kid |
| Health and schedules | schedules, controls |
| Page layout | desktop, phone, a11y |
Projects are named in app/frontend/playwright.config.ts. A project that depends on another runs it first.
Where a new test goes
| Kind | Where |
|---|---|
| Go | Beside the file it tests |
| Web unit | Beside its file, as name.test.ts |
| End-to-end | app/tests/<module>/ |
Never call a real outside service. Add a stand-in under app/tests/fakes. A test that passes only on your machine is broken.
If it does not work
Symptom. A structure check fails. Cause. The message names the file and the rule you broke. Fix. Move the code, or pass plain data in from the higher module. See Modules.
Symptom. End-to-end tests cannot start.
Cause. The test browser is not installed.
Fix. Run the playwright install line above.
Symptom. The Go tests fail on missing pages.
Cause. The web app is not built.
Fix. Run npm --prefix app/frontend run build.
For experts
Build tags
A test with a tag skips in a plain go test ./.... Pass -tags <tag> to run it.
| Tag | Holds |
|---|---|
structure |
The layer, file-layout, SQL-placement and comment rules in tools/structurecheck |
measure |
Download timing checks |
lab |
Tests against real apps in a test lab. They read HOLYARRS_LAB_* and skip when those are unset |
labms, livesubs |
Lab runs for media servers and live subtitle sites |
Long and fault runs
CI runs these on top of the plain tests. Run them before a release.
go test -count=1 -run '^TestFault|^TestCrashInEveryLoopIsContained$' ./app/backend/cmd/holyarrs
HOLYARRS_RESTART_TIMING=1 go test -count=1 -run '^TestCrashRestartTime$' ./app/backend/cmd/holyarrs
HOLYARRS_MEMORY=1 go test -count=1 -timeout 30m -run '^TestMemoryPartsRuns$' ./app/backend/cmd/holyarrs
go test -count=1 -run '^TestDownloaderConformance$' ./app/backend/internal/downloads
| Variable | Does |
|---|---|
HOLYARRS_FAULT_OUT |
File the fault runs write their table to |
HOLYARRS_RESTART_TIMING, HOLYARRS_RESTART_TIMING_OUT |
Turn on the crash-to-sign-in timing and name its output file |
HOLYARRS_MEMORY, HOLYARRS_MEMORY_OUT |
Turn on the memory runs and name their output file |
HOLYARRS_CRASH_RUNS, HOLYARRS_CRASH_SEED |
Rounds and seed for the crash loop in app/backend/internal/mediafiles/dbtests (crash_test.go), which skips under -short |
Stand-in servers and variables
End-to-end tests start stand-in servers for Downloaders, Sources and the metadata server, so no test reaches a real service. The tests set variables such as HOLYARRS_TMDB_URL, HOLYARRS_MUSICBRAINZ_URL, HOLYARRS_PLEX_URL and HOLYARRS_QBITTORRENT_URL to point the program at them. A real install never needs them.
The API contract has its own tests. TestOpenAPIGenerated fails when openapi.json drifts from the routes, and the route-table snapshot in tools/structurecheck/testdata fails on any added, removed or moved route.