The OSINT investigator's workbench. One case, one folder, every tool. Local-first: your media and your investigations never leave your machine.
JavaScript
Teach the e2e fixture the route the app now reads on mount, and let a slow runner drain (#74)
OsintMeThat63761973fe4e
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.ymlindex 1f94a85..f471467 100644--- a/.github/workflows/ci.yml+++ b/.github/workflows/ci.yml@@ -45,6 +45,18 @@ jobs:# alone is half of it. That range is split rather than trusted, since a# slow hour on the 30-minute timeout is a red run for no reason.#+ # These ranges look unbalanced by CPU seconds, and re-cutting them by that+ # number made the run worse rather than better: a split that put+ # `test_media_api.py` and `test_links_api.py` in one piece — even weight on+ # paper — failed the shard, because those two are where the suite contends+ # on locks and threads, and contention does not add up the way CPU seconds+ # do. `test_concurrent_sidecar_merges_do_not_drop_each_other` alone went to+ # two minutes there and the thread joins under it ran out. So the weight to+ # balance is Windows wall clock under `-n auto`, measured on Windows; a+ # Linux profile is not a stand-in for it, and cheap parallelism is not worth+ # a red run. Anyone re-cutting these should keep the lock-heavy files apart+ # and prove it on the box itself.+ ## The ranges have to leave no gap: one stopping at test_sc* beside one# starting at test_se* runs everything except the test_sd* somebody adds# next year, and says nothing. `tests/test_ci_shards.py` reads them backdiff --git a/frontend/e2e/app.fixture.js b/frontend/e2e/app.fixture.jsindex 8ac1b1d..c3f1775 100644--- a/frontend/e2e/app.fixture.js+++ b/frontend/e2e/app.fixture.js@@ -991,6 +991,17 @@ export async function installAppFixture(page, options = {}) {return json(route, { name: payload.name, path: made });}if (path === '/api/settings/scrapers') return json(route, { scrapers: [] });+ // Read on mount by `checkForUpdatesOnStart`, before any tool and whatever the+ // update switch says, because it touches disk rather than the network. The+ // fixture has no extension folder, which is the state a fresh install is in.+ if (path === '/api/settings/extension') {+ return json(route, {+ path: '/tmp/fixture/extension',+ bundled: { version: '0.0.0-fixture', payload: 'fixture-payload' },+ folder: null,+ staged: false,+ });+ }if (path === '/api/settings/ffmpeg') {return json(route, { available: true, version: 'fixture', source: 'bundled', path: '/tmp/ffmpeg' });}diff --git a/pyproject.toml b/pyproject.tomlindex 78c9ebc..6714065 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -169,6 +169,13 @@ packages = ["src/azimut"][tool.pytest.ini_options]testpaths = ["tests"]asyncio_mode = "auto"+# Print every thread's stack when one test runs longer than this, using the+# stdlib's own faulthandler rather than a plugin. A test that hangs on a runner+# we don't have takes the job's whole budget and is then killed with no name+# attached: the shard reports a percentage and nothing else. This turns that+# into a traceback. Well above the slowest test the suite has (`jobwait.WAIT`+# is 60), so a healthy run never prints.+faulthandler_timeout = 180[tool.ruff]# Match the codebase's existing wrap. Default rule set (E/F/W) only — this is adiff --git a/tests/conftest.py b/tests/conftest.pyindex defa1d0..d401141 100644--- a/tests/conftest.py+++ b/tests/conftest.py@@ -1,4 +1,6 @@import tempfile+import threading+import timeimport pytestfrom fastapi.testclient import TestClient@@ -62,10 +64,23 @@ def a_worker_of_its_own():Set up before the fixtures that own a temporary workspace, so the state isclean before a case exists at all. The worker is waited out first, becauseclearing the flag under a live thread would let a second one start.++ Waited out *to the end of the thread*, not just until the queue reads empty.+ `_reset_for_tests` keeps the flag when a worker is still alive — rightly, the+ flag is true then — but a test that starts in that state queues its work, is+ told a worker already has it, and starts none; and if the live one reaches+ the end of its queue in between, it leaves with the new work undrained. The+ test then waits its whole budget for a thumbnail nobody is rendering. Which+ box shows it is a coin toss, since what leaks the thread is the test that ran+ before it, and that is the randomized order's to choose."""from azimut.engine import workqueue- workqueue.wait_until_idle(timeout=10)+ workqueue.wait_until_idle(timeout=60)+ deadline = time.monotonic() + 60+ for thread in threading.enumerate():+ if thread.name == workqueue._WORKER_NAME and thread.is_alive():+ thread.join(max(0.0, deadline - time.monotonic()))workqueue._reset_for_tests()yield@@ -78,10 +93,19 @@ def _let_the_worker_finish() -> None:still leave the worker mid-write when its temp directory goes — which onPOSIX surfaces as a "directory not empty" teardown error and on Windows as alocked file. This is the same orderly-shutdown wait the app itself performs.++ The wait is generous because running out of it is silent: `wait_until_idle`+ reports the timeout in its return value and nothing here can act on it, so+ the only symptom is the rmtree that follows, failing in whichever test owned+ the directory. Ten seconds was enough on Linux and not on Windows, where the+ concurrent-download tests queue eight items' worth of enrichment behind+ `synchronous = FULL`. An idle worker returns immediately, so the ceiling+ costs nothing in the normal case; it only has to be above the slowest+ runner."""from azimut.engine import workqueue- workqueue.wait_until_idle(timeout=10)+ workqueue.wait_until_idle(timeout=60)def _settle_before_the_workspace_goes() -> None:diff --git a/tests/test_ingest_maptools.py b/tests/test_ingest_maptools.pyindex 1ae7ba4..783ae14 100644--- a/tests/test_ingest_maptools.py+++ b/tests/test_ingest_maptools.py@@ -198,15 +198,19 @@ def test_reference_media_is_searched_and_bounded(client):assert page["total"] == 5-def test_sky_answers_without_the_network(client, monkeypatch):- # the panel has to work on a map tab with no connection: the computation is- # local, and a test that lets it reach the network would not prove that- import socket-- def refuse(*args, **kwargs):- raise AssertionError("the sky route must not touch the network")-- monkeypatch.setattr(socket, "socket", refuse)+def test_sky_answers_the_panel_for_a_point(client):+ # The panel has to work on a map tab with no connection, and that guarantee+ # is held one layer down, where it can be held honestly:+ # `test_sky.py::test_sky_does_not_touch_the_network` makes every socket+ # explode and calls the computation directly.+ #+ # It cannot be re-proved through this route, because the client that would+ # ask is itself a loop that opens sockets to talk to its own server thread:+ # taking `socket.socket` away does not stop the route reaching out, it stops+ # the request arriving, and the call never returns. That deadlock has no+ # timeout under it, so it costs the whole Windows shard — twice, before the+ # traceback said so. What is left here is what only this layer can answer:+ # the route exists, takes a point, and hands the panel the shape it draws.day = client.get("/api/ingest/sky", params={"lat": 48.8584, "lon": 2.2945}, headers=_headers(client)).json()