AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.
Rust agent-memoryai-agentsbm25claude-codeclicode-searchcursordatabaseelasticsearchfull-text-searchhybrid-searchjsonllocal-firstmcpmcp-servermodel-context-protocolragrustsearch-enginevector-search
docs(air-gapped): copy the archive once — the double read was the defect all along
xerj-org4fa1b02ed63d
diff --git a/docs/recipes/air-gapped-deployment.md b/docs/recipes/air-gapped-deployment.mdindex 99ba9422e..a9cea952c 100644--- a/docs/recipes/air-gapped-deployment.md+++ b/docs/recipes/air-gapped-deployment.md@@ -108,9 +108,14 @@ VERSION="${TAG#v}"STAGE="xerj-${VERSION}-x86_64-unknown-linux-musl"ASSET="${STAGE}.tar.gz"+work="$(mktemp -d)"+trap 'rm -rf "$work"' EXIT+cp "/opt/xerj-staging/release/$ASSET" "$work/$ASSET"+cp "/opt/xerj-staging/release/$ASSET.sha256" "$work/$ASSET.sha256"+(set -eu- cd /opt/xerj-staging/release+ cd "$work"want="$(sha256sum "$ASSET" | cut -d ' ' -f 1)"if [[ ! "$want" =~ ^[0-9a-f]{64}$ ]]; then@@ -120,10 +125,8 @@ ASSET="${STAGE}.tar.gz"LC_ALL=C grep -qxF -e "$want $ASSET" -e "$want *$ASSET" \< <(tr -d '\r' < "$ASSET.sha256")- work="$(mktemp -d)"- trap 'rm -rf "$work"' EXIT- tar -xzf "$ASSET" -C "$work"- test -x "$work/$STAGE/xerj"+ tar -xzf "$ASSET"+ test -x "$STAGE/xerj"if ! getent passwd xerj >/dev/null; thensudo useradd --system --user-group --home-dir /var/lib/xerj \@@ -131,7 +134,7 @@ ASSET="${STAGE}.tar.gz"fisudo install -d -m 0755 /opt/xerj/bin /etc/xerjsudo install -d -o xerj -g xerj -m 0750 /var/lib/xerj- sudo install -m 0755 "$work/$STAGE/xerj" /opt/xerj/bin/xerj+ sudo install -m 0755 "$STAGE/xerj" /opt/xerj/bin/xerjecho "installed $ASSET ($want)")```@@ -159,21 +162,35 @@ operator's login shell on a bad digest — actively harmful over a serial consolwhere the natural recovery is to re-paste without it, which is the fail-openform. Inside, a failure ends the subshell and the operator keeps their session.-**The block that verifies is the block that installs.** Six earlier versions of-this recipe handed the verified tree to a later section through a path on disk,-and every one was broken through that handoff rather than through the digest:-a directory planted at the agreed name, a `mv` that nested onto it, an attacker-who computed the digest-named path themselves and swapped the tree before the-install ran. There is no handoff now — the binary is installed out of the-private work directory it was extracted into, inside the same subshell — so-there is nothing for a later step to trust and no exit status that has to be+**The archive is copied once, then hashed and extracted from that copy.** This+is the defect eight earlier versions of this recipe shared without noticing.+Whatever the digest check looked like, the archive at the staging path was+opened TWICE — once by `sha256sum`, once by `tar` — and anyone who can write+that directory swaps the file between the two opens. The genuine, untouched+`.sha256` then verifies, the block prints the genuine digest, exits `0`, and+installs a backdoor. Reproduced with plain regular files, no FIFOs, on a+correctly configured system. Copying first collapses that to a single read: the+bytes that are hashed are the same bytes that are extracted, because they are+the same file, and it is a file in a directory the attacker cannot reach.++**The block that verifies is the block that installs.** Six versions before that+handed the verified tree to a later section through a path on disk, and every+one was broken through the handoff rather than through the digest. There is no+handoff now, so nothing later has to be trusted and no exit status has to bechecked.-`mktemp -d` with no argument puts that directory under `TMPDIR`, deliberately-NOT under the staging directory: renaming or removing a directory entry needs-write permission on its PARENT, so a work directory inside an attacker-writable-staging area can be moved aside and replaced however restrictive its own mode-is.+**`mktemp -d` puts the work directory under `TMPDIR`, and that is an+assumption.** It is safe when `TMPDIR` is sticky and not attacker-writable —+the default `/tmp` on any normal system. It is NOT safe if `/tmp` has been made+world-writable without the sticky bit, or if `TMPDIR` points into a directory+the attacker controls: renaming a directory entry needs write on the parent, so+the work directory can then be swapped however restrictive its own mode is. If+your enclave is unusual here, set `TMPDIR` to a root-owned directory before+running these blocks.++An attacker who rewrites the archive AND recomputes its `.sha256` still passes,+and always will: that is the one case a checksum cannot answer, and the prose+above already says this is an integrity check and not a signature.## 2. Configure the base lexical node@@ -238,9 +255,15 @@ installing anything — the same fail-closed rule as section 1, and the reasonthe check is a command here rather than a sentence:```bash+mwork="$(mktemp -d)"+trap 'rm -rf "$mwork"' EXIT+cp /opt/xerj-staging/model/all-MiniLM-L6-v2/{config.json,tokenizer.json,model.safetensors} \+ "$mwork/"+cp /opt/xerj-staging/model/model.sha256 "$mwork/model.sha256"+(set -eu- cd /opt/xerj-staging/model/all-MiniLM-L6-v2+ cd "$mwork"for f in config.json tokenizer.json model.safetensors; dowant="$(sha256sum "$f" | cut -d ' ' -f 1)"@@ -249,7 +272,7 @@ the check is a command here rather than a sentence:exit 1fiLC_ALL=C grep -qxF -e "$want $f" -e "$want *$f" \- < <(tr -d '\r' < ../model.sha256)+ < <(tr -d '\r' < model.sha256)donesudo install -d -o xerj -g xerj -m 0750 /opt/xerj/models/all-MiniLM-L6-v2diff --git a/landing/docs/recipes/air-gapped-deployment.html b/landing/docs/recipes/air-gapped-deployment.htmlindex eb0da76ef..dfb50a246 100644--- a/landing/docs/recipes/air-gapped-deployment.html+++ b/landing/docs/recipes/air-gapped-deployment.html@@ -112,9 +112,14 @@ VERSION="${TAG#v}"STAGE="xerj-${VERSION}-x86_64-unknown-linux-musl"ASSET="${STAGE}.tar.gz"+work="$(mktemp -d)"+trap 'rm -rf "$work"' EXIT+cp "/opt/xerj-staging/release/$ASSET" "$work/$ASSET"+cp "/opt/xerj-staging/release/$ASSET.sha256" "$work/$ASSET.sha256"+(set -eu- cd /opt/xerj-staging/release+ cd "$work"want="$(sha256sum "$ASSET" | cut -d ' ' -f 1)"if [[ ! "$want" =~ ^[0-9a-f]{64}$ ]]; then@@ -124,10 +129,8 @@ ASSET="${STAGE}.tar.gz"LC_ALL=C grep -qxF -e "$want $ASSET" -e "$want *$ASSET" \< <(tr -d '\r' < "$ASSET.sha256")- work="$(mktemp -d)"- trap 'rm -rf "$work"' EXIT- tar -xzf "$ASSET" -C "$work"- test -x "$work/$STAGE/xerj"+ tar -xzf "$ASSET"+ test -x "$STAGE/xerj"if ! getent passwd xerj >/dev/null; thensudo useradd --system --user-group --home-dir /var/lib/xerj \@@ -135,10 +138,10 @@ ASSET="${STAGE}.tar.gz"fisudo install -d -m 0755 /opt/xerj/bin /etc/xerjsudo install -d -o xerj -g xerj -m 0750 /var/lib/xerj- sudo install -m 0755 "$work/$STAGE/xerj" /opt/xerj/bin/xerj+ sudo install -m 0755 "$STAGE/xerj" /opt/xerj/bin/xerjecho "installed $ASSET ($want)")</pre>- <p>Three details are load-bearing, and each replaces something that looked right and was not. <strong>The digest is computed here and the file is searched for it, not the other way round</strong> — <code class="inline">sha256sum -c</code> reads the filename out of the <code class="inline">.sha256</code>, a file that crossed the airgap on the same medium as the archive, skips <code class="inline">#</code> comments silently, and exits <code class="inline">0</code> if any one line verified, so a decoy digest plus a comment naming the archive passes while the archive is never hashed. Filtering that file first does not fix it: <code class="inline">awk '$2 == asset'</code> compares one token while <code class="inline">sha256sum</code> takes the whole rest of the line as the filename, so a line naming <code class="inline">$ASSET decoy</code> satisfies the filter and hashes a decoy. Building the expected line in the shell and demanding it with <code class="inline">grep -qxF</code> removes the disagreement. The two patterns accept GNU text mode and <code class="inline">-b</code> binary mode; <code class="inline">tr -d '\r'</code> accepts a checksum file written on Windows. <strong><code class="inline">set -eu</code> is inside the subshell</strong> — at the top of the block it kills the operator’s login shell on a bad digest, and the natural recovery is to re-paste without it, which is the fail-open form. <strong>Section 2 installs from <code class="inline">$STAGE.verified</code></strong>, which <code class="inline">mv</code> creates only after the digest, the extraction and <code class="inline">test -x</code> have all succeeded — so the block is safe even when its exit status is ignored, because on any failure there is nothing at the install path. That last part is only true because <code class="inline">rm -rf</code> runs <strong>first</strong>: placed after the digest check it is unreachable on exactly the failure that matters, and a <code class="inline">$STAGE.verified</code> left by an earlier run — or planted by anyone who can write to the staging directory — would survive and be installed.</p>+ <p><strong>The archive is copied once, then hashed and extracted from that copy.</strong> Eight earlier versions of this recipe opened the archive twice at the staging path — once by <code class="inline">sha256sum</code>, once by <code class="inline">tar</code> — and anyone who can write that directory swaps the file between the two opens: the genuine <code class="inline">.sha256</code> verifies, the block prints the genuine digest and exits 0, and a backdoor is installed. Copying first collapses that to a single read, because the bytes hashed and the bytes extracted are the same file, in a directory the attacker cannot reach. <strong>The digest is computed here and the file is searched for it</strong>, not the other way round: <code class="inline">sha256sum -c</code> reads filenames out of the <code class="inline">.sha256</code> and exits 0 if any one line verified. <strong>The block that verifies is the block that installs</strong>, so nothing later has to be trusted and no exit status has to be checked. <strong><code class="inline">mktemp -d</code> puts the work directory under <code class="inline">TMPDIR</code></strong>, which is safe when that is sticky and not attacker-writable (the default <code class="inline">/tmp</code>); if yours is not, point <code class="inline">TMPDIR</code> at a root-owned directory first.</p><h2>2. Configure the base lexical node</h2><p>Create the service account first, then install only the staged binary. No model directory is needed for lexical mode:</p>@@ -174,9 +177,15 @@ donesha256sum config.json tokenizer.json model.safetensors > ../model.sha256)</pre><p>Transfer <code class="inline">OUT/model</code> with the release. Verify it inside the enclave before installing anything — the same fail-closed rule as section 1, and the reason the check is a command here rather than a sentence:</p>-<pre class="code">(+<pre class="code">mwork="$(mktemp -d)"+trap 'rm -rf "$mwork"' EXIT+cp /opt/xerj-staging/model/all-MiniLM-L6-v2/{config.json,tokenizer.json,model.safetensors} \+ "$mwork/"+cp /opt/xerj-staging/model/model.sha256 "$mwork/model.sha256"++(set -eu- cd /opt/xerj-staging/model/all-MiniLM-L6-v2+ cd "$mwork"for f in config.json tokenizer.json model.safetensors; dowant="$(sha256sum "$f" | cut -d ' ' -f 1)"@@ -185,7 +194,7 @@ doneexit 1fiLC_ALL=C grep -qxF -e "$want $f" -e "$want *$f" \- < <(tr -d '\r' < ../model.sha256)+ < <(tr -d '\r' < model.sha256)donesudo install -d -o xerj -g xerj -m 0750 /opt/xerj/models/all-MiniLM-L6-v2