Skip to content

Turn any technical book PDF into a Claude Code skill — ready to study, reference, and use while you work.

fix(config): give each run its own workdir so concurrent extractions cannot clobber each other (#184)

Jean Giet7bcfcd526232
diff --git a/SKILL.md b/SKILL.md
index bb860ba..12e33b2 100644
--- a/SKILL.md
+++ b/SKILL.md
@@ -163,17 +163,21 @@ Before extraction, the script checks optional Python packages needed for the det
**Tip — preflight the environment:** run `"$PYTHON_BIN" "$SCRIPT_PATH" --check` to print a per-format report of which extractors are installed and the exact command to install whatever is missing, without processing any file. Useful when a user reports a setup or quality problem.
-This creates:
-- `<tempdir>/book_skill_work/full_text.txt` — combined extracted text of all sources with clear visually demarcated boundaries.
-- `<tempdir>/book_skill_work/metadata.json` — overall combined size, words, pages, token counts, dropped EPUB image counts, and a detailed list of individual processed `sources`.
+This creates a **per-run** work directory — `<tempdir>/book_skill_work-<pid>/` by default, or exactly the path you set in `BOOK_SKILL_WORKDIR` — containing:
+- `full_text.txt` — combined extracted text of all sources with clear visually demarcated boundaries.
+- `metadata.json` — overall combined size, words, pages, token counts, dropped EPUB image counts, the resolved `workdir`, and a detailed list of individual processed `sources`.
-Read `<tempdir>/book_skill_work/metadata.json` to inspect the results.
+The run prints all three paths on completion (`Workdir ->`, `Text ->`, `Meta ->`). **Take the paths from that output (or from `metadata.json`'s own `workdir` field) rather than assuming a fixed location** — the directory name differs per run so that concurrent extractions on one machine cannot overwrite each other's results.
+
+Read that run's `metadata.json` to inspect the results.
+
+**Always confirm the extraction is the document you asked for** before generating anything: check `filename` / `source_file` in `metadata.json`, or the `SOURCE:` header on the first line of `full_text.txt`. If you are waiting on a background run, wait on *its* specific workdir — polling a shared path can surface a different run's output.
---
## Step 2.5 — Pre-flight cost estimate
-Read `<tempdir>/book_skill_work/metadata.json` and present the user with an estimate **before doing any generation**:
+Read this run's `metadata.json` (the `Meta ->` path from the extraction output) and present the user with an estimate **before doing any generation**:
```
📖 Sources detected: <total_sources> source(s)
@@ -542,18 +546,35 @@ if ! command -v "$PYTHON_BIN" >/dev/null 2>&1; then
PYTHON_BIN="python"
fi
-"$PYTHON_BIN" - <<'PY'
-import os
+Remove **the work directory this run actually used** — the `Workdir ->` path from the
+extraction output, which is also stored as `workdir` in `metadata.json`. Never delete a
+directory you did not create: another extraction may be running beside yours.
+
+```bash
+# WORKDIR is the path this run reported; quote it in case of spaces.
+rm -rf "$WORKDIR"
+```
+
+Equivalently, if you still have the metadata file:
+
+```bash
+"$PYTHON_BIN" - "$WORKDIR_METADATA_JSON" <<'PY'
+import json
import shutil
-import tempfile
+import sys
from pathlib import Path
-shutil.rmtree(
- os.environ.get("BOOK_SKILL_WORKDIR", Path(tempfile.gettempdir()) / "book_skill_work"),
- ignore_errors=True,
-)
+
+meta_path = Path(sys.argv[1])
+workdir = json.loads(meta_path.read_text(encoding="utf-8")).get("workdir")
+if workdir:
+ shutil.rmtree(workdir, ignore_errors=True)
PY
```
+Older copies of this file removed a single fixed `book_skill_work` directory. That path is
+no longer used, so such a cleanup is now a harmless no-op rather than something that could
+delete a concurrent run's output.
+
Then report to the user:
```
@@ -653,7 +674,7 @@ Read and parse the existing skill's files:
- Read `$SKILLS_HOME/<skill_name>/glossary.md`, `$SKILLS_HOME/<skill_name>/patterns.md`, and `$SKILLS_HOME/<skill_name>/cheatsheet.md` to see what terms and frameworks are already indexed.
### 2. Match Content & Identify Revisions vs. Additions
-Analyze the new extracted text in `<tempdir>/book_skill_work/full_text.txt` to identify if the new content represents:
+Analyze the new extracted text in this run's `full_text.txt` (the `Text ->` path from the extraction output) to identify if the new content represents:
- **Updates/Revisions to existing chapters**: If a section of the new content directly updates or expands an existing chapter's topic, read the existing chapter file, merge the new details into it, and rewrite the file.
- **New additions**: If the content introduces new chapters, papers, or separate sections, create **new chapter summary files** under `chapters/`. Start numbering these files after the highest existing chapter number (e.g. if the existing chapters stop at `ch12`, create `ch13-*.md`, `ch14-*.md`, etc.).
diff --git a/book_to_skill/config.py b/book_to_skill/config.py
index 18ca0ce..b33130f 100644
--- a/book_to_skill/config.py
+++ b/book_to_skill/config.py
@@ -2,12 +2,29 @@ import os
import tempfile
from pathlib import Path
-OUTPUT_DIR = Path(
- os.environ.get(
- "BOOK_SKILL_WORKDIR",
- str(Path(tempfile.gettempdir()) / "book_skill_work"),
- )
-)
+def default_output_dir() -> Path:
+ """Per-run work directory, unique to this process.
+
+ The PID is part of the name so two extractions running at the same time
+ cannot overwrite each other. Every run previously shared one fixed path
+ ($TMPDIR/book_skill_work), so whichever run finished second silently
+ replaced the first run's full_text.txt and metadata.json — and an agent
+ polling for metadata.json could pick up a *different document's*
+ extraction without any error, then build a skill from the wrong source.
+
+ The name is deliberately a sibling of the old fixed path rather than a
+ child of it. An older cleanup routine that removes "book_skill_work"
+ then simply finds nothing, instead of deleting a live concurrent run.
+
+ BOOK_SKILL_WORKDIR still overrides this completely.
+ """
+ return Path(tempfile.gettempdir()) / f"book_skill_work-{os.getpid()}"
+
+
+# `or` rather than a get() default: BOOK_SKILL_WORKDIR set to an empty string
+# would otherwise become Path(""), i.e. the current directory — which the run
+# would then populate and chmod to 0700.
+OUTPUT_DIR = Path(os.environ.get("BOOK_SKILL_WORKDIR") or default_output_dir())
OUTPUT_TEXT = OUTPUT_DIR / "full_text.txt"
OUTPUT_META = OUTPUT_DIR / "metadata.json"
diff --git a/book_to_skill/utils.py b/book_to_skill/utils.py
index f0eb504..2b2d7ee 100644
--- a/book_to_skill/utils.py
+++ b/book_to_skill/utils.py
@@ -1066,6 +1066,9 @@ def main():
"estimated_tokens": total_tokens,
"estimated_tokens_human": f"~{total_tokens // 1000}K",
"images_dropped": total_images_dropped,
+ # Self-describing so a consumer can clean up exactly the directory this
+ # run created, without having to reconstruct the per-run default path.
+ "workdir": str(OUTPUT_DIR),
"output_text": str(OUTPUT_TEXT),
"total_sources": len(extracted_sources),
"sources": [
@@ -1127,8 +1130,9 @@ def main():
" WARN : No table of contents detected — chapter mapping in Step 3 "
"will rely on heading scan only, which may miss or duplicate sections."
)
- print(f"\n Text -> {OUTPUT_TEXT}")
- print(f" Meta -> {OUTPUT_META}")
+ print(f"\n Workdir -> {OUTPUT_DIR}")
+ print(f" Text -> {OUTPUT_TEXT}")
+ print(f" Meta -> {OUTPUT_META}")
if errors:
print(f"\n WARNING: {len(errors)} source(s) skipped due to errors:")
for path, err in errors:
diff --git a/tests/test_per_run_workdir.py b/tests/test_per_run_workdir.py
new file mode 100644
index 0000000..ae3cf70
--- /dev/null
+++ b/tests/test_per_run_workdir.py
@@ -0,0 +1,99 @@
+"""Regression tests: each run gets its own work directory.
+
+Every extraction used to write to one fixed path ($TMPDIR/book_skill_work),
+so two runs started at the same time — routine when several agent sessions
+share a machine — silently overwrote each other's full_text.txt and
+metadata.json. Nothing errored: the run that finished second simply replaced
+the first one's output, and an agent waiting on metadata.json could pick up a
+*different document's* extraction and build a skill from the wrong source.
+
+The default now carries the PID. BOOK_SKILL_WORKDIR still overrides it, and an
+empty BOOK_SKILL_WORKDIR must not collapse to Path("") — i.e. the current
+directory, which the run would then populate and chmod to 0700.
+"""
+
+import importlib
+import os
+import sys
+import tempfile
+from pathlib import Path
+
+ROOT_DIR = Path(__file__).resolve().parent.parent
+sys.path.insert(0, str(ROOT_DIR))
+
+from book_to_skill.config import default_output_dir # noqa: E402
+
+
+def _reimport_config(monkeypatch, workdir_env):
+ """Re-evaluate config.py's module-level OUTPUT_DIR under a given env."""
+ if workdir_env is None:
+ monkeypatch.delenv("BOOK_SKILL_WORKDIR", raising=False)
+ else:
+ monkeypatch.setenv("BOOK_SKILL_WORKDIR", workdir_env)
+ import book_to_skill.config as config
+
+ return importlib.reload(config)
+
+
+def test_default_workdir_is_unique_per_process(monkeypatch):
+ """Two concurrent runs must not share a directory."""
+ monkeypatch.setattr(os, "getpid", lambda: 1111)
+ first = default_output_dir()
+ monkeypatch.setattr(os, "getpid", lambda: 2222)
+ second = default_output_dir()
+
+ assert first != second
+ assert "1111" in first.name
+ assert "2222" in second.name
+
+
+def test_default_workdir_is_sibling_not_child_of_legacy_path(monkeypatch):
+ """An older cleanup that removes "book_skill_work" must not hit a live run.
+
+ Nesting under the legacy directory would mean a stale cleanup routine
+ deleting a concurrent extraction's output, which is worse than the bug
+ being fixed. The per-run directory is therefore a sibling.
+ """
+ monkeypatch.setattr(os, "getpid", lambda: 4242)
+ legacy = Path(tempfile.gettempdir()) / "book_skill_work"
+
+ assert legacy not in default_output_dir().parents
+
+
+def test_default_workdir_lives_in_tempdir(monkeypatch):
+ monkeypatch.setattr(os, "getpid", lambda: 4242)
+
+ assert default_output_dir().parent == Path(tempfile.gettempdir())
+
+
+def test_env_override_wins(monkeypatch, tmp_path):
+ chosen = tmp_path / "private-workdir"
+ config = _reimport_config(monkeypatch, str(chosen))
+
+ assert config.OUTPUT_DIR == chosen
+ assert config.OUTPUT_TEXT == chosen / "full_text.txt"
+ assert config.OUTPUT_META == chosen / "metadata.json"
+
+
+def test_empty_env_override_does_not_become_cwd(monkeypatch):
+ """BOOK_SKILL_WORKDIR="" must fall back, not resolve to Path("")."""
+ config = _reimport_config(monkeypatch, "")
+
+ assert config.OUTPUT_DIR != Path("")
+ assert config.OUTPUT_DIR != Path.cwd()
+ assert config.OUTPUT_DIR.parent == Path(tempfile.gettempdir())
+
+
+def test_unset_env_uses_per_run_default(monkeypatch):
+ config = _reimport_config(monkeypatch, None)
+
+ assert config.OUTPUT_DIR.name.startswith("book_skill_work-")
+ assert str(os.getpid()) in config.OUTPUT_DIR.name
+
+
+def test_config_restored_for_other_tests(monkeypatch):
+ """Reloading config in this module must not leak into the rest of the suite."""
+ config = _reimport_config(monkeypatch, None)
+
+ assert config.OUTPUT_TEXT.parent == config.OUTPUT_DIR
+ assert config.OUTPUT_META.parent == config.OUTPUT_DIR