"CLI-Anything: Making ALL Software Agent-Native" -- CLI-Hub: https://clianything.cc/
Python
test: add calibre harness smoke validation
yuhaob2c469244635
diff --git a/calibre/agent-harness/FIX_NOTES.md b/calibre/agent-harness/FIX_NOTES.mdnew file mode 100644index 000000000..c353f17ac--- /dev/null+++ b/calibre/agent-harness/FIX_NOTES.md@@ -0,0 +1,60 @@+# FIX_NOTES.md - PR #223 Calibre Harness Validation++## Blocker Status++- No code blocker was identified in the Calibre harness.+- Remaining validation/documentation blocker addressed by adding no-Calibre subprocess smoke coverage and explicit real-backend validation steps.++## No-Calibre Smoke Validation++These commands validate importability, Click entrypoint behavior, and missing-library+error handling without requiring `calibredb`, `ebook-convert`, or `ebook-meta`.++```bash+cd calibre/agent-harness+python -m py_compile \+ cli_anything/calibre/calibre_cli.py \+ cli_anything/calibre/core/*.py \+ cli_anything/calibre/utils/*.py+python -m pytest cli_anything/calibre/tests/test_core.py -v+```++To require the installed console script for smoke validation:++```bash+cd calibre/agent-harness+pip install -e .+CLI_ANYTHING_FORCE_INSTALLED=1 python -m pytest \+ cli_anything/calibre/tests/test_core.py::TestCLISubprocessSmoke -v+```++## Real Calibre Backend Validation++Install Calibre first and confirm all wrapped commands resolve:++```bash+which calibredb+which ebook-convert+which ebook-meta+```++Then run the E2E suite:++```bash+cd calibre/agent-harness+pip install -e .+CLI_ANYTHING_FORCE_INSTALLED=1 python -m pytest \+ cli_anything/calibre/tests/test_full_e2e.py -v -s+```++Expected E2E coverage:++- Temporary Calibre libraries are created for test isolation.+- Generated EPUB fixtures are imported through `calibredb`.+- Metadata changes are round-tripped through the real backend.+- EPUB to TXT/MOBI conversion runs through `ebook-convert`.+- Exported and converted artifacts are checked for existence and nonzero size.++## Remaining Gaps++- Catalog generation, file-level metadata embedding, and real custom-column workflows are documented as E2E gaps in `cli_anything/calibre/tests/TEST.md`.diff --git a/calibre/agent-harness/cli_anything/calibre/README.md b/calibre/agent-harness/cli_anything/calibre/README.mdindex d102a5606..2e84d545e 100644--- a/calibre/agent-harness/cli_anything/calibre/README.md+++ b/calibre/agent-harness/cli_anything/calibre/README.md@@ -177,12 +177,38 @@ Override with the `CALIBRE_LIBRARY` environment variable.```bashcd agent-harness-# Unit tests (no Calibre required)-pytest cli_anything/calibre/tests/test_core.py -v+# Syntax check+python -m py_compile \+ cli_anything/calibre/calibre_cli.py \+ cli_anything/calibre/core/*.py \+ cli_anything/calibre/utils/*.py++# Unit and CLI smoke tests (no Calibre required)+python -m pytest cli_anything/calibre/tests/test_core.py -v++# Installed-command smoke tests (no Calibre required)+pip install -e .+CLI_ANYTHING_FORCE_INSTALLED=1 python -m pytest \+ cli_anything/calibre/tests/test_core.py::TestCLISubprocessSmoke -v# Full E2E tests (Calibre required)-pytest cli_anything/calibre/tests/test_full_e2e.py -v -s+python -m pytest cli_anything/calibre/tests/test_full_e2e.py -v -s-# All tests with installed binary-CLI_ANYTHING_FORCE_INSTALLED=1 pytest cli_anything/calibre/tests/ -v -s+# All tests with installed binary and real Calibre backend+CLI_ANYTHING_FORCE_INSTALLED=1 python -m pytest cli_anything/calibre/tests/ -v -s```++### Real Backend Validation++Before running E2E validation, verify Calibre's commands are on PATH:++```bash+which calibredb+which ebook-convert+which ebook-meta+```++The E2E suite creates temporary Calibre libraries, imports generated EPUB files,+updates metadata with `calibredb`, converts EPUB files with `ebook-convert`, and+checks exported/converted artifacts for real output files. These tests require a+real Calibre installation; `test_core.py` remains the no-Calibre validation path.diff --git a/calibre/agent-harness/cli_anything/calibre/tests/TEST.md b/calibre/agent-harness/cli_anything/calibre/tests/TEST.mdindex 8487e4425..92c4e7a67 100644--- a/calibre/agent-harness/cli_anything/calibre/tests/TEST.md+++ b/calibre/agent-harness/cli_anything/calibre/tests/TEST.md@@ -5,7 +5,13 @@This document covers the test plan and results for `cli-anything-calibre`, a CLI harnesswrapping the real Calibre tools (`calibredb`, `ebook-convert`, `ebook-meta`).-**Hard dependency:** Calibre must be installed (`calibredb` in PATH).+**Unit/smoke dependency:** `test_core.py` does not require Calibre. It includes+subprocess smoke checks for help, version, and missing-library behavior using the+installed `cli-anything-calibre` command when available, or `python -m+cli_anything.calibre` as a development fallback.++**E2E hard dependency:** `test_full_e2e.py` requires Calibre (`calibredb`,+`ebook-convert`, and `ebook-meta` in PATH) for real backend validation.---@@ -13,8 +19,8 @@ wrapping the real Calibre tools (`calibredb`, `ebook-convert`, `ebook-meta`).| File | Tests Planned | Description ||------|--------------|-------------|-| `test_core.py` | 38 | Unit tests: synthetic data, no external deps, no Calibre needed |-| `test_full_e2e.py` | 20 | E2E tests: real Calibre library operations + subprocess CLI tests |+| `test_core.py` | 41 | Unit tests: synthetic data, no external deps, no Calibre needed, plus subprocess smoke |+| `test_full_e2e.py` | 21 | E2E tests: real Calibre library operations + subprocess CLI tests |---@@ -72,13 +78,36 @@ wrapping the real Calibre tools (`calibredb`, `ebook-convert`, `ebook-meta`).| `test_find_ebook_meta_missing` | find_ebook_meta() raises RuntimeError if not in PATH || `test_error_message_contains_install_hint` | Error messages include install instructions |-### `calibre_cli.py` — 3 tests+### `calibre_cli.py` — 6 tests| Test | Description ||------|-------------|| `test_cli_help_exits_zero` | `--help` prints usage and returns 0 || `test_cli_version` | `--version` returns version string || `test_cli_missing_library_error` | Commands without library set print clear error |+| `test_installed_or_module_help_smoke` | Subprocess `--help` smoke test; uses installed entry point if present |+| `test_installed_or_module_version_smoke` | Subprocess `--version` smoke test; uses installed entry point if present |+| `test_missing_library_error_without_calibre` | Subprocess missing-library error works without Calibre installed |++Run no-backend validation:++```bash+cd calibre/agent-harness+python -m py_compile \+ cli_anything/calibre/calibre_cli.py \+ cli_anything/calibre/core/*.py \+ cli_anything/calibre/utils/*.py+python -m pytest cli_anything/calibre/tests/test_core.py -v+```++Installed-command smoke mode:++```bash+cd calibre/agent-harness+pip install -e .+CLI_ANYTHING_FORCE_INSTALLED=1 python -m pytest \+ cli_anything/calibre/tests/test_core.py::TestCLISubprocessSmoke -v+```---@@ -91,14 +120,15 @@ These tests invoke the **real Calibre** tools and verify the output.E2E tests use a temporary Calibre library created with `calibredb add` from a real EPUB.A minimal EPUB is generated programmatically (valid ZIP structure) for reproducibility.-### `TestLibraryOperations` — 5 tests+### `TestLibraryOperations` — 6 tests| Test | Description | Verified ||------|-------------|---------|-| `test_create_library_and_add_epub` | Add EPUB to new library | Book ID returned, book count > 0 |+| fixture setup | Add EPUB to new library | Book ID returned, book count > 0 || `test_list_books` | list_books() returns book entries | ID, title, authors present |+| `test_list_books_custom_fields` | list_books() honors explicit field list | Requested fields present, omitted fields absent || `test_search_books` | search_books() with query | Returns matching IDs |-| `test_show_metadata` | get_metadata() returns parsed OPF | title, authors fields present |+| `test_get_metadata` | get_metadata() returns parsed OPF | title, authors fields present || `test_export_books` | export_books() exports to directory | Files exported, cover.jpg present |### `TestMetadataOperations` — 3 tests@@ -182,7 +212,39 @@ Tests the installed `cli-anything-calibre` command directly via subprocess.## Test Results-Run command:+No-backend validation run:++```bash+cd calibre/agent-harness+python -m py_compile \+ cli_anything/calibre/calibre_cli.py \+ cli_anything/calibre/core/*.py \+ cli_anything/calibre/utils/*.py+python -m pytest cli_anything/calibre/tests/test_core.py -v+```++Current no-backend result:++```text+41 passed in 0.74s+```++Installed-command smoke run:++```bash+cd calibre/agent-harness+pip install -e .+CLI_ANYTHING_FORCE_INSTALLED=1 python -m pytest \+ cli_anything/calibre/tests/test_core.py::TestCLISubprocessSmoke -v+```++Current installed-command smoke result:++```text+3 passed in 0.63s+```++Historical real-backend run:```bashCLI_ANYTHING_FORCE_INSTALLED=1 python -m pytest cli_anything/calibre/tests/ -v --tb=no```@@ -250,14 +312,38 @@ cli_anything/calibre/tests/test_full_e2e.py::TestCLISubprocess::test_version PAS| Metric | Value ||--------|-------|-| Total tests | 50 |-| Passed | 50 |+| Total tests expected now | 62 |+| No-backend tests passed in current run | 41 |+| Installed smoke tests passed in current run | 3 |+| Historical full-suite tests passed | 50 || Failed | 0 |-| Pass rate | 100% |-| Execution time | 15.93s |+| Current no-backend execution time | 0.74s |+| Current installed-smoke execution time | 0.63s |+| Historical full-suite execution time | 15.93s || Calibre version | 7.6 || Subprocess backend | `/home/orgleaf/py-base-venv/bin/cli-anything-calibre` (installed) |+## Real Backend Validation Steps++Use these steps to validate the harness against a real Calibre install:++```bash+cd calibre/agent-harness+pip install -e .+which calibredb+which ebook-convert+which ebook-meta+CLI_ANYTHING_FORCE_INSTALLED=1 python -m pytest \+ cli_anything/calibre/tests/test_full_e2e.py -v -s+```++Expected behavior:++- Tests create temporary Calibre libraries and seed them with generated EPUB files.+- `calibredb` is used for add/list/search/metadata/export operations.+- `ebook-convert` is used for EPUB to TXT/MOBI conversion and output files are checked for existence and nonzero size.+- Subprocess E2E tests require the installed `cli-anything-calibre` entry point when `CLI_ANYTHING_FORCE_INSTALLED=1` is set.+## Coverage Notes- All session persistence operations are tested (save, load, env override, error paths)diff --git a/calibre/agent-harness/cli_anything/calibre/tests/test_core.py b/calibre/agent-harness/cli_anything/calibre/tests/test_core.pyindex 1cc40b1c2..69672669e 100644--- a/calibre/agent-harness/cli_anything/calibre/tests/test_core.py+++ b/calibre/agent-harness/cli_anything/calibre/tests/test_core.py@@ -5,6 +5,8 @@ All tests use synthetic data — no real Calibre installation required.import jsonimport os+import shutil+import subprocessimport sysimport tempfileimport unittest@@ -614,5 +616,59 @@ class TestCLIHelp(unittest.TestCase):)+class TestCLISubprocessSmoke(unittest.TestCase):+ """Subprocess smoke tests that do not require Calibre to be installed."""++ @staticmethod+ def _resolve_cli():+ force = os.environ.get("CLI_ANYTHING_FORCE_INSTALLED", "").strip() == "1"+ installed = shutil.which("cli-anything-calibre")+ if installed:+ return [installed]+ if force:+ raise RuntimeError(+ "cli-anything-calibre not found in PATH. Install with: pip install -e ."+ )+ return [sys.executable, "-m", "cli_anything.calibre"]++ def _run(self, args, *, home=None):+ env = os.environ.copy()+ harness_root = Path(__file__).resolve().parents[3]+ env["PYTHONPATH"] = (+ str(harness_root)+ if not env.get("PYTHONPATH")+ else f"{harness_root}{os.pathsep}{env['PYTHONPATH']}"+ )+ env.pop("CALIBRE_LIBRARY", None)+ if home:+ env["HOME"] = home+ return subprocess.run(+ self._resolve_cli() + args,+ capture_output=True,+ text=True,+ env=env,+ check=False,+ )++ def test_installed_or_module_help_smoke(self):+ result = self._run(["--help"])+ self.assertEqual(result.returncode, 0, result.stderr)+ self.assertIn("Usage:", result.stdout)+ self.assertIn("library", result.stdout)++ def test_installed_or_module_version_smoke(self):+ result = self._run(["--version"])+ self.assertEqual(result.returncode, 0, result.stderr)+ self.assertIn("cli-anything-calibre", result.stdout)++ def test_missing_library_error_without_calibre(self):+ with tempfile.TemporaryDirectory() as tmp:+ result = self._run(["books", "list"], home=tmp)+ combined = result.stdout + result.stderr+ self.assertNotEqual(result.returncode, 0)+ self.assertIn("No Calibre library connected", combined)+ self.assertIn("CALIBRE_LIBRARY", combined)++if __name__ == "__main__":unittest.main()