Self-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.
Python agent-contextagent-memoryagent-pluginsagentic-ragcontext-databasedsh-pluginknowledge-basememorymemory-managementself-evolvingvector-database
Add ov-add-paper skill example (#2548)
Zayn Jarvisc780c8336e95
diff --git a/examples/skills/ov-add-paper/SKILL.md b/examples/skills/ov-add-paper/SKILL.mdnew file mode 100644index 000000000..625a29ea8--- /dev/null+++ b/examples/skills/ov-add-paper/SKILL.md@@ -0,0 +1,91 @@+---+name: ov-add-paper+description: "Load when the user asks to add, import, compile, or ingest a research paper/PDF into OpenViking, especially when they mention ov-add-paper, ARA, claims, evidence, figures, tables, or paper-to-OV knowledge resources."+compatibility: OpenViking CLI configured at `~/.openviking/ovcli.conf`+version: 0.1.1+last_updated: 2026-06-10+allowed-tools:+ - Read+ - Write+ - Edit+ - Bash+ - Glob+ - Grep+tags:+ - openviking+ - paper-ingestion+ - research+ - ara+---++# ov-add-paper++## Goal++Turn a research paper into an OpenViking-ready structured resource, then complete ingestion with `ov add-resource`. The job is not done until the generated artifact directory has been validated and submitted to OpenViking.++## Inputs++- Required: a paper source, usually a local PDF path or paper URL.+- Optional: output directory, OpenViking target URI, domain notes, related repo/source files, and whether to wait for OV processing.+- If the paper source is missing or inaccessible, ask for it before starting.++## Workflow++1. Read the paper completely, including appendices and all numbered figures/tables.+2. Compile an ARA-style artifact directory using `references/ara-compiler-profile.md`.+3. Validate the artifact with `scripts/validate_ara.py`.+4. Fix validation failures unless the user explicitly accepts them.+5. Ingest the validated artifact directory with `ov add-resource` directly.+6. Confirm the target with `ov stat`/`ov tree` and return the artifact path, OV target/root URI, validation result, ingest result, and any unresolved gaps.++## Output Contract++The generated artifact must include:++- `PAPER.md`+- `logic/problem.md`, `logic/claims.md`, `logic/concepts.md`, `logic/experiments.md`, `logic/related_work.md`+- `logic/solution/constraints.md`+- `src/environment.md`+- `trace/exploration_tree.yaml`+- `evidence/README.md`+- Markdown plus PNG evidence files for every filed numbered table and figure++The final response must include the `ov add-resource` command result, or the recovery checks proving the target landed despite a broken `--wait`, or the exact blocker that prevented ingestion.++## Verification++Run from this skill directory, or use an absolute path to the validator:++```bash+python3 scripts/validate_ara.py <artifact-dir>+```++Then ingest with the OV CLI:++```bash+ov add-resource <artifact-dir> --to viking://resources/papers/<slug> --wait --timeout 300+```++Before ingest, use `ov -o json stat <target-uri>` to check whether the target already exists. After ingest, verify with `ov -o json stat <target-uri>` and `ov tree <target-uri>`.++## Permissions++- Writing a new OpenViking resource is allowed when the user asked to add or ingest the paper.+- Ask before intentionally reusing a target URI that may overwrite or replace an existing resource.+- `--skip-validation` may be used only when the user explicitly accepts the listed validation errors.++## Boundaries++- Do not use `ov add-skill`; this skill creates paper resources, not OV skills.+- Do not silently skip `ov add-resource`; if ingestion fails, report the command, error, and recovery path.+- If `ov add-resource --wait` exits with a connection error after creating the target, do not immediately retry the same URI. Run `ov stat`, `ov wait --timeout <seconds>`, `ov observer queue`, and `ov tree` to determine whether ingestion completed.+- Do not invent claims, evidence, source refs, code, numbers, or research history.+- Do not overwrite an explicit existing OV target unless the user asked for that target.+- Mark unsupported or unreadable content as unavailable instead of filling it in.++## Runtime Resources++- Load `references/ara-compiler-profile.md` before compiling the artifact.+- Load `references/openviking-ingest.md` before running OV ingestion or debugging an ingestion failure.+- Use `scripts/validate_ara.py` for deterministic checks.diff --git a/examples/skills/ov-add-paper/references/ara-compiler-profile.md b/examples/skills/ov-add-paper/references/ara-compiler-profile.mdnew file mode 100644index 000000000..0f6ec1579--- /dev/null+++ b/examples/skills/ov-add-paper/references/ara-compiler-profile.md@@ -0,0 +1,127 @@+# ARA Compiler Profile for ov-add-paper++This profile adapts the Agent-Native Research Artifact compiler pattern for OpenViking ingestion. It keeps the ARA compiler's epistemic structure, but the final deliverable is an OV resource directory.++Source reference: https://github.com/AmberLJC/Agent-Native-Research-Artifact++## Compilation Principles++- Treat the paper as evidence first, narrative second.+- Read the entire paper, including appendices and supplementary sections available in the provided source.+- Preserve raw evidence before synthesis.+- Separate exact source facts, visual estimates, model inference, and unavailable information.+- Strong claims require direct evidence; use weaker wording when evidence is narrower.+- Every source reference should point to an actual page, section, figure, table, equation, or provided repo file.++## Required ARA Layout++```text+PAPER.md+logic/+ problem.md+ claims.md+ concepts.md+ experiments.md+ related_work.md+ solution/+ constraints.md+src/+ environment.md+trace/+ exploration_tree.yaml+evidence/+ README.md+ figures/+ tables/+```++Additional files are allowed only when the paper warrants them, such as `logic/solution/algorithm.md`, `logic/solution/architecture.md`, `data/dataset.md`, `src/configs/`, or `evidence/proofs/`.++## Evidence Pass++Build an evidence ledger before writing claims:++1. Enumerate every numbered `Figure N` and `Table N` in the paper, in order.+2. For each filed object, save both:+ - a cropped or full-page PNG preserving the source visual+ - a Markdown transcription or structured description+3. If an object cannot be filed, account for it in `evidence/README.md` with the reason.+4. Keep raw source evidence separate from derived subsets.++Figure Markdown should include:++- Source+- Caption+- Figure type: `quantitative_plot`, `diagram`, `qualitative_sample`, or `mixed`+- Extraction method: `exact_from_labels`, `digitized_estimate`, or `visual_description`+- Reading confidence+- Supports+- Transcription or visual description++Table Markdown should include:++- Source+- Caption+- Supports+- faithful table transcription++## Cognitive Layer++`logic/problem.md` should capture observations, gaps, key insight, and assumptions.++`logic/claims.md` should use `C01`, `C02`, ... headings. Each claim needs:++- Statement+- Status+- Falsification criteria+- Proof, referencing experiment IDs such as `E01`+- Evidence basis+- Interpretation when useful+- Dependencies+- Tags++`logic/experiments.md` should use `E01`, `E02`, ... headings. It describes verification plans, not exact result numbers. Exact numbers belong in evidence files.++`logic/concepts.md` should define paper-specific concepts. Do not pad with generic terms.++`logic/related_work.md` should describe typed dependencies such as imports, extends, baseline, bounds, or refutes.++`logic/solution/constraints.md` is always required and should state boundary conditions, assumptions, and limitations.++## Artifact Layer++`src/environment.md` is always required. Other `src/` files should capture concrete artifacts only when they exist in the paper or provided source material.++Do not manufacture code stubs from prose-only methods. If code is included, mark whether it is transcribed from source or reconstructed from explicitly printed pseudocode/equations.++## Exploration Trace++`trace/exploration_tree.yaml` records the research DAG:++- central questions+- experiments+- decisions+- dead ends+- pivots+- support level: `explicit` or `inferred`++Do not invent failures or decisions. If the paper hides the process, use a smaller trace and mark reconstructed nodes as inferred.++## Coverage Loop++Before validation, do up to three coverage passes:++1. Re-read source headings, figures, tables, equations, appendix sections, and references.+2. Compare missing items against the artifact.+3. Patch omissions, weak claim wording, missing evidence links, or unresolved source refs.+4. Stop early if a pass finds no material gaps.++## Done State++The ARA compile phase is complete only when:++- mandatory files exist and are non-empty+- figure/table evidence has Markdown plus PNG when filed+- claims and experiments cross-reference correctly+- `PAPER.md` has a useful layer index+- the artifact passes `scripts/validate_ara.py`diff --git a/examples/skills/ov-add-paper/references/openviking-ingest.md b/examples/skills/ov-add-paper/references/openviking-ingest.mdnew file mode 100644index 000000000..9094a3a17--- /dev/null+++ b/examples/skills/ov-add-paper/references/openviking-ingest.md@@ -0,0 +1,71 @@+# OpenViking Ingest Contract++`ov-add-paper` must end by importing the generated paper artifact into OpenViking with `ov add-resource`.++## Recommended CLI Flow++```bash+python3 scripts/validate_ara.py <artifact-dir>+ov -o json stat viking://resources/papers/<slug>+ov add-resource <artifact-dir> --to viking://resources/papers/<slug> --wait+ov -o json stat viking://resources/papers/<slug>+ov tree viking://resources/papers/<slug>+```++If the user did not provide a target URI, derive a stable slug from the paper title, arXiv ID, DOI, or file stem.++Use `--timeout 300` or a larger value for medium-sized papers:++```bash+ov add-resource <artifact-dir> --to viking://resources/papers/<slug> --wait --timeout 300+```++The `stat` preflight should return NOT_FOUND for a new target. If it succeeds, ask before overwriting or choose a different target.++## Required Preconditions++- `ov` CLI is installed and configured.+- `~/.openviking/ovcli.conf` or equivalent environment config is present.+- The artifact directory exists locally.+- Validation passes or the user explicitly accepts the listed validation errors.++## Directory Upload Pitfall++The CLI zips a local directory before upload. `--include`/`--exclude` are request parameters and may not reduce the client-side ZIP payload. If directory upload repeatedly ends with `Could not reach OpenViking` while `ov health` succeeds and single-file imports work, suspect upload timeout or an unstable large directory payload.++Mitigations:++- Keep the OV upload artifact lean: required ARA Markdown/YAML plus filed figure/table PNG evidence.+- Do not duplicate large raw PDFs, source tarballs, or extraction scratch files inside the upload directory unless the user explicitly needs them in OV.+- Keep raw source files in a local `source/` or full `artifact/` working copy and record their paths in `src/environment.md`.+- Optimize evidence PNGs while preserving readability, for example by rendering full pages at 1.5x instead of 2x when that is still legible.+- If needed, create a separate upload copy such as `<artifact-dir>-ov/` rather than mutating the full local artifact.+- `upload.mode = "shared"` can help distributed deployments, but it does not make an oversized directory payload smaller.++If `ov add-resource --wait` exits with a connection error after the target appears, treat it as an interrupted wait, not necessarily a failed ingest. Check:++```bash+ov -o json stat viking://resources/papers/<slug>+ov wait --timeout 300+ov observer queue+ov tree viking://resources/papers/<slug>+```++If `stat` shows `isLocked=false`, `count` is nonzero, the queue is empty, and `tree`/`read` can access content, report the ingest as completed with a note that the original `--wait` connection broke. If the target is not visible, shrink the upload payload and retry.++## Reporting++Report:++- artifact directory path+- target URI or returned root URI+- whether `--wait` completed or required recovery checks+- validation summary+- exact error and recovery path if ingestion failed++## Boundaries++- Do not run `ov add-skill` for this workflow.+- Do not delete or overwrite local artifacts as cleanup.+- Do not hide asynchronous ingestion status. If `--wait` is not used, say processing continues in the background.+- Do not invent a successful OV URI when the CLI fails.diff --git a/examples/skills/ov-add-paper/scripts/validate_ara.py b/examples/skills/ov-add-paper/scripts/validate_ara.pynew file mode 100755index 000000000..8408ec7a2--- /dev/null+++ b/examples/skills/ov-add-paper/scripts/validate_ara.py@@ -0,0 +1,218 @@+#!/usr/bin/env python3+"""Validate an ARA-style paper artifact before OpenViking ingestion."""++from __future__ import annotations++import argparse+import json+import re+import sys+from pathlib import Path+++REQUIRED_FILES = [+ "PAPER.md",+ "logic/problem.md",+ "logic/claims.md",+ "logic/concepts.md",+ "logic/experiments.md",+ "logic/related_work.md",+ "logic/solution/constraints.md",+ "src/environment.md",+ "trace/exploration_tree.yaml",+ "evidence/README.md",+]++PAPER_FRONTMATTER_FIELDS = ["title", "authors", "year"]+CLAIM_FIELDS = [+ "Statement",+ "Status",+ "Falsification criteria",+ "Proof",+ "Evidence basis",+]+EXPERIMENT_FIELDS = ["Verifies", "Setup", "Procedure", "Metrics", "Expected outcome"]+++def read_text(path: Path) -> str:+ try:+ return path.read_text(encoding="utf-8")+ except UnicodeDecodeError:+ return path.read_text(encoding="utf-8", errors="replace")+++def has_field(block: str, field: str) -> bool:+ pattern = rf"(?im)^\s*[-*]\s+(?:\*\*)?{re.escape(field)}(?:\*\*)?\s*:"+ return bool(re.search(pattern, block))+++def split_heading_blocks(text: str, prefix: str) -> dict[str, str]:+ heading = re.compile(rf"(?m)^##\s+({prefix}\d{{2,}})\b.*$")+ matches = list(heading.finditer(text))+ blocks: dict[str, str] = {}+ for index, match in enumerate(matches):+ start = match.start()+ end = matches[index + 1].start() if index + 1 < len(matches) else len(text)+ blocks[match.group(1)] = text[start:end]+ return blocks+++def extract_refs(text: str, prefix: str) -> set[str]:+ return set(re.findall(rf"\b{prefix}\d{{2,}}\b", text))+++def validate(root: Path) -> dict:+ errors: list[str] = []+ warnings: list[str] = []++ if not root.exists():+ return {+ "ok": False,+ "errors": [f"artifact directory does not exist: {root}"],+ "warnings": [],+ "summary": {},+ }+ if not root.is_dir():+ return {+ "ok": False,+ "errors": [f"artifact path is not a directory: {root}"],+ "warnings": [],+ "summary": {},+ }++ for rel in REQUIRED_FILES:+ path = root / rel+ if not path.exists():+ errors.append(f"missing required file: {rel}")+ elif path.stat().st_size == 0:+ errors.append(f"required file is empty: {rel}")++ paper_path = root / "PAPER.md"+ if paper_path.exists():+ paper = read_text(paper_path)+ frontmatter = re.match(r"(?s)^---\n(.*?)\n---\n", paper)+ if not frontmatter:+ errors.append("PAPER.md missing YAML frontmatter")+ else:+ fm = frontmatter.group(1)+ for field in PAPER_FRONTMATTER_FIELDS:+ if not re.search(rf"(?m)^{re.escape(field)}\s*:", fm):+ errors.append(f"PAPER.md frontmatter missing field: {field}")+ if "Layer Index" not in paper:+ errors.append("PAPER.md missing Layer Index")++ claim_blocks: dict[str, str] = {}+ claims_path = root / "logic/claims.md"+ if claims_path.exists():+ claim_blocks = split_heading_blocks(read_text(claims_path), "C")+ if not claim_blocks:+ errors.append("logic/claims.md has no C## claim blocks")+ for claim_id, block in claim_blocks.items():+ for field in CLAIM_FIELDS:+ if not has_field(block, field):+ errors.append(f"{claim_id} missing field: {field}")++ experiment_blocks: dict[str, str] = {}+ experiments_path = root / "logic/experiments.md"+ if experiments_path.exists():+ experiment_blocks = split_heading_blocks(read_text(experiments_path), "E")+ if not experiment_blocks:+ errors.append("logic/experiments.md has no E## experiment blocks")+ for exp_id, block in experiment_blocks.items():+ for field in EXPERIMENT_FIELDS:+ if not has_field(block, field):+ errors.append(f"{exp_id} missing field: {field}")++ claim_ids = set(claim_blocks)+ experiment_ids = set(experiment_blocks)++ for claim_id, block in claim_blocks.items():+ proof_match = re.search(+ r"(?im)^\s*[-*]\s+(?:\*\*)?Proof(?:\*\*)?\s*:\s*(.+)$", block+ )+ if proof_match:+ refs = extract_refs(proof_match.group(1), "E")+ if not refs:+ errors.append(f"{claim_id} Proof does not reference any E## experiment")+ for ref in refs:+ if ref not in experiment_ids:+ errors.append(f"{claim_id} Proof references missing experiment: {ref}")++ for exp_id, block in experiment_blocks.items():+ verifies_match = re.search(+ r"(?im)^\s*[-*]\s+(?:\*\*)?Verifies(?:\*\*)?\s*:\s*(.+)$", block+ )+ if verifies_match:+ refs = extract_refs(verifies_match.group(1), "C")+ if not refs:+ errors.append(f"{exp_id} Verifies does not reference any C## claim")+ for ref in refs:+ if ref not in claim_ids:+ errors.append(f"{exp_id} Verifies references missing claim: {ref}")++ evidence_counts = {"figure_md": 0, "figure_png": 0, "table_md": 0, "table_png": 0}+ for kind, dirname in [("figure", "evidence/figures"), ("table", "evidence/tables")]:+ directory = root / dirname+ if not directory.exists():+ warnings.append(f"optional evidence directory missing: {dirname}")+ continue+ md_files = sorted(directory.glob("*.md"))+ png_files = sorted(directory.glob("*.png"))+ evidence_counts[f"{kind}_md"] = len(md_files)+ evidence_counts[f"{kind}_png"] = len(png_files)+ for md in md_files:+ content = read_text(md)+ rel = md.relative_to(root)+ if not re.search(+ r"(?im)^\s*[-*]?\s*(?:\*\*)?Source(?:\*\*)?\s*:", content+ ):+ errors.append(f"{rel} missing Source field")+ if kind == "figure":+ for field in ["Figure type", "Extraction method", "Reading confidence"]:+ if not re.search(rf"(?im){re.escape(field)}\s*:", content):+ warnings.append(f"{rel} missing recommended field: {field}")+ sibling_png = md.with_suffix(".png")+ if not sibling_png.exists():+ errors.append(f"{rel} missing sibling PNG: {sibling_png.name}")++ trace_path = root / "trace/exploration_tree.yaml"+ if trace_path.exists():+ trace = read_text(trace_path)+ if "support_level" not in trace:+ errors.append("trace/exploration_tree.yaml missing support_level entries")+ if not re.search(r"(?m)^\s*-\s*id\s*:", trace) and not re.search(+ r"(?m)^\s*id\s*:", trace+ ):+ warnings.append("trace/exploration_tree.yaml has no obvious node id entries")++ summary = {+ "required_files": len(REQUIRED_FILES),+ "claims": len(claim_blocks),+ "experiments": len(experiment_blocks),+ **evidence_counts,+ }+ return {"ok": not errors, "errors": errors, "warnings": warnings, "summary": summary}+++def main() -> int:+ parser = argparse.ArgumentParser(description=__doc__)+ parser.add_argument("artifact_dir", type=Path)+ parser.add_argument("--json", action="store_true", help="print machine-readable JSON")+ args = parser.parse_args()++ result = validate(args.artifact_dir)+ if args.json:+ print(json.dumps(result, indent=2, ensure_ascii=False))+ else:+ status = "PASS" if result["ok"] else "FAIL"+ print(f"ARA validation: {status}")+ print(json.dumps(result["summary"], indent=2, ensure_ascii=False))+ for warning in result["warnings"]:+ print(f"WARN: {warning}")+ for error in result["errors"]:+ print(f"ERROR: {error}")+ return 0 if result["ok"] else 1+++if __name__ == "__main__":+ sys.exit(main())