Graph-Native Infrastructure for Context and Accountable AI Systems
Python agent-memoryaiai-governanceai-infrastructureartificial-intelligencecontext-engineeringcontext-graphsdata-engineeringdecision-intelligencedeveloper-toolsexplainable-aigenerative-aigraph-ragknowledge-graphllmontologyprovenancepythonreasoningsemantic-search
docs(readme): redesign for better traction and narrative clarity (#559)
Mohd Kaif9686508434bb
diff --git a/README.md b/README.mdindex 360e3d7a..6d2fde8e 100644--- a/README.md+++ b/README.md@@ -2,55 +2,32 @@<img src="Semantica Logo.png" alt="Semantica Logo" width="420"/>-# ๐ง Semantica+# Semantica-**A Framework for Building Context Graphs and Decision Intelligence Layers for AI**+The Accountability and Context Layer for AI โ Context Graphs ยท Decision Intelligence ยท Full Provenance-๐ **[Website](https://getsemantica.ai/)** | ๐ **[Documentation](https://docs.getsemantica.ai/)**--[](https://www.python.org/)-[](https://opensource.org/licenses/MIT)+[](https://getsemantica.ai/)+[](https://docs.getsemantica.ai/)[](https://pypi.org/project/semantica/)[](https://github.com/Hawksight-AI/semantica/releases/tag/v0.5.0)[](https://pepy.tech/project/semantica)+[](https://www.python.org/)+[](https://opensource.org/licenses/MIT)[](https://github.com/Hawksight-AI/semantica/actions)[](https://discord.gg/sV34vps5hH)[](https://x.com/BuildSemantica)[](https://openclaw.ai)-### โญ Give us a Star ยท ๐ด Fork us ยท ๐ฌ Join our Discord ยท ๐ฆ Follow on X+**[Website](https://getsemantica.ai/)** ยท **[Docs](https://docs.getsemantica.ai/)** ยท **[Discord](https://discord.gg/sV34vps5hH)** ยท **[Changelog](CHANGELOG.md)**-> **Transform Chaos into Intelligence. Build AI systems with context graphs, decision tracking, and advanced knowledge engineering that are explainable, traceable, and trustworthy โ not black boxes.**+โญ **Star us if this solves your problem** ยท ๐ด Fork us ยท ๐ฌ [Join our Discord](https://discord.gg/sV34vps5hH) ยท ๐ฆ [Follow on X](https://x.com/BuildSemantica)+> Most AI agents act without a trail. Semantica adds the layer your stack is missing: structured context graphs, auditable decision records, and full provenance from every output back to its source โ so your AI isn't just powerful, it's accountable.๐ [๐บ๐ธ English](https://readme-i18n.com/Hawksight-AI/semantica?lang=en) ยท [๐ฉ๐ช Deutsch](https://readme-i18n.com/Hawksight-AI/semantica?lang=de) ยท [๐ซ๐ท Franรงais](https://readme-i18n.com/Hawksight-AI/semantica?lang=fr) ยท [๐ช๐ธ Espaรฑol](https://readme-i18n.com/Hawksight-AI/semantica?lang=es) ยท [๐ฎ๐น Italiano](https://readme-i18n.com/Hawksight-AI/semantica?lang=it) ยท [๐ต๐น Portuguรชs](https://readme-i18n.com/Hawksight-AI/semantica?lang=pt) ยท [๐ธ๐ฆ ุงูุนุฑุจูุฉ](https://readme-i18n.com/Hawksight-AI/semantica?lang=ar) ยท [๐ต๐ฐ ุงุฑุฏู](https://readme-i18n.com/Hawksight-AI/semantica?lang=ur) ยท [๐ฎ๐ณ เคนเคฟเคจเฅเคฆเฅ](https://readme-i18n.com/Hawksight-AI/semantica?lang=hi) ยท [๐จ๐ณ ไธญๆ](https://readme-i18n.com/Hawksight-AI/semantica?lang=zh) ยท [๐ฏ๐ต ๆฅๆฌ่ช](https://readme-i18n.com/Hawksight-AI/semantica?lang=ja) ยท [๐ฐ๐ท ํ๊ตญ์ด](https://readme-i18n.com/Hawksight-AI/semantica?lang=ko)</div>--## ๐ What's New in v0.5.0--**Released May 11, 2026** ยท [Full Release Notes](RELEASE_NOTES.md) ยท [Changelog](CHANGELOG.md)--### Distance Intelligence Framework-- **10x+ embedding cache performance** โ per-session revision-based caching with thread-safe invalidation-- **Semantic Neighborhood Search** โ context-aware similarity with proximity metrics-- **Distance Matrix API** โ NรN semantic distance calculations with caching-- **Advanced UI** โ ego mode, overlays, heatmap, and path inspector--### Complete Ontology Hub Suite-- **Alignments Tab** โ cross-ontology alignment authoring with ML-powered confidence scoring-- **Health Dashboard** โ quality scoring across 5 dimensions with actionable issue tracking-- **SHACL Studio** โ interactive shape authoring with Monaco editor-- **Visual Editor** โ drag-and-drop ontology construction without hand-coding-- **16 new backend endpoints** โ full CRUD and analysis for ontology management--### More in This Release-- **Parquet Ingestion** โ PyArrow backend with column selection and partition support-- **O(log n) Indexed Search** โ 118k nodes: 24ms โ 0.004ms-- **12 Security Vulnerabilities Fixed** โ eval injection, XXE, SQL injection, SSRF, and more-- **Windows & cross-platform fixes** โ Unicode console, pipx install, optional dependency cleanup----## The Problem@@ -58,24 +35,22 @@AI agents today are powerful but not trustworthy:- โ **No memory structure** โ agents store embeddings, not meaning. There's no way to ask *why* something was recalled.-- โ **No decision trail** โ agents make decisions continuously but record nothing. When something breaks, there's no history to audit.+- โ **No decision trail** โ agents act continuously but record nothing. When something breaks, there's no history to audit.- โ **No provenance** โ outputs can't be traced back to source facts. In regulated industries, this is a hard compliance blocker.- โ **No reasoning transparency** โ black-box answers with zero explanation of how a conclusion was reached.- โ **No conflict detection** โ contradictory facts silently coexist in vector stores, producing unpredictable outputs.-These aren't edge cases. They're the reason AI can't be deployed in healthcare, finance, legal, and government without custom guardrails built from scratch every time.-## The SolutionSemantica is the **context and intelligence layer** you add on top of your existing AI stack:-- โ **Context Graphs** โ a structured, queryable graph of everything your agent knows, decides, and reasons about.-- โ **Decision Intelligence** โ every decision is tracked as a first-class object with causal links, precedent search, and impact analysis.-- โ **Full Provenance** โ every fact links back to its source. W3C PROV-O compliant. No more mystery answers.-- โ **Reasoning Engines** โ forward chaining, Rete networks, deductive, abductive, and SPARQL. Explainable paths, not black boxes.-- โ **Quality & Deduplication** โ conflict detection, entity resolution, and pipeline validation built in.+- โ **Context Graphs** โ structured, queryable graph of everything your agent knows, decides, and reasons about+- โ **Decision Intelligence** โ every decision tracked as a first-class object with causal links, precedent search, and impact analysis+- โ **Full Provenance** โ every fact links back to its source. W3C PROV-O compliant.+- โ **Reasoning Engines** โ forward chaining, Rete, deductive, abductive, SPARQL. Explainable paths, not black boxes.+- โ **Quality & Deduplication** โ conflict detection, entity resolution, and pipeline validation built in-> Works alongside **Agno** and any LLM โ Semantica is the **accountability layer** on top, not a replacement. LangChain, LangGraph, CrewAI, and more coming soon.+> Works alongside **Agno** and any LLM. LangChain, LangGraph, CrewAI, and more coming soon.```bashpip install semantica@@ -83,13 +58,115 @@ pip install semantica---+## ๐ Quick Start++```python+from semantica.context import ContextGraph++graph = ContextGraph(advanced_analytics=True)++# Every decision is a first-class, queryable object+loan_id = graph.record_decision(+ category="loan_approval",+ scenario="Mortgage โ 780 credit score, 28% DTI",+ reasoning="Strong credit history, stable 8-year income, low DTI",+ outcome="approved",+ confidence=0.95,+)+rate_id = graph.record_decision(+ category="interest_rate",+ scenario="Set rate for approved mortgage",+ outcome="rate_set_6.2pct",+ confidence=0.98,+)++# Build an auditable causal chain+graph.add_causal_relationship(loan_id, rate_id, relationship_type="enables")++# Answer "why did this happen?" instantly+chain = graph.trace_decision_chain(loan_id)+similar = graph.find_similar_decisions("mortgage approval", max_results=5)+impact = graph.analyze_decision_impact(loan_id)+compliance = graph.check_decision_rules({"category": "loan_approval", "confidence": 0.95})+```++---++## ๐ What's New in v0.5.0++**Released May 11, 2026** ยท [Full Release Notes](RELEASE_NOTES.md) ยท [Changelog](CHANGELOG.md)++### ๐ Distance Intelligence++- **10x+ embedding cache** โ per-session revision-based caching with thread-safe invalidation+- **Distance Matrix API** โ NรN semantic distance calculations (upper-triangle mirrored, capped at 200 nodes)+- **Semantic Neighborhood Search** โ `get_neighbors()` blends graph proximity with semantic score+- **5 new API endpoints** โ `/distance-matrix`, `/semantic-neighborhood`, `/causal-distance`, `/temporal/distance-history`, `/export/distance-enriched`+- **Explorer UI** โ Ego Mode with BFS depth-of-field fading (depth slider 1โ8), Structural/Semantic overlays, Heatmap (greenโred by hop), Path inspector with distance band chips+- **Bidirectional path finding** โ `directed=false` on BFS and Dijkstra; `PathResponse` gains `hop_count` and `distance_band` (direct/near/mid-range/distant)++### ๐ท Complete Ontology Hub Suite++- **Alignments Tab** โ cross-ontology alignment authoring; ML confidence scoring (0.4รlabel + 0.6รTF-IDF); one-click accept+- **Health Dashboard** โ 5-dimension quality scoring (Completeness, Consistency, SHACL, Alignment, Documentation); downloadable JSON report+- **SHACL Studio** โ interactive shape authoring with Monaco editor and custom Turtle syntax highlighting+- **Visual Ontology Editor** โ drag-and-drop canvas; context menus for rename, add super/subclass, SKOS metadata; edits staged as diffs โ nothing commits until published+- **Versions & Proposals Tab** โ version timeline, proposal review, SHACL pre-validation, side-by-side diff+- **Ontology Registry** โ full CRUD with status/format badges, live search, filter pills (All/OWL/SKOS/Internal/External)+- **Ontology Loader** โ URL import with preview, file upload (`.ttl/.rdf/.owl/.nt/.jsonld/.n3`), create from scratch/data/text+- **Entity Search Panel** โ 320 ms debounced search across all loaded ontologies with type filters+- **SKOS Vocabulary Manager** โ hierarchical concept browser with recursive tree and full SKOS annotation detail+- **16 new backend endpoints** under `/api/ontology`++### ๐ฆ More in This Release++- **Parquet Ingestion** โ `ParquetIngestor` with PyArrow; partitioned directory, selective columns, Hive-style partition discovery ยท `pip install semantica[ingest-parquet]`+- **O(log n) Indexed Search** โ inverted index with exact/token/prefix tiers; 118k nodes: 24ms โ 0.004ms+- **DuplicateDetector result limiting** โ `max_results`, `top_k_per_entity`, `min_similarity`, `sort_by` with construction-time validation+- **DeepSeek via OpenAI SDK** โ `OpenAIProvider` rewritten via `openai.OpenAI(base_url=...)` replacing the defunct `deepseek` package++### ๐ Security & Fixes++- **12 vulnerabilities fixed** โ eval injection (CWE-95), pickle deserialization (CWE-502), SQL injection (CWE-89), XXE (CWE-611), SSRF, prompt injection (CWE-1336), ReDoS (CWE-1333), path traversal (CWE-22)+- **Windows** โ `semantica[all]` no longer pulls `faiss-gpu`; `UnicodeEncodeError` on cp1252 consoles fixed+- **Circular import** in `semantic_extract` fixed; `TripleExtractor` alias added for backward compatibility+- **Lazy-load ingest backends** โ core imports no longer fail when optional packages are absent++---++## ๐ Previous Releases++### v0.4.0 โ Temporal Intelligence & Ontology++- **Temporal GraphRAG** โ retrieve knowledge as it existed at any past point; zero LLM calls+- **Allen Interval Algebra** โ 13 deterministic interval relations; gap detection, coverage, cycle analysis+- **Point-in-time Query Engine** โ consistent graph snapshots at any timestamp with a built-in consistency validator+- **TemporalNormalizer** โ converts ISO 8601, relative phrases ("Q1 2024"), and 13 domain maps to UTC; zero LLM calls+- **Bi-temporal Provenance** โ every record stamped with transaction time; OWL-Time RDF export+- **SKOS Vocabulary Management** โ add concepts with labels, hierarchy, definitions; SPARQL-backed search; REST API+- **SHACL Constraints** โ auto-derive data contract shapes from any ontology; three strictness tiers; CI-ready validation+- **ContextGraph pagination** โ O(N) โ O(limit); Ollama remote support; API key logging removed++### v0.3.0 โ First Stable Release++First `Production/Stable` release on PyPI โ the foundation everything builds on.++- **Context Graphs** ยท **Decision Intelligence** ยท **KG Algorithms** (PageRank, Louvain, Node2Vec, link prediction)+- **Deduplication v2** โ 63.6% faster candidate generation; semantic dedup 6.98x faster+- **Delta Processing** โ SPARQL-based incremental diff, `delta_mode` pipelines, snapshot versioning+- **Export** โ Parquet (Spark/BigQuery/Databricks ready), ArangoDB AQL, RDF format aliases+- **Graph Backends** โ Apache AGE, AWS Neptune, FalkorDB, PgVector++โ [Full changelog](CHANGELOG.md) ยท [Release notes](RELEASE_NOTES.md)++---+## ๐ Works With Every AI Tool-Semantica ships **native plugin bundles** for Claude Code, Cursor, and Codex, an **MCP server** (`python -m semantica.mcp_server`) for Windsurf, Cline, Continue, VS Code, Claude Desktop, and OpenClaw, and a **REST API** (FastAPI, port 8000) for any other tool.+Semantica ships **native plugin bundles** for Claude Code, Cursor, and Codex, an **MCP server** for Windsurf, Cline, Continue, VS Code, Claude Desktop, and OpenClaw, and a **REST API** (109 endpoints, FastAPI, port 8000) for any other tool.<table>-<!-- โโ Native Plugin Bundle โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ --><tr><th colspan="3" align="left">๐ Native Plugin Bundle</th><th colspan="5" align="left">โก MCP Server + Plugin</th>@@ -137,7 +214,6 @@ Semantica ships **native plugin bundles** for Claude Code, Cursor, and Codex, an</td></tr>-<!-- โโ MCP Server only ยท REST API โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ --><tr><th colspan="1" align="left">โ๏ธ MCP Server</th><th colspan="7" align="left">๐ REST API</th>@@ -185,13 +261,12 @@ Semantica ships **native plugin bundles** for Claude Code, Cursor, and Codex, an</td></tr>-<!-- โโ Any tool via REST โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ --><tr><th colspan="8" align="left">๐ง Any Tool</th></tr><tr><td align="center" colspan="8">-<img src="https://img.shields.io/badge/109-endpoints-1f6feb?style=flat-square" alt="REST API" width="48" /><br/>+<img src="https://img.shields.io/badge/109-endpoints-1f6feb?style=flat-square" alt="REST API" /><br/><strong>Any agent</strong><br/><sub>109 REST endpoints ยท FastAPI ยท port 8000</sub></td>@@ -201,8 +276,6 @@ Semantica ships **native plugin bundles** for Claude Code, Cursor, and Codex, an### Agentic Frameworks-Semantica integrates with **Agno** today. Coming soon: LangChain, LangGraph, CrewAI, LlamaIndex, AutoGen, OpenAI Agents SDK, Google ADK, and more.-<table><tr><th colspan="8" align="left">โ Supported</th>@@ -256,21 +329,19 @@ Semantica integrates with **Agno** today. Coming soon: LangChain, LangGraph, Cre</tr></table>-> **Agno โ First-Class Integration** ยท `pip install semantica[agno]`->-> Five integration modules live in [`integrations/agno/`](integrations/agno/):->-> | Module | Class | What it does |-> |---|---|---|-> | `context_store.py` | `AgnoContextStore` | Graph-backed agent memory โ store and retrieve structured context |-> | `knowledge_graph.py` | `AgnoKnowledgeGraph` | Implements Agno's `AgentKnowledge` protocol; full extraction pipeline |-> | `decision_kit.py` | `AgnoDecisionKit` | 6 decision-intelligence tools for Agno agents |-> | `kg_toolkit.py` | `AgnoKGToolkit` | 7 KG pipeline tools (build, query, enrich, export) |-> | `shared_context.py` | `AgnoSharedContext` | Shared context graph for multi-agent team coordination |+### Agno โ First-Class Integration ยท `pip install semantica[agno]`-### Plugin Bundles (Claude Code ยท Cursor ยท Codex)+Five integration modules in [`integrations/agno/`](integrations/agno/):-Native plugin bundles live under [`plugins/`](plugins/). Each directory contains a `plugin.json`, `marketplace.json`, and `README.md`.+| Class | What it does |+|---|---|+| `AgnoContextStore` | Graph-backed agent memory |+| `AgnoKnowledgeGraph` | Implements Agno's `AgentKnowledge` protocol; full extraction pipeline |+| `AgnoDecisionKit` | 6 decision-intelligence tools for Agno agents |+| `AgnoKGToolkit` | 7 KG pipeline tools (build, query, enrich, export) |+| `AgnoSharedContext` | Shared context graph for multi-agent team coordination |++### Plugin Bundles| Bundle | Directory | Tools ||---|---|---|@@ -283,441 +354,87 @@ Native plugin bundles live under [`plugins/`](plugins/). Each directory contains| VS Code | [`plugins/.vscode-plugin/`](plugins/.vscode-plugin/) | 17 skills ยท 3 agents ยท MCP config || OpenClaw | [`plugins/.openclaw-plugin/`](plugins/.openclaw-plugin/) | 17 skills ยท 3 agents ยท MCP config |-**17 domain skills:**--| Skill | What it does |-|---|---|-| `extract` | Full semantic extraction pipeline: NER, relations, events, coreference, triplets |-| `ingest` | Data ingestion from files, databases, APIs, streams, and MCP servers |-| `query` | SPARQL, Cypher, keyword search, structured graph patterns |-| `ontology` | Schema management, concepts, relationships, alignments |-| `validate` | Pipeline, extraction, schema, and ontology validation |-| `deduplicate` | Duplicate detection and entity merging with fuzzy matching |-| `embed` | Node2Vec embeddings, similarity scoring, link prediction |-| `reason` | Deductive, abductive, Datalog, SPARQL, and Rete reasoning engines |-| `decision` | Record, query, and analyze decisions; find precedents; causal analysis |-| `causal` | Cause-effect chains, interventions, counterfactuals, causal influence |-| `temporal` | Point-in-time queries, snapshots, timelines, temporal causal analysis |-| `provenance` | Data lineage, source attribution, audit trails |-| `policy` | Policy definition, enforcement, compliance checks, access control |-| `explain` | Decision logic transparency, causal context, audit-ready explanations |-| `export` | Multi-format export: JSON, RDF, Parquet, CSV, GraphML |-| `change` | Graph change tracking, diffs, temporal updates, impact analysis |-| `visualize` | Topology, centrality, communities, paths, embeddings, decision graphs |--**3 specialized agents:**--| Agent | Role |-|---|---|-| `kg-assistant` | General-purpose KG-aware assistant โ knows all APIs and method signatures |-| `decision-advisor` | Decision intelligence specialist: causal reasoning, precedents, policy violations |-| `explainability` | Reasoning transparency specialist โ generates audit-ready explanation reports |+**17 domain skills:** `extract` ยท `ingest` ยท `query` ยท `ontology` ยท `validate` ยท `deduplicate` ยท `embed` ยท `reason` ยท `decision` ยท `causal` ยท `temporal` ยท `provenance` ยท `policy` ยท `explain` ยท `export` ยท `change` ยท `visualize`-**Hooks** (`plugins/hooks/hooks.json`) โ `PreToolUse` / `PostToolUse` matchers for syntax validation and automated warnings.+**3 specialized agents:** `kg-assistant` ยท `decision-advisor` ยท `explainability`-โ [`plugins/.claude-plugin/README.md`](plugins/.claude-plugin/README.md)+**Hooks** (`plugins/hooks/hooks.json`) โ `PreToolUse` / `PostToolUse` matchers for syntax validation and automated warnings. โ [`plugins/.claude-plugin/README.md`](plugins/.claude-plugin/README.md)-### MCP Server (expose Semantica to any MCP-aware tool)--Semantica ships a full **MCP server** (`semantica/mcp_server.py`) โ run it once and any MCP-compatible tool connects automatically:+### MCP Server```bashpython -m semantica.mcp_server```-Add to your tool's config (Claude Desktop, Windsurf, Cline, Continue, VS Code, Roo Code):-```json{"mcpServers": {- "semantica": {- "command": "python",- "args": ["-m", "semantica.mcp_server"]- }+ "semantica": { "command": "python", "args": ["-m", "semantica.mcp_server"] }}}```-**12 tools exposed:** `extract_entities`, `extract_relations`, `record_decision`, `query_decisions`, `find_precedents`, `get_causal_chain`, `add_entity`, `add_relationship`, `run_reasoning`, `get_graph_analytics`, `export_graph`, `get_graph_summary`--**3 resources:** `semantica://graph/summary`, `semantica://decisions/list`, `semantica://schema/info`+**12 tools:** `extract_entities` ยท `extract_relations` ยท `record_decision` ยท `query_decisions` ยท `find_precedents` ยท `get_causal_chain` ยท `add_entity` ยท `add_relationship` ยท `run_reasoning` ยท `get_graph_analytics` ยท `export_graph` ยท `get_graph_summary`-See [`plugins/.claude-plugin/README.md`](plugins/.claude-plugin/README.md) for per-tool config snippets.+**3 resources:** `semantica://graph/summary` ยท `semantica://decisions/list` ยท `semantica://schema/info`-### MCP Client (Ingest from MCP Servers)--Semantica also includes an **MCP client** (`semantica/ingest/mcp_client.py`) that lets you pull data from any Python/FastMCP server into a knowledge graph:+### MCP Client (Ingest from any MCP server)```pythonfrom semantica.ingest import MCPClient-client = MCPClient("http://your-mcp-server:8080")-resources = client.list_resources() # discover available resources+client = MCPClient("http://your-mcp-server:8080")+resources = client.list_resources()data = client.read_resource("resource://your-data")```-Supported connection schemes: `http://`, `https://`, `mcp://`, `sse://` ยท JSON-RPC ยท auth support ยท dynamic capability discovery.+Supported schemes: `http://` ยท `https://` ยท `mcp://` ยท `sse://` ยท JSON-RPC ยท auth ยท dynamic capability discovery----## ๐ฅ๏ธ Semantica Knowledge Explorer+## ๐ฅ๏ธ Knowledge Explorer-A real-time visual interface for exploring every dimension of your knowledge graph โ built into the repo under [`explorer/`](explorer/).+A real-time visual interface under [`explorer/`](explorer/) โ React 19 + Sigma.js.| Workspace | What you can do ||---|---|-| **Knowledge Graph** | Pan, zoom, and inspect a live Sigma.js graph canvas with ForceAtlas2 layout |+| **Knowledge Graph** | Pan, zoom, and inspect a live graph canvas with ForceAtlas2 layout || **Timeline** | Scrub through temporal events and watch the graph evolve |-| **Decisions** | Browse the causal chain behind every recorded decision with outcome badges |+| **Decisions** | Browse the causal chain behind every recorded decision || **Registry** | Live audit log of every graph mutation โ add-node, add-edge, merge, delete |-| **Entity Resolution** | Review and merge duplicate entities detected by the deduplication engine |+| **Entity Resolution** | Review and merge duplicates detected by the deduplication engine || **KG Overview** | Aggregate stats, community breakdown, centrality heatmap || **Ontology** | SKOS/OWL vocabulary hierarchy and auto-generated schema summary |-### Run locally-```bash-# 1. Start the Semantica backend (port 8000)-python -m semantica.server--# 2. In a second terminal-cd explorer-npm install-npm run dev+python -m semantica.server # Terminal 1 โ backend (port 8000)+cd explorer && npm install && npm run dev # Terminal 2 โ UI```-Open **http://localhost:5173** โ the Explorer connects automatically. All `/api` and `/ws` traffic is proxied to `127.0.0.1:8000` by Vite, so no CORS configuration is needed.--> **Requirements:** Node 18+ ยท Python 3.8+ ยท npm 9+--For the full setup guide, troubleshooting, and production build instructions see [`explorer/README.md`](explorer/README.md).+Open **http://localhost:5173** โ all `/api` and `/ws` traffic proxied by Vite, no CORS config needed. โ [`explorer/README.md`](explorer/README.md)----## Features--### Context & Decision Intelligence-- **Context Graphs** โ structured graph of entities, relationships, and decisions; queryable, causal, persistent-- **Decision tracking** โ record, link, and analyze every agent decision with `add_decision()`, `record_decision()`-- **Causal chains** โ link decisions with `add_causal_relationship()`, trace lineage with `trace_decision_chain()`-- **Precedent search** โ hybrid similarity search over past decisions with `find_similar_decisions()`-- **Influence analysis** โ `analyze_decision_impact()`, `analyze_decision_influence()` โ understand downstream effects-- **Policy engine** โ enforce business rules with `check_decision_rules()`; automated compliance validation-- **Agent memory** โ `AgentMemory` with short/long-term storage, conversation history, and statistics-- **Cross-system context capture** โ `capture_cross_system_inputs()` for multi-agent pipelines--### Knowledge Graphs-- **Knowledge graph construction** โ entities, relationships, properties, typed edges-- **Graph algorithms** โ PageRank, betweenness centrality, clustering coefficient, community detection-- **Node embeddings** โ Node2Vec embeddings via `NodeEmbedder`-- **Similarity** โ cosine similarity via `SimilarityCalculator`-- **Link prediction** โ score potential new edges via `LinkPredictor`-- **Temporal graphs** โ time-aware nodes and edges-- **Incremental / delta processing** โ update graphs without full recompute--### Semantic Extraction-- **Entity extraction** โ named entity recognition, normalization, classification-- **Relation extraction** โ triplet generation from raw text using LLMs or rule-based methods-- **LLM-typed extraction** โ extraction with typed relation metadata-- **Deduplication v1** โ Jaro-Winkler similarity, basic blocking-- **Deduplication v2** โ `blocking_v2`, `hybrid_v2`, `semantic_v2` strategies with `max_candidates_per_entity`-- **Triplet deduplication** โ `dedup_triplets()` for removing duplicate (subject, predicate, object) triples--### Reasoning Engines-- **Forward chaining** โ `Reasoner` with IF/THEN string rules and dict facts-- **Rete network** โ `ReteEngine` for high-throughput production rule matching-- **Deductive reasoning** โ `DeductiveReasoner` for classical inference-- **Abductive reasoning** โ `AbductiveReasoner` for hypothesis generation from observations-- **SPARQL reasoning** โ `SPARQLReasoner` for query-based inference over RDF graphs--### Provenance & Auditability-- **Entity provenance** โ `ProvenanceTracker.track_entity(entity_id, source, metadata)`-- **Algorithm provenance** โ `AlgorithmTrackerWithProvenance` tracks computation lineage-- **Graph builder provenance** โ `GraphBuilderWithProvenance` records entity source lineage from URLs-- **W3C PROV-O compliant** โ lineage tracking across all modules-- **Change management** โ version control with checksums, audit trails, compliance support--### Vector Store-- **Backends** โ FAISS, Pinecone, Weaviate, Qdrant, Milvus, PgVector, in-memory-- **Semantic search** โ top-k retrieval by embedding similarity-- **Hybrid search** โ vector + keyword with configurable weights-- **Filtered search** โ metadata-based filtering on any field-- **Custom similarity weights** โ tune retrieval per use case--### ๐ Graph Database Support-- **AWS Neptune** โ Amazon Neptune graph database with IAM authentication-- **Apache AGE** โ PostgreSQL graph extension with openCypher via SQL-- **FalkorDB** โ native support; `DecisionQuery` and `CausalChainAnalyzer` work directly with FalkorDB row/header shapes--### Data Ingestion-- **File formats** โ PDF, DOCX, HTML, JSON, CSV, Excel, PPTX, archives-- **Web crawl** โ `WebIngestor` with configurable depth-- **Databases** โ `DBIngestor` with SQL query support-- **Snowflake** โ `SnowflakeIngestor` with table/query ingestion, pagination, and key-pair/OAuth auth-- **Docling** โ advanced document parsing with table and layout extraction (PDF, DOCX, PPTX, XLSX)-- **Media** โ image OCR, audio/video metadata extraction--### Export Formats-- **RDF** โ Turtle (`.ttl`), JSON-LD, N-Triples (`.nt`), XML via `RDFExporter`-- **Parquet** โ `ParquetExporter` for entities, relationships, and full KG export-- **ArangoDB AQL** โ ready-to-run INSERT statements via `ArangoAQLExporter`-- **OWL ontologies** โ export generated ontologies in Turtle or RDF/XML-- **SHACL shapes** โ export auto-derived constraint shapes via `RDFExporter.export_shacl()` (`.ttl`, `.jsonld`, `.nt`, `.shacl`)--### Pipeline & Production-- **Pipeline builder** โ `PipelineBuilder` with stage chaining and parallel workers-- **Validation** โ `PipelineValidator` returns `ValidationResult(valid, errors, warnings)` before execution-- **Failure handling** โ `FailureHandler` with `RetryPolicy` and `RetryStrategy` (exponential backoff, fixed, etc.)-- **Parallel processing** โ configurable worker count per pipeline stage-- **LLM providers** โ 100+ models via LiteLLM (OpenAI, Anthropic, Cohere, Mistral, Ollama, and more)--### Ontology-- **Auto-generation** โ derive OWL ontologies from knowledge graphs via `OntologyGenerator`-- **Import** โ load existing OWL, RDF, Turtle, JSON-LD ontologies via `OntologyImporter`-- **Validation** โ HermiT/Pellet compatible consistency checking-- **SHACL shape generation** โ `OntologyEngine.to_shacl()` auto-derives SHACL node and property shapes from any Semantica ontology dict; zero hand-authoring; deterministic (same ontology โ same shapes)-- **SHACL validation** โ `OntologyEngine.validate_graph()` runs shapes against a data graph and returns a `SHACLValidationReport` with machine-readable violations and plain-English explanations-- **Quality tiers** โ `"basic"` (structure + cardinality), `"standard"` (+ enumerations, inheritance), `"strict"` (+ `sh:closed` rejects undeclared properties)-- **Inheritance propagation** โ child shapes automatically include all ancestor property shapes (up to 3+ levels), cycle-safe-- **Three output formats** โ Turtle (`.ttl`), JSON-LD, N-Triples; file export via `export_shacl()`--## ๐ What's New in v0.4.0--### ๐ Temporal Intelligence Stack--Everything you need to reason about *when* โ not just *what*.--- **Temporal GraphRAG** โ retrieve knowledge as it existed at any point in the past. Natural-language queries like *"what did we know before the 2024 merger?"* are automatically parsed for temporal intent, with zero LLM calls.-- **Allen Interval Algebra** โ 13 deterministic interval relations (before, meets, overlaps, during, starts, finishes, equals, and their converses). Find gaps, measure coverage, detect cycles โ all without touching an LLM.-- **Point-in-time Query Engine** โ reconstruct a self-consistent graph snapshot at any timestamp. Comes with a consistency validator that catches 5 classes of temporal errors: inverted intervals, dangling edges, overlapping relations, temporal gaps, and missing entities.-- **Temporal Metadata Extraction** โ ask the LLM to annotate each extracted relation with `valid_from`, `valid_until`, and a calibrated confidence score (0โ1 scale with baked-in anchors, so the model doesn't cluster near 1.0).-- **TemporalNormalizer** โ converts ISO 8601, partial dates, relative phrases ("last year", "Q1 2024"), and 13 domain-specific phrase maps (Healthcare, Finance, Cybersecurity, Supply Chain, Energyโฆ) into UTC datetime pairs. Zero LLM calls.-- **Bi-temporal Provenance** โ every provenance record is automatically stamped with transaction time. Full revision history and audit log export in JSON or CSV. Temporal relationships export as OWL-Time RDF triples.-- **Decision validity windows** โ decisions now carry `valid_from` / `valid_until`. Superseded decisions stay in the graph โ history is immutable. Point-in-time causal chain reconstruction included.-- **Named checkpoints** โ snapshot the full agent context at any moment and diff two snapshots to see exactly what changed.--โ [Temporal docs](docs/reference/) ยท [Temporal examples](cookbook/)--### ๐ SKOS Vocabulary Management--Build and query controlled vocabularies inside your knowledge graph--- Add SKOS concepts with labels, alt-labels, broader/narrower hierarchy, and definitions โ all required triples assembled automatically.-- Query and search vocabularies with SPARQL-backed APIs (injection-sanitized).-- REST API for the Explorer: list schemes, fetch full hierarchy trees (cycle-safe), and import `.ttl` / `.rdf` / `.owl` files.--โ [SKOS docs](docs/reference/ontology.md)--### ๐ท SHACL Constraints--Turn ontologies into executable data contracts โ no hand-authoring.--- Auto-derive SHACL node and property shapes from any Semantica ontology. Deterministic: same ontology always produces the same shapes.-- Three strictness tiers: `"basic"` (structure + cardinality), `"standard"` (+ enumerations and inheritance), `"strict"` (closes shapes โ rejects undeclared properties).-- Validate any RDF graph and get back a report with plain-English violation explanations ready to feed into an LLM or pipeline.-- Use in CI to catch breaking ontology changes before they reach production.--โ SHACL shape generation and validation are available via the `OntologyEngine` โ see [ontology docs](docs/reference/ontology.md)--### ๐ง Infrastructure & Fixes--- **ContextGraph pagination** โ memory complexity dropped from O(N) to O(limit). A 50k-node graph no longer allocates 2.5M dicts per paginated request.-- **Named graph support** โ full config-flag enforcement, duplicate clause prevention, backward-compat URI alias, and safe URI encoding in SPARQL pruning.-- **Ollama remote support** โ `OllamaProvider` now correctly connects to remote Ollama servers instead of silently falling back to `localhost`.-- **Security** โ API key logging removed from extractors; CI workflows locked to least-privilege `contents: read`.--โ [Full changelog](CHANGELOG.md) ยท [Release notes](RELEASE_NOTES.md)-------## ๐ฆ What Was in v0.3.0--First stable `Production/Stable` release on PyPI.--- **Context Graphs** โ temporal validity windows, weighted BFS, cross-graph navigation with full save/load persistence.-- **Decision Intelligence** โ complete lifecycle from recording to impact analysis; `PolicyEngine` with versioned rules.-- **KG Algorithms** โ PageRank, betweenness centrality, Louvain community detection, Node2Vec embeddings, link prediction.-- **Deduplication v2** โ blocking/hybrid candidate generation **63.6% faster**; semantic dedup **6.98x faster**.-- **Delta Processing** โ SPARQL-based incremental diff, `delta_mode` pipelines, snapshot versioning.-- **Export** โ Parquet (Spark/BigQuery/Databricks ready), ArangoDB AQL, RDF format aliases.-- **Pipeline** โ exponential/fixed/linear backoff, `PipelineValidator`, fixed retry loop.-- **Graph Backends** โ Apache AGE (SQL injection fixed), AWS Neptune, FalkorDB, PgVector.-------## โจ What Semantica Does--### ๐งฉ Context & Decision Intelligence--Track every decision your agent makes as a structured, queryable graph node โ with causal links, precedent search, impact analysis, and policy enforcement.--- Every decision records who made it, why, what outcome was chosen, and how confident.-- Decisions are linked causally so you can trace the full chain of reasoning that led to any outcome.-- Hybrid similarity search finds past decisions that match the current scenario.-- Policy rules validate decisions against business constraints before or after they're made.-- `AgentMemory` handles short/long-term storage and conversation history across sessions.--โ [Decision tracking docs](docs/reference/) ยท [Decision tracking example](cookbook/)--### ๐ Temporal Reasoning--Ask not just *what* your agent knows, but *when it was true*.--- Reconstruct your knowledge graph at any point in the past without modifying the current graph.-- Parse natural-language temporal queries and rewrite them into structured datetime constraints.-- Reason over time intervals using a full deterministic Allen algebra implementation.-- Normalize any date expression โ ISO 8601, relative phrases, domain-specific vocabulary โ into UTC.-- Extract temporal validity from text using the LLM, with calibrated confidence scores.--โ [Temporal docs](docs/reference/) ยท [Temporal cookbook](cookbook/)--### ๐บ๏ธ Knowledge Graphs--Build, enrich, and analyze knowledge graphs with production-grade algorithms.--- Add entities, relationships, and typed properties with full metadata support.-- Run PageRank, betweenness centrality, and Louvain community detection out of the box.-- Generate Node2Vec embeddings and score potential new links.-- Delta processing keeps large graphs fresh without full recomputes.--โ [KG docs](docs/reference/)--### ๐ Semantic Extraction--Pull structured knowledge out of raw text.--- Extract entities and relationships from text using LLMs or rule-based methods.-- Generate (subject, predicate, object) triplets ready to load into any KG.-- Deduplicate entities intelligently โ Jaro-Winkler, semantic, and hybrid strategies. v2 is up to **6.98x faster**.-- Optionally have the LLM annotate each relation with temporal validity and confidence.--โ [Extraction docs](docs/reference/)--### ๐ง Reasoning Engines--Go beyond retrieval โ derive new facts from what you know.--- **Forward chaining** โ IF/THEN rules over facts.-- **Rete network** โ high-throughput production rule matching for real-time event streams.-- **Deductive** โ classical inference from axioms.-- **Abductive** โ generate plausible hypotheses from observations.-- **SPARQL** โ query-based inference over RDF graphs.-- **Temporal** โ deterministic Allen algebra, no LLM required.--โ [Reasoning docs](docs/reference/)--### ๐ Provenance & Auditability--Every fact, decision, and computation links back to where it came from.--- Auto-stamp transaction time on every provenance record.-- Query full revision history for any fact โ version, author, validity window, supersession chain.-- Export audit logs in JSON or CSV.-- Export temporal relationships as OWL-Time RDF triples.-- W3C PROV-O compliant across all modules.--โ [Provenance docs](docs/reference/)--### ๐ท Ontology & SHACL--Build, import, and enforce data contracts for your knowledge graphs.--- Auto-generate OWL ontologies from any KG โ no hand-authoring.-- Import existing ontologies in OWL, RDF, Turtle, or JSON-LD.-- Derive SHACL shapes from any ontology and validate graphs against them.-- Manage SKOS controlled vocabularies with hierarchy, search, and REST APIs.+## โจ Features-โ [Ontology docs](docs/reference/ontology.md)--### ๐ญ Pipeline & Production--Orchestrate multi-stage KG pipelines with reliability built in.--- Chain ingest โ extract โ deduplicate โ build โ export stages with a fluent builder API.-- Validate the pipeline config before running it.-- Retry failed stages with exponential backoff, fixed delay, or linear backoff.-- 100+ LLM providers via LiteLLM โ OpenAI, Anthropic, Mistral, Ollama, Azure, Bedrock, and more.--โ [Pipeline docs](docs/reference/)--### ๐ Vector Store--Semantic memory with hybrid search and metadata filtering.--- FAISS, Pinecone, Weaviate, Qdrant, Milvus, PgVector, and in-memory โ one API for all.-- Hybrid search mixes vector similarity and keyword matching with configurable weights.-- Filter by any metadata field, or tune similarity weights per use case.+| Capability | Highlights |+|---|---|+| **Context Graphs** | Structured, queryable graph of entities, decisions, and relationships; causal links; cross-graph navigation |+| **Decision Intelligence** | `record_decision()`, `trace_decision_chain()`, `find_similar_decisions()`, `analyze_decision_impact()`, `check_decision_rules()` |+| **Temporal Intelligence** | Point-in-time snapshots, Allen interval algebra, `TemporalNormalizer`, bi-temporal provenance, decision validity windows |+| **Semantic Extraction** | NER, relation extraction, triplet generation, temporal bounds; deduplication v2 up to **6.98x faster** |+| **Reasoning Engines** | Forward chaining, Rete network, deductive, abductive, SPARQL, Datalog โ explainable output |+| **Provenance** | W3C PROV-O compliant; every fact traced to source; audit log export in JSON/CSV; OWL-Time RDF export |+| **Ontology & SHACL** | Auto-generate OWL ontologies; import OWL/RDF/Turtle/JSON-LD; auto-derive SHACL shapes; SKOS vocabularies |+| **Vector Store** | FAISS, Pinecone, Weaviate, Qdrant, Milvus, PgVector, in-memory; hybrid + filtered search |+| **Pipeline** | `PipelineBuilder` with stage chaining, parallel workers, validation, and retry policies |+| **Graph Databases** | Neo4j, FalkorDB, Apache AGE, AWS Neptune |+| **LLM Providers** | 100+ models via LiteLLM โ OpenAI, Anthropic, Cohere, Mistral, Ollama, Groq, Azure, Bedrock, and more |----## Modules--| Module | What it provides |-|---|---|-| `semantica.context` | Context graphs, agent memory, decision tracking, causal analysis, precedent search, policy engine |-| `semantica.kg` | Knowledge graph construction, graph algorithms, centrality, community detection, embeddings, link prediction, provenance |-| `semantica.semantic_extract` | NER, relation extraction, event extraction, coreference, triplet generation, LLM-enhanced extraction |-| `semantica.reasoning` | Forward chaining, Rete network, deductive, abductive, SPARQL reasoning, explanation generation |-| `semantica.vector_store` | FAISS, Pinecone, Weaviate, Qdrant, Milvus, PgVector, in-memory; hybrid & filtered search |-| `semantica.export` | RDF (Turtle/JSON-LD/N-Triples/XML), Parquet, ArangoDB AQL, CSV, YAML, OWL, graph formats |-| `semantica.ingest` | Files (PDF, DOCX, CSV, HTML), web crawl, feeds, databases, Snowflake, MCP, email, repositories |-| `semantica.ontology` | Auto-generation (6-stage pipeline), OWL/RDF export, import (OWL/RDF/Turtle/JSON-LD), validation, versioning, **SHACL shape generation & validation** |-| `semantica.pipeline` | Pipeline DSL, parallel workers, validation, retry policies, failure handling, resource scheduling |-| `semantica.graph_store` | Graph database backends โ Neo4j, FalkorDB, Apache AGE, Amazon Neptune; Cypher queries |-| `semantica.embeddings` | Text embedding generation โ Sentence-Transformers, FastEmbed, OpenAI, BGE; similarity calculation |-| `semantica.deduplication` | Entity deduplication, similarity scoring, merging, clustering; blocking and semantic strategies |-| `semantica.provenance` | W3C PROV-O compliant end-to-end lineage tracking, source attribution, audit trails |-| `semantica.parse` | Document parsing โ PDF, DOCX, PPTX, HTML, code, email, structured data, media with OCR |-| `semantica.split` | Document chunking โ recursive, semantic, entity-aware, relation-aware, graph-based, ontology-aware |-| `semantica.normalize` | Data normalization for text, entities, dates, numbers, quantities, languages, encodings |-| `semantica.conflicts` | Multi-source conflict detection (value, type, relationship, temporal, logical) with resolution strategies |-| `semantica.change_management` | Version storage, change tracking, checksums, audit trails, compliance support for KGs and ontologies |-| `semantica.triplet_store` | RDF triplet store integration โ Blazegraph, Jena, RDF4J; SPARQL queries and bulk loading |-| `semantica.visualization` | Interactive and static visualization of KGs, ontologies, embeddings, analytics, and temporal graphs |-| [`explorer/`](explorer/) | **Semantica Knowledge Explorer** โ React 19 + Sigma.js UI: graph canvas, decision viewer, causal chains, entity resolution, ontology browser, and registry audit log |-| `semantica.seed` | Seed data management for initial KG construction from CSV, JSON, databases, and APIs |-| `semantica.core` | Framework orchestration, configuration management, knowledge base construction, plugin system |-| `semantica.llms` | LLM provider integrations โ Groq, OpenAI, Novita AI, HuggingFace, LiteLLM |-| `semantica.utils` | Shared utilities โ logging, validation, exception handling, constants, types, progress tracking |-## ๐ป Code Examples-### Decision Tracking--```python-from semantica.context import ContextGraph--graph = ContextGraph(advanced_analytics=True)--# Record decisions with full reasoning context-# record_decision() accepts keyword args and returns the decision ID-loan_id = graph.record_decision(- category="loan_approval",- scenario="Mortgage โ 780 credit score, 28% DTI",- reasoning="Strong credit history, stable 8-year income, low DTI",- outcome="approved",- confidence=0.95,-)-rate_id = graph.record_decision(- category="interest_rate",- scenario="Set rate for approved mortgage",- reasoning="Prime applicant qualifies for lowest tier",- outcome="rate_set_6.2pct",- confidence=0.98,-)--# Link decisions causally โ builds an auditable chain-graph.add_causal_relationship(loan_id, rate_id, relationship_type="enables")--# Query the graph-similar = graph.find_similar_decisions("mortgage approval", max_results=5)-chain = graph.trace_decision_chain(loan_id)-impact = graph.analyze_decision_impact(loan_id)-compliance = graph.check_decision_rules({"category": "loan_approval", "confidence": 0.95})-```--โ [Full decision tracking guide](docs/reference/) ยท [Cookbook examples](cookbook/)-### Temporal GraphRAG```python@@ -730,9 +447,7 @@ rewriter = TemporalQueryRewriter()result = rewriter.rewrite("What decisions were made before the 2024 merger?")# result.temporal_intent โ "before"# result.at_time โ datetime(2024, ..., tzinfo=UTC)-# result.rewritten_query โ "What decisions were made"-# Filter any retriever to a point in time โ drop-in wrapperretriever = TemporalGraphRetriever(base_retriever=your_retriever,at_time=datetime(2024, 3, 1, tzinfo=timezone.utc),@@ -740,294 +455,102 @@ retriever = TemporalGraphRetriever(ctx = retriever.retrieve("supplier approval decisions")# Normalize any date expression to UTC โ zero LLM calls-normalizer = TemporalNormalizer()-start, end = normalizer.normalize("Q1 2024")+start, end = TemporalNormalizer().normalize("Q1 2024")# โ (datetime(2024, 1, 1, UTC), datetime(2024, 3, 31, UTC))--start, end = normalizer.normalize("effective from 2023-09-01")-# โ (datetime(2023, 9, 1, UTC), None)-```--โ [Temporal GraphRAG docs](docs/reference/) ยท [Temporal cookbook](cookbook/)--### Point-in-Time Graph Snapshots--```python-from semantica.context import ContextGraph, AgentContext-from semantica.vector_store import VectorStore-from datetime import datetime, timezone--graph = ContextGraph()--# record_decision() accepts keyword args and supports validity windows-graph.record_decision(- category="policy",- scenario="Approve supplier A",- outcome="approved",- confidence=0.9,- valid_from=datetime(2024, 1, 1, tzinfo=timezone.utc),- valid_until=datetime(2024, 6, 30, tzinfo=timezone.utc),-)--# Reconstruct the graph exactly as it was on any date-# The source graph is never mutated-snapshot = graph.state_at(datetime(2024, 3, 15, tzinfo=timezone.utc))--# Named checkpoints โ checkpoint() and diff_checkpoints() live on AgentContext-context = AgentContext(- vector_store=VectorStore(backend="inmemory"),- knowledge_graph=graph,- decision_tracking=True,-)-context.checkpoint("before_merge")-# ... make changes ...-diff = context.diff_checkpoints("before_merge", "after_merge")-# โ {"decisions_added": [...], "relationships_added": [...], ...}```### Semantic Extraction```pythonfrom semantica.semantic_extract import NERExtractor, RelationExtractor, TripletExtractor--text = """-OpenAI released GPT-4 in March 2023. Microsoft integrated GPT-4 into Azure.-Anthropic, founded by former OpenAI researchers, released Claude as a competing model.-"""--# Step 1 โ extract entities-entities = NERExtractor().extract_entities(text)-# โ [Entity(label="OpenAI", ...), Entity(label="GPT-4", ...), ...]--# Step 2 โ extract relations (requires entities)-relations = RelationExtractor().extract_relations(text, entities=entities)-# โ [Relation(source="OpenAI", type="released", target="GPT-4"), ...]--# Step 3 โ extract full (subject, predicate, object) triplets-triplets = TripletExtractor().extract_triplets(text)-```--### Semantic Extraction with Temporal Bounds--```python-from semantica.semantic_extract import NERExtractorfrom semantica.semantic_extract.methods import extract_relations_llm-text = "The partnership was effective from January 2022 until the merger in Q3 2024."+text = "OpenAI released GPT-4 in March 2023. Microsoft integrated it into Azure."-# extract_relations_llm requires pre-extracted entities as second argentities = NERExtractor().extract_entities(text)-relations = extract_relations_llm(- text,- entities,- provider="openai",- extract_temporal_bounds=True, # LLM annotates each relation with validity window-)--for rel in relations:- print(f"{rel.source} โ {rel.target}")- print(f" valid: {rel.metadata['valid_from']} โ {rel.metadata['valid_until']}")- print(f" confidence: {rel.metadata['temporal_confidence']}")-```--โ [Extraction docs](docs/reference/)--### Knowledge Graphs & Algorithms--```python-from semantica.kg import GraphBuilder, CentralityCalculator, NodeEmbedder, LinkPredictor--# Build a KG from entity/relationship dicts-builder = GraphBuilder()-graph = builder.build({- "entities": [- {"id": "bert", "label": "BERT", "type": "Model"},- {"id": "transformer", "label": "Transformer", "type": "Architecture"},- {"id": "gpt4", "label": "GPT-4", "type": "Model"},- ],- "relationships": [- {"source": "bert", "target": "transformer", "type": "based_on"},- {"source": "gpt4", "target": "transformer", "type": "based_on"},- ],-})+relations = RelationExtractor().extract_relations(text, entities=entities)+triplets = TripletExtractor().extract_triplets(text)-# Graph algorithms-centrality = CentralityCalculator().calculate_pagerank(graph)-embeddings = NodeEmbedder().compute_embeddings(- graph, node_labels=["Model"], relationship_types=["based_on"]+# With temporal bounds โ LLM annotates each relation with validity window+relations_temporal = extract_relations_llm(+ text, entities, provider="openai", extract_temporal_bounds=True)-link_score = LinkPredictor().score_link(graph, "gpt4", "bert", method="common_neighbors")```-โ [KG algorithm docs](docs/reference/) ยท [KG cookbook](cookbook/)-### Reasoning```python-from semantica.reasoning import Reasoner, ReteEngine+from semantica.reasoning import Reasoner, ReteEngine, Rule, Fact, RuleType-# Forward chaining โ derive new facts from rules+# Forward chainingreasoner = Reasoner()reasoner.add_rule("IF Person(?x) THEN Mortal(?x)")-results = reasoner.infer_facts(["Person(Socrates)"])-# โ ["Mortal(Socrates)"]--# Rete network โ build a rule network, add facts, then run pattern matching-from semantica.reasoning import Rule, Fact, RuleType+results = reasoner.infer_facts(["Person(Socrates)"]) # โ ["Mortal(Socrates)"]+# High-throughput Rete networkrete = ReteEngine()-rule = Rule(- rule_id="r1",- name="flag_high_risk",+rete.build_network([Rule(+ rule_id="r1", name="flag_high_risk",conditions=[{"field": "amount", "operator": ">", "value": 10000},{"field": "country", "operator": "in", "value": ["IR", "KP", "SY"]},],conclusion="flag_for_compliance_review",rule_type=RuleType.IMPLICATION,-)-rete.build_network([rule])--fact = Fact(fact_id="f1", predicate="transaction", arguments=[{"amount": 15000, "country": "IR"}])-rete.add_fact(fact)-matches = rete.match_patterns() # returns List[Match]+)])+rete.add_fact(Fact("f1", "transaction", [{"amount": 15000, "country": "IR"}]))+matches = rete.match_patterns()```-โ [Reasoning docs](docs/reference/)--### Ontology Generation & Validation--```python-from semantica.ontology import OntologyEngine--engine = OntologyEngine()--# Derive an OWL ontology from any data dict-ontology = engine.from_data(your_data_dict)--# Export as OWL (Turtle or RDF/XML)-engine.export_owl(ontology, path="domain_ontology.owl", format="turtle")--# Validate ontology consistency-result = engine.validate(ontology)--# Generate ontology from raw text using an LLM-ontology = engine.from_text("Employees work at companies. Companies have departments.")--# Convert ontology to OWL string-owl_str = engine.to_owl(ontology, format="turtle")-```--โ [Ontology docs](docs/reference/ontology.md)--### Pipeline Orchestration--```python-from semantica.pipeline import PipelineBuilder, PipelineValidator-from semantica.pipeline import RetryPolicy, RetryStrategy--# Build a multi-stage pipeline using add_step(name, type, **config)-builder = (- PipelineBuilder()- .add_step("ingest", "file_ingest", source="./documents/", recursive=True)- .add_step("extract", "triplet_extract")- .add_step("deduplicate", "entity_dedup", threshold=0.85)- .add_step("build_kg", "kg_build")- .add_step("export", "rdf_export", format="turtle", output="output/kg.ttl")- .set_parallelism(4) # set_parallelism(), not with_parallel_workers()-)--pipeline = builder.build(name="kg_pipeline")--# Validate before running โ catches config errors early-result = PipelineValidator().validate(pipeline)-if result.valid:- pipeline.run()-```--โ [Pipeline docs](docs/reference/)----## ๐ฆ Modules-- **`semantica.context`** โ context graphs, decisions, causal chains, precedent search, policy engine, checkpoints-- **`semantica.kg`** โ KG construction, graph algorithms, embeddings, link prediction, temporal query engine, Allen algebra, `TemporalNormalizer`, provenance-- **`semantica.semantic_extract`** โ NER, relation extraction, triplet generation, LLM extraction with temporal bounds, deduplication-- **`semantica.reasoning`** โ forward chaining, Rete, deductive, abductive, SPARQL, temporal algebra-- **`semantica.vector_store`** โ FAISS, Pinecone, Weaviate, Qdrant, Milvus, PgVector, in-memory; hybrid & filtered search-- **`semantica.export`** โ RDF (Turtle/JSON-LD/N-Triples/XML), OWL-Time, Parquet, ArangoDB AQL, OWL, SHACL-- **`semantica.ingest`** โ files, web crawl, databases, Snowflake, email, repositories-- **`semantica.ontology`** โ OWL generation & import, SHACL generation & validation, SKOS vocabulary management-- **`semantica.pipeline`** โ stage chaining, parallel workers, validation, retry policies, failure handling-- **`semantica.graph_store`** โ Neo4j, FalkorDB, Apache AGE, Amazon Neptune; Cypher queries-- **`semantica.embeddings`** โ Sentence-Transformers, FastEmbed, OpenAI, BGE; similarity-- **`semantica.deduplication`** โ entity dedup, similarity scoring, blocking and semantic strategies-- **`semantica.provenance`** โ W3C PROV-O lineage, revision history, audit log export-- **`semantica.parse`** โ PDF, DOCX, PPTX, HTML, code, email, media with OCR-- **`semantica.split`** โ recursive, semantic, entity-aware, graph-based, ontology-aware chunking-- **`semantica.conflicts`** โ multi-source conflict detection with resolution strategies-- **`semantica.change_management`** โ version storage, checksums, audit trails, compliance support-- **`semantica.triplet_store`** โ Blazegraph, Jena, RDF4J; SPARQL, bulk loading, SKOS helpers-- **`semantica.visualization`** โ KG, ontology, embedding, and temporal graph visualization-- **`semantica.llms`** โ Groq, OpenAI, Novita AI, HuggingFace, LiteLLM-- **[`explorer/`](explorer/)** โ **Semantica Knowledge Explorer** โ browser UI for live graph inspection, decisions, entity resolution, and ontology browsing (`npm run dev` in `explorer/`)--### Graph Databases-- **Neo4j** โ Cypher queries via `semantica.graph_store`-- **FalkorDB** โ native support; `DecisionQuery` and `CausalChainAnalyzer` work directly with FalkorDB row/header shapes-- **Apache AGE** โ PostgreSQL + openCypher via SQL-- **AWS Neptune** โ Amazon Neptune with IAM authentication--### Vector Databases-- **FAISS** โ built-in, zero extra dependencies-- **Pinecone** โ `pip install semantica[vectorstore-pinecone]`-- **Weaviate** โ `pip install semantica[vectorstore-weaviate]`-- **Qdrant** โ `pip install semantica[vectorstore-qdrant]`-- **Milvus** โ `pip install semantica[vectorstore-milvus]`-- **PgVector** โ `pip install semantica[vectorstore-pgvector]`--### Data Sources-- **Files** โ PDF, DOCX, HTML, JSON, CSV, Excel, PPTX, archives-- **Web** โ configurable-depth crawler-- **Databases** โ SQL via `DBIngestor`-- **Snowflake** โ table/query ingestion, pagination, password/key-pair/OAuth/SSO auth ยท `pip install semantica[db-snowflake]`-- **Docling** โ advanced table and layout extraction (PDF, DOCX, PPTX, XLSX)-- **Email** โ inbox ingestion via `EmailIngestor`-- **Repositories** โ Git repo ingestion for code graph construction--### LLM Providers-- **LiteLLM** โ 100+ models: OpenAI, Anthropic, Cohere, Mistral, Ollama, Azure, AWS Bedrock, and more-- **Novita AI** โ OpenAI-compatible (`deepseek/deepseek-v3.2` and more) ยท set `NOVITA_API_KEY`-- **Groq** โ ultra-low latency inference ยท set `GROQ_API_KEY`-- **HuggingFace** โ local and hosted models via `HuggingFaceProvider`-- **Ollama** โ local models including remote server support+| Module | What it provides |+|---|---|+| `semantica.context` | Context graphs, agent memory, decision tracking, causal analysis, precedent search, policy engine |+| `semantica.kg` | KG construction, graph algorithms, centrality, community detection, embeddings, link prediction, provenance |+| `semantica.semantic_extract` | NER, relation extraction, event extraction, coreference, triplet generation, LLM-enhanced extraction |+| `semantica.reasoning` | Forward chaining, Rete, deductive, abductive, SPARQL, Datalog reasoning |+| `semantica.vector_store` | FAISS, Pinecone, Weaviate, Qdrant, Milvus, PgVector, in-memory; hybrid & filtered search |+| `semantica.export` | RDF (Turtle/JSON-LD/N-Triples/XML), Parquet, ArangoDB AQL, OWL, SHACL, graph formats |+| `semantica.ingest` | Files (PDF, DOCX, CSV, HTML), web crawl, databases, Snowflake, MCP, email, repositories, Parquet |+| `semantica.ontology` | OWL auto-generation, import, validation, SHACL shape generation & validation, SKOS vocabulary management |+| `semantica.pipeline` | Pipeline DSL, parallel workers, validation, retry policies, failure handling |+| `semantica.graph_store` | Neo4j, FalkorDB, Apache AGE, Amazon Neptune; Cypher queries |+| `semantica.embeddings` | Sentence-Transformers, FastEmbed, OpenAI, BGE; similarity calculation |+| `semantica.deduplication` | Entity deduplication โ blocking, hybrid, semantic strategies; result limiting |+| `semantica.provenance` | W3C PROV-O lineage, revision history, audit log export |+| `semantica.parse` | PDF, DOCX, PPTX, HTML, code, email, media with OCR (Docling integration) |+| `semantica.split` | Recursive, semantic, entity-aware, graph-based, ontology-aware chunking |+| `semantica.conflicts` | Multi-source conflict detection with resolution strategies |+| `semantica.change_management` | Version storage, checksums, audit trails, compliance support |+| `semantica.triplet_store` | Blazegraph, Jena, RDF4J; SPARQL queries and bulk loading |+| `semantica.visualization` | KG, ontology, embedding, and temporal graph visualization |+| [`explorer/`](explorer/) | React 19 + Sigma.js browser UI โ graph canvas, decisions, entity resolution, ontology |+| `semantica.llms` | Groq, OpenAI, Novita AI, HuggingFace, LiteLLM |---## ๐ ๏ธ Installation```bash-# Core-pip install semantica--# All optional dependencies-pip install semantica[all]+pip install semantica # core+pip install semantica[all] # everything-# Pick only what you need+# pick what you need+pip install semantica[agno]pip install semantica[vectorstore-pinecone]pip install semantica[vectorstore-weaviate]pip install semantica[vectorstore-qdrant]pip install semantica[vectorstore-milvus]pip install semantica[vectorstore-pgvector]-pip install semantica[db-snowflake] # Snowflake ingestion-pip install semantica[agno] # Agno integration+pip install semantica[db-snowflake]+pip install semantica[ingest-parquet]-# From source+# from sourcegit clone https://github.com/Hawksight-AI/semantica.git-cd semantica-pip install -e ".[dev]"-pytest tests/+cd semantica && pip install -e ".[dev]" && pytest tests/```---@@ -1046,38 +569,22 @@ pytest tests/----## ๐ค Community & Support--- ๐ฌ **[Discord](https://discord.gg/sV34vps5hH)** โ real-time help and showcases-- ๐ก **[GitHub Discussions](https://github.com/Hawksight-AI/semantica/discussions)** โ Q&A and feature requests-- ๐ **[GitHub Issues](https://github.com/Hawksight-AI/semantica/issues)** โ bug reports-- ๐ **[Documentation](https://github.com/Hawksight-AI/semantica/tree/main/docs)** โ full reference docs-- ๐ณ **[Cookbook](https://github.com/Hawksight-AI/semantica/tree/main/cookbook)** โ runnable notebooks and recipes-- ๐ **[Changelog](CHANGELOG.md)** โ what changed and why-- ๐ **[Release Notes](RELEASE_NOTES.md)** โ per-contributor breakdown------## ๐ข Enterprise Support-**Production-grade Semantica for your organization**+**[Website](https://getsemantica.ai/)** โ enterprise solutions, private cloud deployment, custom domain implementations, professional services.-๐ **[Website](https://getsemantica.ai/)** โ Enterprise solutions and custom domain-specific implementations--**Domain-Specific Solutions:**-- ๐ฅ **Healthcare** โ Clinical decision support, patient safety, compliance-- ๏ฟฝ **Finance** โ Fraud detection, regulatory compliance, risk management-- โ๏ธ **Legal** โ Evidence analysis, contract review, case law reasoning-- ๏ฟฝ **Cybersecurity** โ Threat attribution, incident response, provenance-- ๐๏ธ **Government** โ Policy tracking, classified information governance+----**Custom Solutions Available:**-- ๏ฟฝ Private cloud deployment-- ๐ก๏ธ Enterprise security compliance-- ๏ฟฝ Custom analytics and reporting-- ๐ฏ Professional implementation services+## ๐ค Community & Support----+| | |+|---|---|+| ๐ฌ **Discord** | [discord.gg/sV34vps5hH](https://discord.gg/sV34vps5hH) โ real-time help and showcases |+| ๐ก **GitHub Discussions** | [Q&A and feature requests](https://github.com/Hawksight-AI/semantica/discussions) |+| ๐ **GitHub Issues** | [Bug reports](https://github.com/Hawksight-AI/semantica/issues) |+| ๐ **Documentation** | [docs.getsemantica.ai](https://docs.getsemantica.ai/) |+| ๐ณ **Cookbook** | [Runnable notebooks and recipes](https://github.com/Hawksight-AI/semantica/tree/main/cookbook) |+| ๐ **Changelog** | [CHANGELOG.md](CHANGELOG.md) ยท [Release Notes](RELEASE_NOTES.md) |## ๐ค Contributing@@ -1096,6 +603,6 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for full guidelines.MIT License ยท Built by [Hawksight AI](https://github.com/Hawksight-AI) ยท [โญ Star on GitHub](https://github.com/Hawksight-AI/semantica)-[GitHub](https://github.com/Hawksight-AI/semantica) ยท [Discord](https://discord.gg/sV34vps5hH) ยท [X / Twitter](https://x.com/BuildSemantica)+[GitHub](https://github.com/Hawksight-AI/semantica) ยท [Discord](https://discord.gg/sV34vps5hH) ยท [X / Twitter](https://x.com/BuildSemantica) ยท [Website](https://getsemantica.ai/)</div>