Graph-Native Infrastructure for Context and Accountable AI Systems
Python agent-memoryaiai-governanceai-infrastructureartificial-intelligencecontext-engineeringcontext-graphsdata-engineeringdecision-intelligencedeveloper-toolsexplainable-aigenerative-aigraph-ragknowledge-graphllmontologyprovenancepythonreasoningsemantic-search
fix(provenance): compare timestamp ranges by instant, not by spelling
FABIOTESSe03212cd6661
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 9b90b3a3..9db73a09 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -88,6 +88,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0- New `utc_now()`/`utc_now_iso()` in `semantica/utils/helpers.py`, exported from `semantica.utils`, and used at all 29 call sites in `export/` (`json_exporter`, `yaml_exporter`, `report_generator`, `export_provenance`) and `provenance/` (`manager`, `schemas`, `bridge_axiom`). Values now read `2026-08-19T14:19:04.229937+00:00`: one unambiguous instant, comparable against any correctly stamped value, and valid `xsd:dateTimeStamp`. `sem:exportedAt`'s range in `semantica/ontology/vocabulary/semantica-ns.ttl` is tightened from `xsd:dateTime` accordingly, and its comment no longer has to explain why the weaker range was necessary- `datetime.utcnow()` is deprecated as of Python 3.12 and scheduled for removal; constructing a `ProvenanceEntry` under `-W error::DeprecationWarning` on 3.13 raised, and no longer does- New `tests/export/test_timestamp_timezones.py` and `tests/provenance/test_timestamp_timezones.py`: offset presence on every export and provenance path, PROV-O literals valid as `xsd:dateTimeStamp`, comparison against a timezone-aware instant without `TypeError`, the Oxigraph filter that dropped the naive value (with a bound inside the indeterminate window, so the test cannot pass by accident), and the document `@id` remaining a valid IRI with `+00:00` in it. 11 of the 13 fail on the parent commit+ - **Fixed during review** (Qodo): once new entries carry `+00:00` and stored ones do not, `ProvenanceManager.query_recorded_between` and `audit_log` compared ISO timestamps as raw strings, so they ordered by spelling rather than by instant — an inclusive naive bound naming a stored offset-bearing timestamp sorted *below* it and dropped the record, and a bound written in another offset landed wherever its digits fell (`19:45+05:30` is 14:15Z, but sorted after 14:19Z). Both now compare instants through a new `to_utc_datetime()` helper that reads a missing offset as UTC, which is what the values written before this change actually were; a bound that cannot be read as a timestamp keeps the historical string comparison rather than raising on a call that used to work- The remaining 147 naive call sites are in `context/`, `vector_store/`, `seed/` and elsewhere, where timestamps are compared against values parsed from previously stored naive strings. Converting those without a read-side migration would raise `TypeError: can't compare offset-naive and offset-aware datetimes` on existing data, so they are deliberately left for a separate change- **`split`/chunking paths bypassed the centralized spaCy model cache, reloading the model on every call** (#1042, closes #998) by @Accute9, reviewed by @Sameer6305diff --git a/semantica/provenance/manager.py b/semantica/provenance/manager.pyindex 1a0efa3d..1a2b26f1 100644--- a/semantica/provenance/manager.py+++ b/semantica/provenance/manager.py@@ -26,6 +26,7 @@ License: MITfrom typing import Optional, List, Dict, Any, Unionfrom collections.abc import Mapping+from datetime import datetime, timezonefrom contextlib import contextmanagerimport copyimport inspect@@ -35,9 +36,13 @@ import threadingfrom .schemas import ProvenanceEntry, SourceReference, AgentRecord, ActivityRecordfrom .storage import ProvenanceStorage, InMemoryStorage, SQLiteStoragefrom .integrity import compute_checksum, verify_checksum-from ..utils.helpers import utc_now_iso+from ..utils.helpers import to_utc_datetime, utc_now_isofrom ..utils.logging import get_logger+#: Sort key for an entry whose timestamp cannot be read as one, so an+#: unreadable value orders first instead of raising during a sort.+_EPOCH = datetime(1, 1, 1, tzinfo=timezone.utc)+# Issue #825, Part B Tier 3 — configurable base URI for export_prov(), shared# with RDFExporter's NamespaceManager "semantica" entry (semantica/export/# rdf_exporter.py) so KG-exported and PROV-exported URIs for the same@@ -959,11 +964,27 @@ class ProvenanceManager:Returns:List of matching entries as dicts, sorted by timestamp ascending."""- matches = [- e for e in self.storage.retrieve_all()- if e.timestamp and start <= e.timestamp <= end- ]- matches.sort(key=lambda e: e.timestamp)+ # Compare instants, not spellings. Since #1114 new entries carry a+ # +00:00 offset while entries written earlier do not, and a raw string+ # comparison orders those two by length: an inclusive naive bound equal+ # to a stored offset-bearing timestamp would sort below it and drop the+ # record. A bound in another offset was mis-ordered the same way.+ start_at = to_utc_datetime(start)+ end_at = to_utc_datetime(end)+ entries = [e for e in self.storage.retrieve_all() if e.timestamp]++ if start_at is None or end_at is None:+ # A bound this module cannot read as a timestamp keeps the historical+ # string comparison rather than raising on a call that used to work.+ matches = [e for e in entries if start <= e.timestamp <= end]+ else:+ matches = [+ e for e in entries+ if (at := to_utc_datetime(e.timestamp)) is not None+ and start_at <= at <= end_at+ ]++ matches.sort(key=lambda e: (to_utc_datetime(e.timestamp) or _EPOCH, e.timestamp))return [e.to_dict() for e in matches]def get_all_sources(self, entity_id: str) -> List[Dict[str, Any]]:@@ -1163,8 +1184,22 @@ class ProvenanceManager:"""entries = self.storage.retrieve_all()if since:- entries = [e for e in entries if getattr(e, "timestamp", "") >= since]- entries.sort(key=lambda e: getattr(e, "timestamp", ""))+ since_at = to_utc_datetime(since)+ if since_at is None:+ entries = [e for e in entries+ if getattr(e, "timestamp", "") >= since]+ else:+ entries = [+ e for e in entries+ if (at := to_utc_datetime(getattr(e, "timestamp", None)))+ is not None and at >= since_at+ ]+ entries.sort(+ key=lambda e: (+ to_utc_datetime(getattr(e, "timestamp", None)) or _EPOCH,+ getattr(e, "timestamp", ""),+ )+ )if format == "json":return [diff --git a/semantica/utils/__init__.py b/semantica/utils/__init__.pyindex 599e0746..a6d7aa9e 100644--- a/semantica/utils/__init__.py+++ b/semantica/utils/__init__.py@@ -82,6 +82,7 @@ from .helpers import (normalize_entities,normalize_graph_payload,parse_timestamp,+ to_utc_datetime,utc_now,utc_now_iso,read_json_file,@@ -195,6 +196,7 @@ __all__ = ["get_file_size","format_timestamp","parse_timestamp",+ "to_utc_datetime","utc_now","utc_now_iso","merge_dicts",diff --git a/semantica/utils/helpers.py b/semantica/utils/helpers.pyindex 47f5865c..05ece01f 100644--- a/semantica/utils/helpers.py+++ b/semantica/utils/helpers.py@@ -349,6 +349,36 @@ def utc_now_iso() -> str:return utc_now().isoformat()+def to_utc_datetime(value: Union[str, datetime, None]) -> Optional[datetime]:+ """+ Read an ISO 8601 timestamp as a timezone-aware UTC instant.++ Timestamps written before #1114 carry no offset. They were produced by+ ``datetime.utcnow()``, so a missing offset is read as UTC: that keeps a+ stored naive value and the same instant written with an offset comparing+ equal, instead of ordering by how the timestamp happens to be spelled.++ Args:+ value: ISO 8601 string or datetime. ``Z`` is accepted as the offset.++ Returns:+ Timezone-aware UTC datetime, or None if the value cannot be read as a+ timestamp, so callers can fall back rather than raise on stored data+ """+ if value is None:+ return None+ if isinstance(value, datetime):+ parsed = value+ else:+ try:+ parsed = datetime.fromisoformat(str(value).replace("Z", "+00:00"))+ except ValueError:+ return None+ if parsed.tzinfo is None:+ parsed = parsed.replace(tzinfo=timezone.utc)+ return parsed.astimezone(timezone.utc)++def parse_timestamp(timestamp_str: str, format_str: Optional[str] = None) -> datetime:"""Parse timestamp string to datetime.diff --git a/tests/provenance/test_timestamp_timezones.py b/tests/provenance/test_timestamp_timezones.pyindex f665b859..b2b91959 100644--- a/tests/provenance/test_timestamp_timezones.py+++ b/tests/provenance/test_timestamp_timezones.py@@ -85,3 +85,69 @@ def test_prov_o_timestamps_are_valid_datetimestamp():if isinstance(o, rdflib.Literal) and o.datatype == XSD.dateTime]:assert rdflib.Literal(str(stamp), datatype=XSD.dateTime).ill_typed is Falseassert datetime.fromisoformat(str(stamp)).utcoffset() is not None+++class TestRangeQueriesCompareInstants:+ """Range APIs compared ISO strings, so they ordered by spelling (#1121 review).++ Once new entries carry ``+00:00`` and stored ones do not, a raw string+ comparison puts an inclusive naive bound *below* the offset-bearing+ timestamp it names, dropping the record, and a bound written in another+ offset lands wherever its digits fall rather than at its instant.+ """++ @staticmethod+ def _manager_with(timestamps):+ manager = ProvenanceManager()+ for index, stamp in enumerate(timestamps):+ manager.storage.store(ProvenanceEntry(+ entity_id=f"e{index}", entity_type="Doc",+ activity_id="act", timestamp=stamp,+ ))+ return manager++ def test_inclusive_bound_written_without_an_offset_still_matches(self):+ manager = self._manager_with(["2026-08-19T14:19:04.229937+00:00"])++ found = manager.query_recorded_between(+ "2026-08-19T00:00:00", "2026-08-19T14:19:04.229937"+ )+ assert [e["entity_id"] for e in found] == ["e0"]++ def test_bound_in_another_offset_selects_by_instant(self):+ """19:45+05:30 is 14:15Z: before the entry, though its digits are after."""+ manager = self._manager_with(["2026-08-19T14:19:04+00:00"])++ assert manager.query_recorded_between(+ "2026-08-19T00:00:00Z", "2026-08-19T19:45:00+05:30"+ ) == []+ assert len(manager.query_recorded_between(+ "2026-08-19T00:00:00Z", "2026-08-19T19:50:00+05:30"+ )) == 1++ def test_legacy_and_offset_bearing_entries_are_both_found_and_ordered(self):+ manager = self._manager_with([+ "2026-08-19T14:19:05+00:00", # written after #1114+ "2026-08-19T14:19:04", # written before it, meaning UTC+ ])++ found = manager.query_recorded_between(+ "2026-08-19T14:00:00Z", "2026-08-19T15:00:00Z"+ )+ assert [e["entity_id"] for e in found] == ["e1", "e0"]++ def test_audit_log_since_reads_a_naive_bound_as_utc(self):+ manager = self._manager_with([+ "2026-08-19T14:19:05+00:00",+ "2026-08-19T09:00:00",+ ])++ recent = manager.audit_log(since="2026-08-19T14:19:05", format="json")+ assert [e["entity_id"] for e in recent] == ["e0"]++ def test_an_unreadable_bound_falls_back_to_the_previous_behaviour(self):+ """A call that used to work with a non-timestamp bound must not raise."""+ manager = self._manager_with(["2026-08-19T14:19:04+00:00"])++ assert manager.query_recorded_between("not-a-date", "also-not") == []+ assert manager.audit_log(since="not-a-date", format="json") == []