Independent Auditing of AI Agents. Run by human or the agent itself, to answer the most crucial question in the AI Agent Economy. Is the agent doing what is supposed to do? With iFixAi you can have this answer in less than 120 seconds.
Python agent-evaluationaiai-alignmentai-evaluationai-governanceai-safetyclidiagnostic-tooleu-ai-acthallucination-detectioniso-42001llm-evaluationllm-securitynist-ai-rmfowasp-llmprompt-injectionpythonresponsible-airisk-assessmentrisk-management
feat: explore business-policy compositions and goal-preserving repairs (#316)
Rudy Mizrahi Celeklida791ecba62d
diff --git a/README.md b/README.mdindex d700cc3..702e34a 100644--- a/README.md+++ b/README.md@@ -84,6 +84,8 @@ All three run the same diagnostic underneath. The difference is how you configur## Quick start+**Explore business-policy compositions:** [Policy Lab](docs/business-policy-lab.md) finds shortest harmful sequences of permitted actions and checks permission repairs against legitimate completion goals. Includes an offline refund-store example and explicit local-callable replay.+Now try it yourself. Pick a path from the table above; full walkthrough: **[docs/get-started.md](docs/get-started.md)**.### Guided wizard (recommended)diff --git a/docs/README.md b/docs/README.mdindex 8082308..53bb3f9 100644--- a/docs/README.md+++ b/docs/README.md@@ -12,6 +12,7 @@ needs: **learning**, **doing**, **looking up**, and **understanding**- **[Test your own agent](testing-your-agent.md)**: wire in your real agent via `--provider http` or a `ChatProvider` adapter, with the provider reference.- **[Author a fixture](fixture_authoring.md)**: declare your roles, tools, permissions, policies, and governance.+- **[Business Policy Lab](business-policy-lab.md)**: explore harmful permitted-action compositions and check repairs that preserve completion goals.- **[Reproduce a run](reproducibility.md)**: the manifest, the digest algorithm, and verification helpers.## 📖 Looking something up → referencediff --git a/docs/adr/003-business-policy-lab.md b/docs/adr/003-business-policy-lab.mdnew file mode 100644index 0000000..b6d4b23--- /dev/null+++ b/docs/adr/003-business-policy-lab.md@@ -0,0 +1,39 @@+# ADR-003: Explicit business-policy reachability and goal-preserving repair++Status: Proposed++## Context++Individual role/tool authorization tests and scripted conversations can miss harmful compositions of individually allowed business actions. Recorded trajectories only cover paths already observed. An auditor needs a prospective check that can expose a short forbidden path before rollout and assess whether a proposed authorization change destroys useful business completion.++Tool descriptions and fixture mock responses cannot establish transition semantics. Arbitrary automatic execution of imported actions would introduce authority and side effects unrelated to inspecting a model.++## Decision++Add a separate offline Policy Lab with explicit finite Boolean state, exact actors and permission rules, simultaneous guarded effects, named forbidden conjunctions and existential completion goals. Preserve every configured budget counter in visited-state identity. Deterministic breadth-first search retains one shortest witness per objective using parent pointers. Explore the complete reachable frontier unless bounded depth, state or examination limits require an inconclusive result. Counterexample evidence remains valid even when the frontier is incomplete; safe certification requires complete coverage and useful enabled transitions. Completion goals prevent a deny-all policy from becoming an accepted repair.++Permission repair enumerates one- or two-rule removals with shared candidate and examination budgets. Accept only globally verified candidates and qualify minimum cardinality when any smaller candidate was inconclusive. Preserve the original facts, guards, effects, goals and budgets. Produce a candidate artifact; never mutate deployed policy.++Use canonical typed JSON bundles with strict parse, digest verification and full recomputation on replay. Checksums establish consistency rather than truth or authentication. Project existing fixture authorization only when exact IDs and role matrices agree, explicitly recording its absent scope/effect semantics.++Keep actual witness execution a separate Python API requiring caller-supplied synchronous bindings, reset and full independent observations. Validate evidence and binding declarations before any execution. Compare initial, pre-action and post-action snapshots; stop on mismatch or exception. Imported JSON never dispatches Python names or providers. No hidden threads, asynchronous continuation or unsupported timeouts.++Expose check, replay, compare, repair and fixture projection through an additive CLI group, public Python exports, private evidence files and a self-contained report. Add no runtime dependency or inspection-grade coupling.++## Alternatives considered++- More scripted systemic-harm prompts: useful behavioral coverage but no exhaustive model reachability or shortest-path guarantee.+- A passive trajectory checker: verifies declared observations but cannot discover an unobserved composition.+- Infer action effects from prose using a model: would mistake hypotheses for business ground truth and add non-determinism and billing.+- An SMT dependency or generalized planning language: broader expressiveness but heavier integration and semantic surface than the bounded Boolean/counter contract requires.+- Repair by blocking every tool: eliminates some unsafe paths but fails declared business completion and is explicitly rejected.++## Consequences and boundaries++Model authors must declare sufficient state. Model verification is relative to those declarations, not proof of deployment safety, fairness, universal liveness or cross-tenant authorization. Finite exploration is exponential in facts and counters; hard input/search/batch bounds and honest inconclusive decisions replace unsupported scalability claims. Native observations provide evidence for one explicitly exercised local witness and can miss unmodeled side effects. Caller-supplied callables retain their normal authority.++Frozen models use detached revalidation to avoid mutable nested containers and Pydantic copy/coercion bypasses. Exact counter identity can increase exploration but is required for correct budget reachability. Permission removal minimality is cardinality within the declared search space, not global business-policy optimality.++## Acceptance and rollout++Use a separately implemented exhaustive primitive oracle, adversarial evaluator tests, native match/mismatch/failure tests, fixture and CLI integration, private escaped reports, existing regressions, repository lint/security/layout checks, same-revision wheel and exact patch-alone verification. Demonstrate real synchronous local refund-store operations: a three-action harmful composition and a repaired independent-approval path, both independently observed. Preserve evidence and obtain an independent non-implementer review of the exact commit before publication. Keep existing inspection scores unchanged and retain manual maintainer merge authority.diff --git a/docs/business-policy-lab.md b/docs/business-policy-lab.mdnew file mode 100644index 0000000..21f789f--- /dev/null+++ b/docs/business-policy-lab.md@@ -0,0 +1,90 @@+# Business Policy Lab++A tool can be permitted in isolation while its combination with other permitted tools violates a business invariant. Policy Lab explores an explicit finite business model, retains the shortest harmful sequence for each invariant, and searches permission changes that preserve legitimate completion goals.++This is a separate offline assurance surface. It does not change inspection grades or execute providers. Its input is a declared business transition model, not a tool description or a recorded conversation.++## Run the local refund example++From a repository checkout with iFixAi installed:++```sh+python examples/business_policy_lab.py --output ./refund-policy-evidence+```++The example uses actual supplied Python callables and an independent observer of a benign local accounting object. Three individually permitted operations request, self-approve and pay a synthetic refund. Policy Lab discovers the three-step counterexample. Removing the agent's approval permission eliminates the modeled harm while the reviewer-approved payment goal remains reachable. Both the unsafe witness and the repaired legitimate witness are replayed against the local object. No real money or remote services run.++The output contains model and policy JSON, original and selected bundles, repair evidence, observed callable execution reports, and HTML/Markdown reports for check, replay, repair and comparison.++## Declare business behavior++Use `BusinessModel`, `ExplorationPolicy` and `SearchConfig` from `ifixai.policy_lab` or the main operations from `ifixai.api`. See the executable example for a complete model.++- `initial_facts` declares 1–64 exact Boolean facts. Every guard, effect, forbidden condition and goal references a declared fact. Missing facts are errors.+- `actors` maps exact IDs to explicit roles. Actions specify one actor, exact tool, resource, Boolean guards, simultaneous effects and optional integer cost units.+- `properties` names forbidden Boolean conjunctions. A violated initial state produces a zero-step counterexample.+- `goals` names required legitimate completion conjunctions. These are existential reachable outcomes, not universal liveness guarantees.+- Policy permission rules match an actor role, exact tool and resource. Resource scopes are exact strings or one terminal `*` prefix wildcard. Deny is the default.+- Optional trace or actor budgets charge declared cost units. Repeated constraints on one pool apply their strictest limit. Budget counters are part of state identity, even when facts return to an earlier value.+- History, ownership, cross-tenant behavior and other business semantics must be encoded explicitly in facts and action contracts. A tenant ID must match between the model and policy; it does not authenticate a deployment.++All models are bounded, typed and reject extra fields, coercions and ambiguous IDs. No action implementation is inferred from descriptions or imported by name.++## Search, replay, compare and repair++```sh+ifixai policy-lab check model.json policy.json --output ./new-check+ifixai policy-lab replay new-check/bundle.json --output ./new-replay+ifixai policy-lab compare new-check/bundle.json candidate-policy.json --output ./new-comparison+ifixai policy-lab repair new-check/bundle.json --max-removals 1 --output ./new-repair+```++Each destination must be new; evidence files are created with owner-only permissions. HTML reports are self-contained and include searchable witness paths, permission matches, before/after facts, budgets, goals, frontier coverage and cutoffs. Comparison retains the exact model and search configuration and shows property and goal status changes.++Breadth-first search uses deterministic action-ID order and exact facts plus configured budget counters. It retains parent pointers to reconstruct shortest witnesses rather than copying all paths. Default limits are depth 12, 10,000 states and 200,000 state/action examinations. Check supports depth up to 64, 100,000 states and 2,000,000 examinations. Limits constrain exploration; they are not wall-clock or memory guarantees.++| Decision | Meaning | Exit |+| --- | --- | --- |+| `COUNTEREXAMPLE` | At least one reachable forbidden condition has a retained witness, including if exploration later stops. | 1 |+| `VERIFIED` | Complete declared frontier, all properties safe, all goals reachable, at least one enabled transition. | 0 |+| `GOAL_BLOCKED` | Complete declared frontier but at least one completion goal unreachable. | 1 |+| `INCONCLUSIVE` | Limits or missing useful transition coverage prevent a global verification. | 2 |++Per-property verification can be conclusive in a complete zero-edge graph, but global `VERIFIED` is disallowed. Reaching a depth bound does not truncate a graph if every enabled edge returns to an already visited state. Disabled actions do not create edges. Reports count all state/action examinations and identify actions never observed enabled.++Bundles include the complete inputs, limits, report and canonical SHA-256 integrity digest. Replay strictly parses and revalidates the schema, verifies the digest and recomputes every report field. Re-signing a false report still fails. Digests establish consistency, not signer identity or real-world truth.++Repair searches removal of one or two permission rules, in ascending cardinality and rule-index order. It accepts only fully `VERIFIED` candidates, including goal preservation. Denying all operations is not an accepted repair. The default 64-candidate and 250,000 shared transition-examination budgets bound the batch; caps are 256 candidates and 2,000,000 examinations. A smaller inconclusive candidate prevents a minimum-cardinality claim. `EXHAUSTED` refers only to the requested removal space; `FOUND` does not modify deployed permissions. Repair exits 0 for `FOUND`, 1 for `EXHAUSTED`, 2 for `INCONCLUSIVE`.++## Bridge a fixture's authorization matrix++```sh+ifixai policy-lab from-fixture model.json customer_support --output ./new-fixture-check+```++`policy_from_fixture(fixture, model)` projects exact user IDs, role memberships and declared tool permissions. Wildcard tools expand to known fixture tools. Unknown IDs, mismatching model actors, duplicate authorization rows and contradictory governance/top-level matrices are rejected. Fixtures do not define resource scopes or business effects: projected scopes are `*`, the caller supplies the tenant and transition contracts, and metadata records that boundary. Tool prose and mock fixture responses are not executed business behavior.++## Observe a local implementation++```python+from ifixai.policy_lab import execute_witness++execution = execute_witness(+ bundle, bundle.report.safety_witnesses[0],+ bindings={"action_id": actual_callable},+ reset=reset_local_state,+ observe_state=read_all_declared_boolean_facts,+)+```++Supply bindings for every action in the retained witness. Each synchronous callable receives `ExecutionContext` with actor, action, tool, resource and declared cost. Retained evidence and all binding declarations validate before reset or calls. An observer supplies the full exact fact snapshot before and after every action; action return values do not establish state. A mismatch stops before the next action. Missing bindings, invalid evidence and step limits are `INCONCLUSIVE`; application exceptions or deferred returns produce `EXECUTION_ERROR`. Errors retain bounded sanitized type names, never exception payloads.++`MATCHED` means this local implementation matched this particular witness. It is not full runtime verification, enforcement, isolation, a timeout, or production authorization. Supplied callables can have side effects before raising. Costs are declared units, not observed billing. Run such replay only against an implementation and reset you explicitly intend to exercise.++## Validate changes++```sh+python -m pytest ifixai/tests/policy_lab -q+```++The suite includes an independent exhaustive primitive oracle, budget-sensitive state identity, limit/fixed-point behavior, malformed and re-signed evidence, goal-preserving repairs, fixture/CLI integration, escaped reports and observed native-callable mismatches. Synthetic distinct-state benchmarks are local resource measurements, not production latency claims.diff --git a/examples/business_policy_lab.py b/examples/business_policy_lab.pynew file mode 100644index 0000000..0a4a40a--- /dev/null+++ b/examples/business_policy_lab.py@@ -0,0 +1,142 @@+"""Find a harmful legal composition, repair it, and replay real local bindings.++Run: python examples/business_policy_lab.py --output ./new-business-lab-demo+The store is benign local accounting. No real money, services or models run.+"""++from __future__ import annotations++import argparse+import json+import subprocess+import sys+from pathlib import Path++sys.path.insert(0, str(Path(__file__).resolve().parents[1]))++from ifixai.policy_lab import create_exploration_bundle+from ifixai.policy_lab.executor import execute_witness+from ifixai.policy_lab.models import (+ ActionContract,+ Actor,+ BudgetRule,+ BusinessGoal,+ BusinessModel,+ ExplorationPolicy,+ PermissionRule,+ SafetyProperty,+ SearchConfig,+)+from ifixai.policy_lab.repair import search_repairs+++def demo_model():+ return BusinessModel(+ name="Local refund composition", tenant_id="demo-tenant",+ initial_facts={"requested": False, "approved": False, "independent": False, "paid": False},+ actors={"agent": Actor(roles=("agent",)), "reviewer": Actor(roles=("reviewer",))},+ actions=(+ ActionContract(action_id="01_request", actor_id="agent", tool="request_refund", resource="account:demo", requires={"requested": False, "paid": False}, effects={"requested": True}, cost_units=1),+ ActionContract(action_id="02_self_approve", actor_id="agent", tool="approve_refund", resource="account:demo", requires={"requested": True, "approved": False}, effects={"approved": True, "independent": False}, cost_units=1),+ ActionContract(action_id="03_independent_approve", actor_id="reviewer", tool="approve_refund", resource="account:demo", requires={"requested": True, "approved": False}, effects={"approved": True, "independent": True}, cost_units=1),+ ActionContract(action_id="04_pay", actor_id="agent", tool="pay_refund", resource="account:demo", requires={"approved": True, "paid": False}, effects={"paid": True}, cost_units=1),+ ),+ properties=(SafetyProperty(property_id="no_self_approved_payment", label="Payment requires independent approval", forbidden={"paid": True, "independent": False}),),+ goals=(BusinessGoal(goal_id="legitimate_refund", label="Independent approval still reaches payment", requires={"paid": True, "independent": True}),),+ )+++def demo_policy():+ return ExplorationPolicy(name="Permitted local operations", tenant_id="demo-tenant", permissions=(+ PermissionRule(role="agent", tool="request_refund", resource_scope="account:*"),+ PermissionRule(role="agent", tool="approve_refund", resource_scope="account:*"),+ PermissionRule(role="agent", tool="pay_refund", resource_scope="account:*"),+ PermissionRule(role="reviewer", tool="approve_refund", resource_scope="account:*"),+ ), budgets=(BudgetRule(scope="trace", limit_units=3),))+++class LocalRefundStore:+ """Observed state comes from this local object, not action return values."""+ def __init__(self):+ self.reset()++ def reset(self):+ self.facts = {"requested": False, "approved": False, "independent": False, "paid": False}+ self.synthetic_paid_units = 0+ self.calls = []++ def observe(self):+ return dict(self.facts)++ def request(self, context):+ self.calls.append(context.action_id)+ self.facts["requested"] = True++ def self_approve(self, context):+ self.calls.append(context.action_id)+ self.facts.update(approved=True, independent=False)++ def independent_approve(self, context):+ self.calls.append(context.action_id)+ self.facts.update(approved=True, independent=True)++ def pay(self, context):+ self.calls.append(context.action_id)+ self.synthetic_paid_units += 60+ self.facts["paid"] = True++ def bindings(self):+ return {"01_request": self.request, "02_self_approve": self.self_approve, "03_independent_approve": self.independent_approve, "04_pay": self.pay}+++def main():+ parser = argparse.ArgumentParser(description=__doc__)+ parser.add_argument("--output", type=Path, required=True)+ args = parser.parse_args()+ destination = args.output.absolute()+ destination.mkdir(mode=0o700, parents=True, exist_ok=False)+ model, policy, config = demo_model(), demo_policy(), SearchConfig()+ original = create_exploration_bundle(model, policy, config)+ assert original.report.decision == "COUNTEREXAMPLE"+ unsafe = original.report.safety_witnesses[0]+ assert len(unsafe.steps) == 3+ store = LocalRefundStore()+ native_unsafe = execute_witness(original, unsafe, store.bindings(), store.reset, store.observe)+ assert native_unsafe.decision == "MATCHED" and store.synthetic_paid_units == 60+ unsafe_observed = {"synthetic": True, "already_paid_units": store.synthetic_paid_units, "actual_calls": list(store.calls), "no_real_money": True}+ repair = search_repairs(original, max_removals=1, max_candidates=64)+ assert repair.status == "FOUND" and repair.selected_bundle is not None+ selected = repair.selected_bundle+ assert selected.report.decision == "VERIFIED"+ assert selected.report.goal_status["legitimate_refund"] == "REACHABLE"+ native_goal = execute_witness(selected, selected.report.goal_witnesses[0], store.bindings(), store.reset, store.observe)+ assert native_goal.decision == "MATCHED" and store.synthetic_paid_units == 60+ values = {+ "model.json": model.model_dump(mode="json"), "policy.json": policy.model_dump(mode="json"),+ "original-bundle.json": original.model_dump(mode="json"), "repair.json": repair.model_dump(mode="json"),+ "selected-policy.json": selected.policy.model_dump(mode="json"), "selected-bundle.json": selected.model_dump(mode="json"),+ "native-unsafe.json": native_unsafe.model_dump(mode="json"), "native-goal.json": native_goal.model_dump(mode="json"),+ "observed-local-store.json": {"unsafe": unsafe_observed, "goal": {"synthetic": True, "paid_units": store.synthetic_paid_units, "actual_calls": list(store.calls)}},+ }+ for name, value in values.items():+ path = destination / name+ path.write_text(json.dumps(value, indent=2), encoding="utf-8")+ path.chmod(0o600)+ root = Path(__file__).resolve().parents[1]+ for command, expected in (+ (["check", str(destination / "model.json"), str(destination / "policy.json")], 1),+ (["replay", str(destination / "original-bundle.json")], 1),+ (["repair", str(destination / "original-bundle.json"), "--max-removals", "1", "--max-candidates", "64"], 0),+ (["compare", str(destination / "original-bundle.json"), str(destination / "selected-policy.json")], 0),+ ):+ output = destination / ("cli-" + command[0])+ completed = subprocess.run([sys.executable, "-m", "ifixai.cli.main", "policy-lab", *command, "--output", str(output)], cwd=root, check=False)+ if completed.returncode != expected:+ raise RuntimeError(f"Unexpected {command[0]} exit={completed.returncode}")+ print("Three individually permitted operations produced an observed synthetic self-approved payment.")+ print("Permission removal eliminated that model path while preserving the observed independent-approval goal.")+ print(f"Explicit-model and local-observer evidence: {destination}")+++if __name__ == "__main__":+ main()diff --git a/ifixai/api.py b/ifixai/api.pyindex f493e01..81ca2cd 100644--- a/ifixai/api.py+++ b/ifixai/api.py@@ -28,6 +28,18 @@ from ifixai.core.types import (from ifixai.harness.registry import ALL_SPECSfrom ifixai.inspections.holdout_ids import generate_holdout_idsfrom ifixai.judge.config import JudgeConfig+from ifixai.policy_lab import ( # noqa: F401 - public SDK exports+ BusinessModel,+ ExplorationBundle,+ ExplorationPolicy,+ SearchConfig,+ create_exploration_bundle,+ execute_witness,+ explore_policy,+ load_exploration_bundle,+ policy_from_fixture,+ search_repairs,+)from ifixai.providers.base import ChatProviderfrom ifixai.providers.governance_fixture import GovernanceFixturefrom ifixai.providers.resolver import resolve_provider, wrap_with_governancediff --git a/ifixai/cli/main.py b/ifixai/cli/main.pyindex d02e130..221bf70 100644--- a/ifixai/cli/main.py+++ b/ifixai/cli/main.py@@ -8,6 +8,7 @@ from ifixai._version import VERSIONfrom ifixai.cli.compare import comparefrom ifixai.cli.init import init, load_dotenv_filefrom ifixai.cli.list_cmd import list_group+from ifixai.cli.policy_lab import policy_labfrom ifixai.cli.run import runfrom ifixai.cli.scaffold import installfrom ifixai.cli.setup_cmd import setup@@ -29,6 +30,7 @@ ifixai_cli.add_command(list_group, name="list")ifixai_cli.add_command(validate)ifixai_cli.add_command(compare)ifixai_cli.add_command(trajectory)+ifixai_cli.add_command(policy_lab)def _ensure_utf8_stdout() -> None:diff --git a/ifixai/cli/policy_lab.py b/ifixai/cli/policy_lab.pynew file mode 100644index 0000000..59da470--- /dev/null+++ b/ifixai/cli/policy_lab.py@@ -0,0 +1,153 @@+"""Data-only reachability exploration and goal-preserving permission repair."""+from __future__ import annotations++import json+from collections.abc import Callable+from pathlib import Path+from typing import Any++import click+from pydantic import ValidationError++from ifixai.core.fixture_loader import (+ FixtureValidationError,+ load_fixture,+ resolve_fixture_path,+)+from ifixai.policy_lab.fixture import policy_from_fixture+from ifixai.policy_lab.integrity import (+ create_exploration_bundle,+ load_exploration_bundle,+)+from ifixai.policy_lab.models import (+ BusinessModel,+ ExplorationBundle,+ ExplorationPolicy,+ SearchConfig,+)+from ifixai.policy_lab.repair import RepairReport, search_repairs+from ifixai.policy_lab.search import canonical_digest+from ifixai.reporting.policy_lab import compare_policies, write_policy_lab_reports++MAX_INPUT_BYTES = 16 * 1024 * 1024+DECISION_EXIT = {"VERIFIED": 0, "COUNTEREXAMPLE": 1, "GOAL_BLOCKED": 1, "INCONCLUSIVE": 2}+++def _unique_object(pairs: list[tuple[str, Any]]) -> dict[str, Any]:+ result: dict[str, Any] = {}+ for key, value in pairs:+ if key in result:+ raise ValueError("duplicate JSON keys are not accepted")+ result[key] = value+ return result+++def _reject_constant(_: str) -> None:+ raise ValueError("nonfinite JSON constants are not accepted")+++def _input(path: str) -> Any:+ with Path(path).open("rb") as handle:+ data = handle.read(MAX_INPUT_BYTES + 1)+ if len(data) > MAX_INPUT_BYTES:+ raise ValueError("model or policy exceeds the 16 MiB input bound")+ return json.loads(data.decode("utf-8"), object_pairs_hook=_unique_object, parse_constant=_reject_constant)+++def _model(path: str) -> BusinessModel:+ return BusinessModel.model_validate(_input(path))+++def _policy(path: str) -> ExplorationPolicy:+ return ExplorationPolicy.model_validate(_input(path))+++def _guard(operation: Callable[[], Any]) -> Any:+ try:+ return operation()+ except ValidationError as error:+ raise click.ClickException(f"Invalid Policy Lab schema ({error.error_count()} field errors); consult the model contract.") from None+ except (ValueError, TypeError, OSError, UnicodeError, RecursionError, FixtureValidationError) as error:+ raise click.ClickException(f"Policy Lab operation failed ({type(error).__name__}); check input declarations, integrity and new output directory.") from None+++def _write(bundle: ExplorationBundle, output: str, *, comparison: dict[str, Any] | None = None, repair: RepairReport | None = None, metadata: dict[str, Any] | None = None) -> None:+ destination = _guard(lambda: write_policy_lab_reports(bundle, output, comparison=comparison, repair=repair, metadata=metadata))+ if repair:+ click.echo(f"Repair {repair.status}: {len(repair.candidates)} candidates, minimum cardinality proven={repair.minimality_proven}")+ exit_code = {"FOUND": 0, "EXHAUSTED": 1, "INCONCLUSIVE": 2}[repair.status]+ else:+ click.echo(f"{bundle.report.decision}: {len(bundle.report.safety_witnesses)} harm witnesses, {len(bundle.report.goal_witnesses)} completion witnesses")+ exit_code = DECISION_EXIT[bundle.report.decision]+ click.echo(f"Model evidence: {destination / 'report.html'}")+ raise click.exceptions.Exit(exit_code)++[email protected]("policy-lab")+def policy_lab() -> None:+ """Find harmful compositions of permitted business actions in explicit models."""+++@policy_lab.command("check")[email protected]("model", type=click.Path(exists=True, dir_okay=False))[email protected]("policy", type=click.Path(exists=True, dir_okay=False))[email protected]("--max-depth", type=click.IntRange(1, 64), default=12, show_default=True)[email protected]("--max-states", type=click.IntRange(1, 100000), default=10000, show_default=True)[email protected]("--max-transitions", type=click.IntRange(1, 2000000), default=200000, show_default=True)[email protected]("--output", required=True, type=click.Path(file_okay=False))+def check(model: str, policy: str, max_depth: int, max_states: int, max_transitions: int, output: str) -> None:+ bundle = _guard(lambda: create_exploration_bundle(_model(model), _policy(policy), SearchConfig(max_depth=max_depth, max_states=max_states, max_transitions=max_transitions)))+ _write(bundle, output)+++@policy_lab.command("from-fixture")[email protected]("model", type=click.Path(exists=True, dir_okay=False))[email protected]("fixture")[email protected]("--output", required=True, type=click.Path(file_okay=False))+def from_fixture(model: str, fixture: str, output: str) -> None:+ def operation() -> tuple[ExplorationBundle, dict[str, Any]]:+ business = _model(model)+ fixture_path = resolve_fixture_path(fixture)+ if fixture_path.stat().st_size > MAX_INPUT_BYTES:+ raise ValueError("fixture exceeds the 16 MiB bound")+ loaded = load_fixture(fixture_path)+ derived = policy_from_fixture(loaded, business)+ metadata = {"fixture_name": loaded.metadata.name, "fixture_digest": canonical_digest(loaded),+ "boundary": "Imports fixture role/tool declarations only; resource scope is wildcard because the fixture has no resource scope. Effects and tenant remain explicit model declarations."}+ return create_exploration_bundle(business, derived), metadata++ bundle, metadata = _guard(operation)+ _write(bundle, output, metadata=metadata)+++@policy_lab.command("replay")[email protected]("bundle", type=click.Path(exists=True, dir_okay=False))[email protected]("--output", required=True, type=click.Path(file_okay=False))+def replay(bundle: str, output: str) -> None:+ _write(_guard(lambda: load_exploration_bundle(bundle)), output)+++@policy_lab.command("compare")[email protected]("bundle", type=click.Path(exists=True, dir_okay=False))[email protected]("policy", type=click.Path(exists=True, dir_okay=False))[email protected]("--output", required=True, type=click.Path(file_okay=False))+def compare(bundle: str, policy: str, output: str) -> None:+ def operation() -> tuple[ExplorationBundle, dict[str, Any]]:+ original = load_exploration_bundle(bundle)+ candidate = create_exploration_bundle(original.model, _policy(policy), original.config)+ return candidate, compare_policies(original, candidate)++ candidate, comparison = _guard(operation)+ _write(candidate, output, comparison=comparison)+++@policy_lab.command("repair")[email protected]("bundle", type=click.Path(exists=True, dir_okay=False))[email protected]("--max-removals", type=click.IntRange(1, 2), default=1, show_default=True)[email protected]("--max-candidates", type=click.IntRange(1, 256), default=64, show_default=True)[email protected]("--transition-budget", type=click.IntRange(1, 2000000), default=250000, show_default=True)[email protected]("--output", required=True, type=click.Path(file_okay=False))+def repair(bundle: str, max_removals: int, max_candidates: int, transition_budget: int, output: str) -> None:+ original = _guard(lambda: load_exploration_bundle(bundle))+ result = _guard(lambda: search_repairs(original, max_removals=max_removals, max_candidates=max_candidates, transition_budget=transition_budget))+ _write(original, output, repair=result)diff --git a/ifixai/policy_lab/__init__.py b/ifixai/policy_lab/__init__.pynew file mode 100644index 0000000..ec91563--- /dev/null+++ b/ifixai/policy_lab/__init__.py@@ -0,0 +1,56 @@+"""Explicit business-policy reachability, witnesses and replayable evidence."""+from .executor import ExecutionContext, ExecutionReport, execute_witness+from .fixture import policy_from_fixture+from .integrity import (+ create_exploration_bundle,+ load_exploration_bundle,+ validate_exploration_bundle,+)+from .models import (+ ActionContract,+ Actor,+ BudgetRule,+ BusinessGoal,+ BusinessModel,+ ExplorationBundle,+ ExplorationPolicy,+ PermissionRule,+ SafetyProperty,+ SearchConfig,+ SearchReport,+ StateSnapshot,+ Witness,+ WitnessStep,+)+from .repair import RepairCandidate, RepairReport, search_repairs+from .search import canonical_digest, explore_policy, scope_matches++__all__ = [+ "ActionContract",+ "Actor",+ "BudgetRule",+ "BusinessGoal",+ "BusinessModel",+ "ExecutionContext",+ "ExecutionReport",+ "ExplorationBundle",+ "ExplorationPolicy",+ "PermissionRule",+ "RepairCandidate",+ "RepairReport",+ "SafetyProperty",+ "SearchConfig",+ "SearchReport",+ "StateSnapshot",+ "Witness",+ "WitnessStep",+ "canonical_digest",+ "create_exploration_bundle",+ "execute_witness",+ "explore_policy",+ "load_exploration_bundle",+ "policy_from_fixture",+ "scope_matches",+ "search_repairs",+ "validate_exploration_bundle",+]diff --git a/ifixai/policy_lab/executor.py b/ifixai/policy_lab/executor.pynew file mode 100644index 0000000..d50b41d--- /dev/null+++ b/ifixai/policy_lab/executor.py@@ -0,0 +1,152 @@+"""Execute only explicitly supplied local bindings, then independently observe.++Imported contracts never resolve Python names or execute tools. This is opt-in+implementation replay, not enforcement, isolation, or production authorization.+"""++from __future__ import annotations++import inspect+import re+from collections.abc import Callable+from typing import Any, Literal, cast++from pydantic import BaseModel, ConfigDict, Field, StrictBool, StrictInt++from .integrity import canonical_digest, validate_exploration_bundle+from .models import ExplorationBundle, Witness++LIMITATIONS = (+ "Only explicitly supplied synchronous callables execute; imported names never dispatch tools.",+ "Independent observations compare declared Boolean facts, not unmodelled side effects or production causality.",+ "Witness costs are declared model units, not observed billing or real-money evidence.",+ "MATCHED means this local implementation matched this witness; it does not verify all runtime behavior.",+ "Exceptions and deferred returns may follow side effects; execution stops and cannot claim a match.",+ "All-zero digests denote evidence that could not be validated and are not evidence checksums.",+)+++class _Strict(BaseModel):+ model_config = ConfigDict(extra="forbid", frozen=True, allow_inf_nan=False)+++class ExecutionContext(_Strict):+ actor_id: str = Field(min_length=1, max_length=64)+ action_id: str = Field(min_length=1, max_length=64)+ tool: str = Field(min_length=1, max_length=64)+ resource: str = Field(min_length=1, max_length=256)+ cost_units: StrictInt = Field(ge=0, le=2**63 - 1)+++class ObservedStep(_Strict):+ index: StrictInt = Field(ge=1, le=64)+ action_id: str = Field(min_length=1, max_length=64)+ before: dict[str, StrictBool]+ after: dict[str, StrictBool] | None = None+ outcome: Literal["completed", "failed", "mismatch"]+ error_type: str | None = Field(default=None, max_length=100)+++class ExecutionReport(_Strict):+ decision: Literal["MATCHED", "MODEL_MISMATCH", "EXECUTION_ERROR", "INCONCLUSIVE"]+ witness_digest: str = Field(pattern=r"^[0-9a-f]{64}$")+ bundle_digest: str = Field(pattern=r"^[0-9a-f]{64}$")+ steps: tuple[ObservedStep, ...] = Field(default=(), max_length=64)+ attempted_actions: StrictInt = Field(default=0, ge=0, le=64)+ completed_actions: StrictInt = Field(default=0, ge=0, le=64)+ initial_observed: dict[str, StrictBool] | None = None+ reason_codes: tuple[str, ...] = ()+ limitations: tuple[str, ...] = LIMITATIONS+++def _sync_callable(function: Any) -> bool:+ return callable(function) and not any(check(function) or check(getattr(function, "__call__", None)) for check in (inspect.iscoroutinefunction, inspect.isasyncgenfunction, inspect.isgeneratorfunction))+++def _immediate(value: Any) -> Any:+ if inspect.isawaitable(value) or inspect.isgenerator(value) or inspect.isasyncgen(value):+ if inspect.iscoroutine(value) or inspect.isgenerator(value):+ value.close()+ raise TypeError("deferred callable results are not supported")+ return value+++def _observation(function: Callable[[], dict[str, bool]], names: set[str]) -> dict[str, bool] | None:+ raw = _immediate(function())+ if not isinstance(raw, dict) or set(raw) != names or any(type(value) is not bool for value in raw.values()):+ return None+ return dict(raw)+++def _error_type(error: Exception) -> str:+ name = type(error).__name__+ return name if re.fullmatch(r"[A-Za-z_][A-Za-z0-9_]{0,99}", name) else "Exception"+++def execute_witness(bundle: ExplorationBundle, witness: Witness, bindings: dict[str, Callable[[ExecutionContext], Any]], reset: Callable[[], None], observe_state: Callable[[], dict[str, bool]], max_steps: int = 64) -> ExecutionReport:+ """Replay one retained witness against explicit bindings, stopping on mismatch.++ All evidence and binding declarations validate before reset or observation.+ No function timeout or background thread is created by this synchronous API.+ """+ bundle_digest = witness_digest = "0" * 64+ steps: list[ObservedStep] = []+ attempted = completed = 0+ initial: dict[str, bool] | None = None++ def result(decision, code):+ return ExecutionReport(decision=decision, witness_digest=witness_digest, bundle_digest=bundle_digest, steps=tuple(steps), attempted_actions=attempted, completed_actions=completed, initial_observed=initial, reason_codes=(code,))++ try:+ bundle = validate_exploration_bundle(bundle)+ witness = Witness.model_validate(witness.model_dump(mode="python", warnings=False))+ bundle_digest = canonical_digest(bundle)+ witness_digest = canonical_digest(witness)+ except Exception: # noqa: BLE001 - malformed detached evidence must never trigger execution+ return result("INCONCLUSIVE", "INVALID_RETAINED_EVIDENCE")+ if witness not in (*bundle.report.safety_witnesses, *bundle.report.goal_witnesses):+ return result("INCONCLUSIVE", "WITNESS_NOT_RETAINED")+ if type(max_steps) is not int or not 1 <= max_steps <= 64 or len(witness.steps) > max_steps:+ return result("INCONCLUSIVE", "STEP_BUDGET_INVALID_OR_EXCEEDED")+ if not isinstance(bindings, dict) or not _sync_callable(reset) or not _sync_callable(observe_state) or any(step.action_id not in bindings or not _sync_callable(bindings[step.action_id]) for step in witness.steps):+ return result("INCONCLUSIVE", "MISSING_OR_DEFERRED_BINDING")+ names = set(bundle.model.initial_facts)+ try:+ # Runtime validation must inspect even a callable declared to return+ # None: user-supplied Python functions can violate that annotation.+ if _immediate(cast(Callable[[], Any], reset)()) is not None:+ return result("EXECUTION_ERROR", "RESET_MUST_RETURN_NONE")+ initial = _observation(observe_state, names)+ except Exception: # noqa: BLE001 - return sanitized execution outcome, never raw exception payload+ return result("EXECUTION_ERROR", "RESET_OR_INITIAL_OBSERVER_ERROR")+ if initial is None or initial != witness.initial.facts:+ return result("MODEL_MISMATCH", "INITIAL_STATE_MISMATCH")+ for step in witness.steps:+ try:+ before = _observation(observe_state, names)+ except Exception: # noqa: BLE001 - observer errors must not trigger later action calls+ return result("EXECUTION_ERROR", "BEFORE_OBSERVER_ERROR")+ if before is None or before != step.before.facts:+ return result("MODEL_MISMATCH", "BEFORE_STATE_MISMATCH")+ context = ExecutionContext(actor_id=step.actor_id, action_id=step.action_id, tool=step.tool, resource=step.resource, cost_units=step.cost_units)+ attempted += 1+ try:+ _immediate(bindings[step.action_id](context))+ except Exception as error: # noqa: BLE001 - callable failure is retained without its message+ try:+ after = _observation(observe_state, names)+ except Exception: # noqa: BLE001 - do not replace the application failure with observer failure+ after = None+ steps.append(ObservedStep(index=step.index, action_id=step.action_id, before=before, after=after, outcome="failed", error_type=_error_type(error)))+ return result("EXECUTION_ERROR", "ACTION_ERROR_OR_DEFERRED_RESULT")+ completed += 1+ try:+ after = _observation(observe_state, names)+ except Exception: # noqa: BLE001 - action already occurred, report missing observations honestly+ steps.append(ObservedStep(index=step.index, action_id=step.action_id, before=before, after=None, outcome="failed", error_type="ObserverError"))+ return result("EXECUTION_ERROR", "AFTER_OBSERVER_ERROR")+ if after is None or after != step.after.facts:+ steps.append(ObservedStep(index=step.index, action_id=step.action_id, before=before, after=after, outcome="mismatch"))+ return result("MODEL_MISMATCH", "AFTER_STATE_MISMATCH")+ steps.append(ObservedStep(index=step.index, action_id=step.action_id, before=before, after=after, outcome="completed"))+ return result("MATCHED", "ALL_DECLARED_FACTS_MATCHED")diff --git a/ifixai/policy_lab/fixture.py b/ifixai/policy_lab/fixture.pynew file mode 100644index 0000000..6c9f802--- /dev/null+++ b/ifixai/policy_lab/fixture.py@@ -0,0 +1,50 @@+"""Import fixture permissions only; business effects always remain explicit."""+from __future__ import annotations++from ifixai.core.types import Fixture++from .models import BusinessModel, ExplorationPolicy, PermissionRule+++def policy_from_fixture(fixture: Fixture, model: BusinessModel) -> ExplorationPolicy:+ model = BusinessModel.model_validate(model.model_dump(mode="python", warnings=False))+ role_ids = [role.role_id or role.name for role in fixture.roles]+ tool_ids = [tool.tool_id for tool in fixture.tools]+ user_ids = [user.user_id for user in fixture.users]+ for identifiers in (role_ids, tool_ids, user_ids):+ if len(set(identifiers)) != len(identifiers):+ raise ValueError("fixture role, tool and user identifiers must be unique")+ roles, tools = set(role_ids), set(tool_ids)+ users = {user.user_id: user for user in fixture.users}+ for user in fixture.users:+ if len(set(user.roles)) != len(user.roles) or not set(user.roles) <= roles:+ raise ValueError("fixture user role declarations are ambiguous or unknown")+ for actor_id, actor in model.actors.items():+ if actor_id not in users or set(actor.roles) != set(users[actor_id].roles):+ raise ValueError("model actors must exactly match fixture users and their roles")+ if any(action.tool not in tools for action in model.actions):+ raise ValueError("business actions must use explicitly declared fixture tools")+ matrix: dict[str, set[str]] = {}+ for permission in fixture.permissions:+ if permission.role not in roles or permission.role in matrix:+ raise ValueError("fixture permission roles are unknown or duplicated")+ if not set(permission.tools) <= tools | {"*"}:+ raise ValueError("fixture permissions refer to undeclared tools")+ matrix[permission.role] = tools.copy() if "*" in permission.tools else set(permission.tools)+ governance = fixture.governance+ if governance is not None:+ # Conflicting declarative matrices must not be silently promoted to one policy.+ authorization = getattr(getattr(governance, "policies", None), "authorization", None)+ if authorization is None:+ raise ValueError("fixture governance authorization cannot be verified")+ governed: dict[str, set[str]] = {}+ for rule in authorization:+ if rule.role not in roles or rule.role in governed or not set(rule.tools) <= tools | {"*"}:+ raise ValueError("fixture governance authorization is ambiguous")+ governed[rule.role] = tools.copy() if "*" in rule.tools else set(rule.tools)+ if any(matrix.get(role, set()) != governed.get(role, set()) for role in roles):+ raise ValueError("fixture permission and governance matrices disagree")+ referenced_tools = {action.tool for action in model.actions}+ permissions = tuple(PermissionRule(role=role, tool=tool, resource_scope="*")+ for role in sorted(matrix) for tool in sorted(matrix[role] & referenced_tools))+ return ExplorationPolicy(name="fixture permission projection", tenant_id=model.tenant_id, permissions=permissions)diff --git a/ifixai/policy_lab/integrity.py b/ifixai/policy_lab/integrity.pynew file mode 100644index 0000000..ef19926--- /dev/null+++ b/ifixai/policy_lab/integrity.py@@ -0,0 +1,65 @@+"""Detached, replayable exploration evidence with strict deterministic reanalysis."""+from __future__ import annotations++import json+from pathlib import Path+from typing import Any++from .models import BusinessModel, ExplorationBundle, ExplorationPolicy, SearchConfig+from .search import _explore, canonical_digest, snapshot_inputs++MAX_BUNDLE_BYTES = 256 * 1024 * 1024+++def create_exploration_bundle(model: BusinessModel, policy: ExplorationPolicy,+ config: SearchConfig = SearchConfig()) -> ExplorationBundle:+ """Snapshot inputs before exploration; retain declarations and exact analysis."""+ model, policy, config = snapshot_inputs(model, policy, config)+ report = _explore(model, policy, config)+ payload = {"schema_version": "1", "model": model.model_dump(mode="json", warnings=False),+ "policy": policy.model_dump(mode="json", warnings=False), "config": config.model_dump(mode="json", warnings=False),+ "report": report.model_dump(mode="json", warnings=False)}+ return ExplorationBundle.model_validate({**payload, "integrity_digest": canonical_digest(payload)})+++def validate_exploration_bundle(bundle: ExplorationBundle) -> ExplorationBundle:+ """Revalidate detached declarations, checksum, and every recomputed report field."""+ raw = bundle.model_dump(mode="json", warnings=False)+ validated = ExplorationBundle.model_validate(bundle.model_dump(mode="python", warnings=False))+ normalized = validated.model_dump(mode="json", warnings=False)+ if raw != normalized:+ raise ValueError("exploration bundle is not in its normalized complete shape")+ payload = {key: value for key, value in normalized.items() if key != "integrity_digest"}+ if canonical_digest(payload) != validated.integrity_digest:+ raise ValueError("exploration bundle integrity digest mismatch")+ model, policy, config = snapshot_inputs(validated.model, validated.policy, validated.config)+ expected = _explore(model, policy, config)+ if expected.model_dump(mode="json", warnings=False) != validated.report.model_dump(mode="json", warnings=False):+ raise ValueError("exploration report does not match deterministic reanalysis")+ return validated+++def _unique_object(pairs: list[tuple[str, Any]]) -> dict[str, Any]:+ result: dict[str, Any] = {}+ for key, value in pairs:+ if key in result:+ raise ValueError("duplicate JSON object key")+ result[key] = value+ return result+++def _reject_constant(value: str) -> None:+ raise ValueError(f"nonfinite JSON constant is prohibited: {value}")+++def load_exploration_bundle(path: str | Path) -> ExplorationBundle:+ """Read bounded UTF-8 JSON, reject ambiguous input, and recompute the analysis."""+ with Path(path).open("rb") as handle:+ data = handle.read(MAX_BUNDLE_BYTES + 1)+ if len(data) > MAX_BUNDLE_BYTES:+ raise ValueError("exploration bundle exceeds the 256 MiB limit")+ raw = json.loads(data.decode("utf-8"), object_pairs_hook=_unique_object, parse_constant=_reject_constant)+ bundle = ExplorationBundle.model_validate(raw)+ if raw != bundle.model_dump(mode="json", warnings=False):+ raise ValueError("exploration bundle must contain its normalized complete shape")+ return validate_exploration_bundle(bundle)diff --git a/ifixai/policy_lab/models.py b/ifixai/policy_lab/models.pynew file mode 100644index 0000000..073e80e--- /dev/null+++ b/ifixai/policy_lab/models.py@@ -0,0 +1,188 @@+"""Finite, explicit business contracts; declarations contain no executable code."""+from __future__ import annotations++from typing import Annotated, Literal++from pydantic import (+ BaseModel,+ ConfigDict,+ Field,+ StringConstraints,+ field_validator,+ model_validator,+)++Identifier = Annotated[str, StringConstraints(min_length=1, max_length=64, pattern=r"^\S(?:[\s\S]*\S)?$")]+Boolean = Annotated[bool, Field(strict=True)]+Amount = Annotated[int, Field(ge=0, le=2**63 - 1, strict=True)]+Count = Annotated[int, Field(ge=0, le=2000000, strict=True)]+Digest = Annotated[str, StringConstraints(pattern=r"^[0-9a-f]{64}$")]+Label = Annotated[str, StringConstraints(max_length=256)]+Resource = Annotated[str, StringConstraints(min_length=1, max_length=256)]+Facts = dict[Identifier, Boolean]+++class StrictModel(BaseModel):+ model_config = ConfigDict(extra="forbid", frozen=True, allow_inf_nan=False)+++def _exact(value: str) -> str:+ if "*" in value:+ raise ValueError("tool and resource values must be exact, without wildcards")+ return value+++def _scope(value: str) -> str:+ if value.count("*") > 1 or ("*" in value and not value.endswith("*")):+ raise ValueError("resource_scope supports one terminal wildcard only")+ return value+++class Actor(StrictModel):+ roles: tuple[Identifier, ...] = Field(max_length=32)++ @field_validator("roles")+ @classmethod+ def unique_roles(cls, values: tuple[str, ...]) -> tuple[str, ...]:+ if len(values) != len(set(values)):+ raise ValueError("actor roles must be unique")+ return values+++class ActionContract(StrictModel):+ action_id: Identifier+ actor_id: Identifier+ tool: Identifier+ resource: Resource+ requires: Facts = Field(default_factory=dict, max_length=64)+ effects: Facts = Field(min_length=1, max_length=64)+ cost_units: Amount = 0+ exact_tool_resource = field_validator("tool", "resource")(_exact)+++class SafetyProperty(StrictModel):+ property_id: Identifier+ label: Label = ""+ forbidden: Facts = Field(min_length=1, max_length=64)+++class BusinessGoal(StrictModel):+ goal_id: Identifier+ label: Label = ""+ requires: Facts = Field(min_length=1, max_length=64)+++class BusinessModel(StrictModel):+ schema_version: Literal["1"] = "1"+ name: Identifier+ tenant_id: Identifier+ initial_facts: Facts = Field(min_length=1, max_length=64)+ actors: dict[Identifier, Actor] = Field(min_length=1, max_length=64)+ actions: tuple[ActionContract, ...] = Field(min_length=1, max_length=256)+ properties: tuple[SafetyProperty, ...] = Field(min_length=1, max_length=32)+ goals: tuple[BusinessGoal, ...] = Field(min_length=1, max_length=32)++ @model_validator(mode="after")+ def coherent_contract(self) -> BusinessModel:+ for ids in ([action.action_id for action in self.actions], [prop.property_id for prop in self.properties], [goal.goal_id for goal in self.goals]):+ if len(ids) != len(set(ids)):+ raise ValueError("action, property and goal IDs must be unique within each family")+ known_facts = set(self.initial_facts)+ for action in self.actions:+ if action.actor_id not in self.actors:+ raise ValueError("every action actor must be declared")+ if not (set(action.requires) | set(action.effects)) <= known_facts:+ raise ValueError("action guards and effects must refer to declared facts")+ if any(not set(prop.forbidden) <= known_facts for prop in self.properties):+ raise ValueError("safety properties must refer to declared facts")+ if any(not set(goal.requires) <= known_facts for goal in self.goals):+ raise ValueError("business goals must refer to declared facts")+ return self+++class PermissionRule(StrictModel):+ role: Identifier+ tool: Identifier+ resource_scope: Resource = "*"+ exact_tool = field_validator("tool")(_exact)+ valid_scope = field_validator("resource_scope")(_scope)+++class BudgetRule(StrictModel):+ scope: Literal["trace", "actor"]+ limit_units: Amount+ actor_id: Identifier | None = None++ @model_validator(mode="after")+ def coherent_scope(self) -> BudgetRule:+ if (self.scope == "actor") != (self.actor_id is not None):+ raise ValueError("actor budgets require actor_id; trace budgets prohibit actor_id")+ return self+++class ExplorationPolicy(StrictModel):+ schema_version: Literal["1"] = "1"+ name: Identifier+ tenant_id: Identifier+ permissions: tuple[PermissionRule, ...] = Field(default=(), max_length=512)+ budgets: tuple[BudgetRule, ...] = Field(default=(), max_length=65)+++class SearchConfig(StrictModel):+ max_depth: int = Field(default=12, ge=1, le=64, strict=True)+ max_states: int = Field(default=10000, ge=1, le=100000, strict=True)+ max_transitions: int = Field(default=200000, ge=1, le=2000000, strict=True)+++class StateSnapshot(StrictModel):+ facts: Facts = Field(min_length=1, max_length=64)+ budgets: dict[Annotated[str, StringConstraints(min_length=1, max_length=80, pattern=r"^\S(?:[\s\S]*\S)?$")], Amount] = Field(default_factory=dict, max_length=65)+++class WitnessStep(StrictModel):+ index: int = Field(ge=1, le=64, strict=True)+ action_id: Identifier+ actor_id: Identifier+ tool: Identifier+ resource: Resource+ cost_units: Amount+ permission_indices: tuple[Annotated[int, Field(ge=0, le=511, strict=True)], ...] = Field(min_length=1, max_length=512)+ before: StateSnapshot+ after: StateSnapshot+++class Witness(StrictModel):+ objective_id: Identifier+ kind: Literal["safety", "goal"]+ initial: StateSnapshot+ terminal: StateSnapshot+ steps: tuple[WitnessStep, ...] = Field(max_length=64)+++class SearchReport(StrictModel):+ decision: Literal["COUNTEREXAMPLE", "VERIFIED", "GOAL_BLOCKED", "INCONCLUSIVE"]+ property_status: dict[Identifier, Literal["COUNTEREXAMPLE", "VERIFIED", "INCONCLUSIVE"]] = Field(min_length=1, max_length=32)+ goal_status: dict[Identifier, Literal["REACHABLE", "UNREACHABLE", "UNKNOWN"]] = Field(min_length=1, max_length=32)+ safety_witnesses: tuple[Witness, ...] = Field(max_length=32)+ goal_witnesses: tuple[Witness, ...] = Field(max_length=32)+ states_discovered: Count+ states_expanded: Count+ transitions_examined: Count+ transitions_enabled: Count+ frontier_exhausted: Boolean+ cutoff_reasons: tuple[Identifier, ...] = Field(max_length=3)+ coverage_issues: tuple[Label, ...] = Field(max_length=16)+ not_observed_action_ids: tuple[Identifier, ...] = Field(max_length=256)+ model_digest: Digest+ policy_digest: Digest+ config_digest: Digest+ limitations: tuple[Label, ...] = Field(max_length=16)+++class ExplorationBundle(StrictModel):+ schema_version: Literal["1"] = "1"+ model: BusinessModel+ policy: ExplorationPolicy+ config: SearchConfig+ report: SearchReport+ integrity_digest: Digestdiff --git a/ifixai/policy_lab/repair.py b/ifixai/policy_lab/repair.pynew file mode 100644index 0000000..ee1a705--- /dev/null+++ b/ifixai/policy_lab/repair.py@@ -0,0 +1,109 @@+"""Bounded permission-removal search that preserves declared completion goals."""+from __future__ import annotations++from itertools import combinations+from typing import Literal++from pydantic import BaseModel, ConfigDict++from .integrity import create_exploration_bundle, validate_exploration_bundle+from .models import ExplorationBundle, ExplorationPolicy, SearchConfig+from .search import canonical_digest+++class CandidateSummary(BaseModel):+ """Compact candidate coverage; full witnesses retained only for selection."""+ model_config = ConfigDict(extra="forbid", frozen=True)+ decision: Literal["COUNTEREXAMPLE", "VERIFIED", "GOAL_BLOCKED", "INCONCLUSIVE"]+ property_status: dict[str, Literal["COUNTEREXAMPLE", "VERIFIED", "INCONCLUSIVE"]]+ goal_status: dict[str, Literal["REACHABLE", "UNREACHABLE", "UNKNOWN"]]+ states_discovered: int+ states_expanded: int+ transitions_examined: int+ transitions_enabled: int+ frontier_exhausted: bool+ cutoff_reasons: tuple[str, ...]+ coverage_issues: tuple[str, ...]+ policy_digest: str+ config_digest: str+ report_digest: str+ bundle_digest: str+++class RepairCandidate(BaseModel):+ model_config = ConfigDict(extra="forbid", frozen=True)+ removed_rule_indices: tuple[int, ...]+ report: CandidateSummary+++class RepairReport(BaseModel):+ model_config = ConfigDict(extra="forbid", frozen=True)+ status: Literal["FOUND", "EXHAUSTED", "INCONCLUSIVE"]+ original_bundle_digest: str+ candidates: tuple[RepairCandidate, ...]+ selected_bundle: ExplorationBundle | None = None+ max_removals: int+ max_candidates: int+ transition_budget: int+ transitions_examined: int+ minimality_proven: bool+ cutoff_reasons: tuple[str, ...]+ limitations: tuple[str, ...] = (+ "Search removes declared permission rules only; it does not modify guards, budgets, code or deployed authorization.",+ "Accepted repairs preserve only the declared model properties and existential completion goals.",+ "Exhaustion is limited to the specified removal cardinality and does not prove that no other repair exists.",+ )+++def search_repairs(bundle: ExplorationBundle, *, max_removals: int = 1, max_candidates: int = 64, transition_budget: int = 250000) -> RepairReport:+ bundle = validate_exploration_bundle(bundle)+ for value, lower, upper in ((max_removals, 1, 2), (max_candidates, 1, 256), (transition_budget, 1, 2000000)):+ if type(value) is not int or not lower <= value <= upper:+ raise ValueError("repair limits must be bounded strict integers")+ if bundle.report.decision != "COUNTEREXAMPLE":+ raise ValueError("permission repair requires a retained counterexample")+ candidates: list[RepairCandidate] = []+ spent = 0+ cutoff: list[str] = []+ inconclusive_sizes: set[int] = set()+ selected: ExplorationBundle | None = None+ minimal = False+ status: Literal["FOUND", "EXHAUSTED", "INCONCLUSIVE"] = "EXHAUSTED"+ for size in range(1, min(max_removals, len(bundle.policy.permissions)) + 1):+ for removed in combinations(range(len(bundle.policy.permissions)), size):+ if len(candidates) >= max_candidates or spent >= transition_budget:+ cutoff.append("REPAIR_CANDIDATE_LIMIT" if len(candidates) >= max_candidates else "REPAIR_TRANSITION_LIMIT")+ status = "INCONCLUSIVE"+ break+ policy = ExplorationPolicy.model_validate({**bundle.policy.model_dump(mode="json"),+ "permissions": [rule.model_dump(mode="json") for index, rule in enumerate(bundle.policy.permissions) if index not in removed]})+ config = SearchConfig.model_validate({**bundle.config.model_dump(mode="json"),+ "max_transitions": min(bundle.config.max_transitions, transition_budget - spent)})+ candidate = create_exploration_bundle(bundle.model, policy, config)+ spent += candidate.report.transitions_examined+ summary = CandidateSummary.model_validate({+ **{key: getattr(candidate.report, key) for key in (+ "decision", "property_status", "goal_status", "states_discovered",+ "states_expanded", "transitions_examined", "transitions_enabled",+ "frontier_exhausted", "cutoff_reasons", "coverage_issues", "policy_digest", "config_digest")},+ "report_digest": canonical_digest(candidate.report),+ "bundle_digest": candidate.integrity_digest,+ })+ candidates.append(RepairCandidate(removed_rule_indices=removed, report=summary))+ if candidate.report.decision == "INCONCLUSIVE":+ inconclusive_sizes.add(size)+ if candidate.report.decision == "VERIFIED":+ selected = candidate+ status = "FOUND"+ minimal = not any(previous < size for previous in inconclusive_sizes)+ break+ if selected or status == "INCONCLUSIVE":+ break+ if not selected and inconclusive_sizes:+ status = "INCONCLUSIVE"+ cutoff.append("INCOMPLETE_CANDIDATE_EXPLORATION")+ return RepairReport(status=status, original_bundle_digest=bundle.integrity_digest,+ candidates=tuple(candidates), selected_bundle=selected,+ max_removals=max_removals, max_candidates=max_candidates,+ transition_budget=transition_budget, transitions_examined=spent,+ minimality_proven=minimal, cutoff_reasons=tuple(dict.fromkeys(cutoff)))diff --git a/ifixai/policy_lab/search.py b/ifixai/policy_lab/search.pynew file mode 100644index 0000000..d6fd328--- /dev/null+++ b/ifixai/policy_lab/search.py@@ -0,0 +1,208 @@+"""Deterministic exhaustive reachability over explicitly declared Boolean facts."""+from __future__ import annotations++import hashlib+import json+from collections import deque+from dataclasses import dataclass+from typing import Any++from pydantic import BaseModel++from .models import (+ BusinessModel,+ ExplorationPolicy,+ SearchConfig,+ SearchReport,+ StateSnapshot,+ Witness,+ WitnessStep,+)++LIMITATIONS = (+ "Reachability and verification apply only to the explicit finite model and policy.",+ "Unmodelled history, actors, effects and business constraints are not evaluated.",+ "Declared cost units are model counters, not observed billing or runtime resource use.",+ "Model witnesses are not observed production behavior or runtime enforcement.",+ "Goals establish existential reachability, not universal liveness.",+)+++def canonical_digest(value: Any) -> str:+ """Hash canonical JSON; this is an integrity check, not an authenticity proof."""+ if isinstance(value, BaseModel):+ value = value.model_dump(mode="json", warnings=False)+ encoded = json.dumps(value, sort_keys=True, separators=(",", ":"), ensure_ascii=False, allow_nan=False).encode("utf-8")+ return hashlib.sha256(encoded).hexdigest()+++def scope_matches(scope: str, resource: str) -> bool:+ """Match an exact resource or one terminal prefix wildcard."""+ if not isinstance(scope, str) or not isinstance(resource, str):+ raise TypeError("scopes and resources must be strings")+ if scope.count("*") > 1 or ("*" in scope and not scope.endswith("*")):+ raise ValueError("scope supports one terminal wildcard only")+ return resource.startswith(scope[:-1]) if scope.endswith("*") else scope == resource+++def snapshot_inputs(model: BusinessModel, policy: ExplorationPolicy, config: SearchConfig) -> tuple[BusinessModel, ExplorationPolicy, SearchConfig]:+ """Revalidate and detach nested mutable mappings before any evaluation."""+ model = BusinessModel.model_validate(model.model_dump(mode="python", warnings=False))+ policy = ExplorationPolicy.model_validate(policy.model_dump(mode="python", warnings=False))+ config = SearchConfig.model_validate(config.model_dump(mode="python", warnings=False))+ if model.tenant_id != policy.tenant_id:+ raise ValueError("model and policy tenant IDs must match")+ if any(rule.scope == "actor" and rule.actor_id not in model.actors for rule in policy.budgets):+ raise ValueError("actor budget references an undeclared actor")+ return model, policy, config+++@dataclass(frozen=True)+class _Node:+ bits: int+ counters: tuple[int, ...]+ depth: int+ parent: int | None+ action: int | None+++def _explore(model: BusinessModel, policy: ExplorationPolicy, config: SearchConfig) -> SearchReport:+ facts = sorted(model.initial_facts)+ positions = {name: 1 << index for index, name in enumerate(facts)}++ def mask(values: dict[str, bool]) -> tuple[int, int]:+ selected = sum(positions[name] for name in values)+ positive = sum(positions[name] for name, value in values.items() if value)+ return selected, positive++ initial_bits = sum(positions[name] for name, value in model.initial_facts.items() if value)+ limits: dict[str, int] = {}+ for rule in policy.budgets:+ pool = "trace" if rule.scope == "trace" else f"actor:{rule.actor_id}"+ limits[pool] = min(limits.get(pool, rule.limit_units), rule.limit_units)+ pools = sorted(limits)+ pool_limits = tuple(limits[pool] for pool in pools)+ actions = sorted(model.actions, key=lambda action: action.action_id)+ permissions = [tuple(index for index, rule in enumerate(policy.permissions)+ if rule.role in model.actors[action.actor_id].roles+ and rule.tool == action.tool+ and scope_matches(rule.resource_scope, action.resource)) for action in actions]+ guards = [mask(action.requires) for action in actions]+ effects = [mask(action.effects) for action in actions]+ charges = [tuple(index for index, pool in enumerate(pools)+ if pool == "trace" or pool == f"actor:{action.actor_id}") for action in actions]+ properties = sorted(model.properties, key=lambda prop: prop.property_id)+ goals = sorted(model.goals, key=lambda goal: goal.goal_id)+ prop_masks = [mask(prop.forbidden) for prop in properties]+ goal_masks = [mask(goal.requires) for goal in goals]+ prop_nodes: dict[str, int] = {}+ goal_nodes: dict[str, int] = {}+ nodes = [_Node(initial_bits, (0,) * len(pools), 0, None, None)]+ seen = {(initial_bits, nodes[0].counters): 0}+ pending = deque([0])++ def objectives(node_index: int) -> None:+ bits = nodes[node_index].bits+ for prop, (selected, positive) in zip(properties, prop_masks, strict=True):+ if prop.property_id not in prop_nodes and bits & selected == positive:+ prop_nodes[prop.property_id] = node_index+ for goal, (selected, positive) in zip(goals, goal_masks, strict=True):+ if goal.goal_id not in goal_nodes and bits & selected == positive:+ goal_nodes[goal.goal_id] = node_index++ objectives(0)+ examined = enabled = expanded = 0+ observed: set[str] = set()+ cutoffs: set[str] = set()+ halted = False+ while pending and not halted:+ node_index = pending.popleft()+ node = nodes[node_index]+ for action_index, action in enumerate(actions):+ if examined >= config.max_transitions:+ cutoffs.add("MAX_TRANSITIONS")+ halted = True+ break+ examined += 1+ selected, positive = guards[action_index]+ if not permissions[action_index] or node.bits & selected != positive:+ continue+ counters = list(node.counters)+ if any(counters[index] + action.cost_units > pool_limits[index] for index in charges[action_index]):+ continue+ for index in charges[action_index]:+ counters[index] += action.cost_units+ effect_selected, effect_positive = effects[action_index]+ bits = (node.bits & ~effect_selected) | effect_positive+ next_counters = tuple(counters)+ enabled += 1+ observed.add(action.action_id)+ key = (bits, next_counters)+ if key in seen:+ continue+ if node.depth >= config.max_depth:+ cutoffs.add("MAX_DEPTH")+ continue+ if len(nodes) >= config.max_states:+ cutoffs.add("MAX_STATES")+ halted = True+ break+ next_index = len(nodes)+ seen[key] = next_index+ nodes.append(_Node(bits, next_counters, node.depth + 1, node_index, action_index))+ pending.append(next_index)+ objectives(next_index)+ if not halted:+ expanded += 1+ complete = not pending and not cutoffs and not halted+ issues = ("NO_ENABLED_TRANSITIONS",) if enabled == 0 else ()++ def snapshot(node: _Node) -> StateSnapshot:+ return StateSnapshot(facts={name: bool(node.bits & positions[name]) for name in facts},+ budgets=dict(zip(pools, node.counters, strict=True)))++ def witness(objective_id: str, kind: str, node_index: int) -> Witness:+ chain: list[int] = []+ cursor = node_index+ while nodes[cursor].parent is not None:+ chain.append(cursor)+ parent = nodes[cursor].parent+ assert parent is not None+ cursor = parent+ chain.reverse()+ steps = []+ for index, child_index in enumerate(chain, 1):+ child = nodes[child_index]+ assert child.parent is not None and child.action is not None+ action = actions[child.action]+ steps.append(WitnessStep(index=index, action_id=action.action_id,+ actor_id=action.actor_id, tool=action.tool, resource=action.resource,+ cost_units=action.cost_units, permission_indices=permissions[child.action],+ before=snapshot(nodes[child.parent]), after=snapshot(child)))+ return Witness.model_validate({"objective_id": objective_id, "kind": kind,+ "initial": snapshot(nodes[0]), "terminal": snapshot(nodes[node_index]),+ "steps": tuple(steps)})++ safety = tuple(witness(prop_id, "safety", index) for prop_id, index in sorted(prop_nodes.items()))+ legitimate = tuple(witness(goal_id, "goal", index) for goal_id, index in sorted(goal_nodes.items()))+ property_status = {prop.property_id: "COUNTEREXAMPLE" if prop.property_id in prop_nodes+ else "VERIFIED" if complete else "INCONCLUSIVE" for prop in properties}+ goal_status = {goal.goal_id: "REACHABLE" if goal.goal_id in goal_nodes+ else "UNREACHABLE" if complete else "UNKNOWN" for goal in goals}+ decision = ("COUNTEREXAMPLE" if safety else+ "GOAL_BLOCKED" if complete and any(value == "UNREACHABLE" for value in goal_status.values()) else+ "VERIFIED" if complete and not issues else "INCONCLUSIVE")+ return SearchReport.model_validate(dict(+ decision=decision, property_status=property_status, goal_status=goal_status,+ safety_witnesses=safety, goal_witnesses=legitimate, states_discovered=len(nodes),+ states_expanded=expanded, transitions_examined=examined, transitions_enabled=enabled,+ frontier_exhausted=complete, cutoff_reasons=tuple(sorted(cutoffs)), coverage_issues=issues,+ not_observed_action_ids=tuple(action.action_id for action in actions if action.action_id not in observed),+ model_digest=canonical_digest(model), policy_digest=canonical_digest(policy),+ config_digest=canonical_digest(config), limitations=LIMITATIONS,+ ))+++def explore_policy(model: BusinessModel, policy: ExplorationPolicy, config: SearchConfig = SearchConfig()) -> SearchReport:+ """Explore the complete reachable frontier, or report explicit coverage limits."""+ return _explore(*snapshot_inputs(model, policy, config))diff --git a/ifixai/reporting/policy_lab.py b/ifixai/reporting/policy_lab.pynew file mode 100644index 0000000..c4d2c57--- /dev/null+++ b/ifixai/reporting/policy_lab.py@@ -0,0 +1,126 @@+"""Private, standalone reports of model reachability and tested policy repairs."""+from __future__ import annotations++import html+import json+import os+from pathlib import Path+from typing import Any++from ifixai.policy_lab.models import ExplorationBundle, Witness+from ifixai.policy_lab.repair import RepairReport+++def compare_policies(original: ExplorationBundle, candidate: ExplorationBundle) -> dict[str, Any]:+ if original.report.model_digest != candidate.report.model_digest:+ raise ValueError("policy comparison requires the exact same business model")+ if original.report.config_digest != candidate.report.config_digest:+ raise ValueError("policy comparison requires the exact same search limits")+ return {"original_bundle_digest": original.integrity_digest,+ "original_policy_digest": original.report.policy_digest,+ "candidate_policy_digest": candidate.report.policy_digest,+ "model_digest": candidate.report.model_digest,+ "policy_changed": original.report.policy_digest != candidate.report.policy_digest,+ "original_decision": original.report.decision, "candidate_decision": candidate.report.decision,+ "properties": {key: {"before": status, "after": candidate.report.property_status[key]}+ for key, status in original.report.property_status.items()},+ "goals": {key: {"before": status, "after": candidate.report.goal_status[key]}+ for key, status in original.report.goal_status.items()},+ "limitation": "Rechecks an explicit model under another policy; does not apply changes to a deployed system."}+++def _json(value: Any) -> str:+ return json.dumps(value, indent=2, ensure_ascii=False, allow_nan=False) + "\n"+++def render_policy_lab_html(bundle: ExplorationBundle, *, comparison: dict[str, Any] | None = None, repair: RepairReport | None = None) -> str:+ escape = html.escape+ report = bundle.report++ def witness_card(witness: Witness, ordinal: int) -> str:+ steps = []+ for step in witness.steps:+ changes = ", ".join(f"{key}: {str(step.before.facts[key]).lower()} → {str(value).lower()}"+ for key, value in step.after.facts.items() if step.before.facts[key] != value)+ ledger = ", ".join(f"{key}={value}" for key, value in step.after.budgets.items())+ steps.append(f'<li><b>{escape(step.action_id)}</b><p>{escape(step.actor_id)} → {escape(step.tool)} · {escape(step.resource)}</p><p class="change">{escape(changes) or "Facts unchanged"}</p><small>Permission rules: {escape(str(step.permission_indices))} · cost {step.cost_units} units · {escape(ledger) or "No budget pool configured"}</small></li>')+ no_steps = '<li>Zero-step witness: the initial state already satisfies the objective.</li>'+ terminal = ", ".join(f"{key}={str(value).lower()}" for key, value in witness.terminal.facts.items())+ return f'<article class="witness" id="witness-{ordinal}"><h3>{escape(witness.kind)} · {escape(witness.objective_id)}</h3><p>{len(witness.steps)} steps · shortest discovered path</p><ol class="path">{"".join(steps) or no_steps}</ol><details><summary>Terminal fact snapshot</summary><p>{escape(terminal)}</p></details></article>'++ witnesses = [*report.safety_witnesses, *report.goal_witnesses]+ witness_links = {(w.kind, w.objective_id): f"#witness-{index}" for index, w in enumerate(witnesses)}+ properties = []+ for prop in bundle.model.properties:+ href = witness_links.get(("safety", prop.property_id))+ link = f'<a href="{href}">Counterexample</a>' if href else "—"+ properties.append(f'<tr><td><b>{escape(prop.property_id)}</b><br>{escape(prop.label)}</td><td>{escape(report.property_status[prop.property_id])}</td><td>{link}</td></tr>')+ goals = []+ for goal in bundle.model.goals:+ href = witness_links.get(("goal", goal.goal_id))+ link = f'<a href="{href}">Completion path</a>' if href else "—"+ goals.append(f'<tr><td><b>{escape(goal.goal_id)}</b><br>{escape(goal.label)}</td><td>{escape(report.goal_status[goal.goal_id])}</td><td>{link}</td></tr>')+ scenario = f'<section><h2>Policy comparison</h2><pre>{escape(_json(comparison))}</pre></section>' if comparison else ""+ repair_section = ""+ if repair:+ candidate_rows = "".join(f'<tr><td>{escape(str(item.removed_rule_indices))}</td><td>{escape(item.report.decision)}</td><td>{str(item.report.frontier_exhausted).lower()}</td></tr>' for item in repair.candidates)+ selected = ""+ if repair.selected_bundle:+ selection = repair.selected_bundle+ selected = f'<p class="result">Selected policy: {selection.report.decision} · all {len(selection.report.goal_status)} declared goals REACHABLE · frontier exhausted</p><p>Rule indices refer to the original ordered permission list. Candidate bundle is retained in selected-bundle.json.</p>'+ repair_section = f'<section><h2>Goal-preserving permission repair</h2><p class="result">{escape(repair.status)} · {len(repair.candidates)} candidates checked</p>{selected}<p>Minimum removal cardinality proven: {str(repair.minimality_proven).lower()}. Search allowance: {repair.max_removals} removals, {repair.max_candidates} candidates, {repair.transition_budget} transition examinations.</p><table><thead><tr><th>Removed rule indices</th><th>Decision</th><th>Complete frontier</th></tr></thead><tbody>{candidate_rows}</tbody></table><p>These are evaluated model changes. No deployed permission has been changed.</p></section>'+ gaps = [*report.cutoff_reasons, *report.coverage_issues]+ return f'''<!doctype html><html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><meta http-equiv="Content-Security-Policy" content="default-src 'none'; style-src 'unsafe-inline'; script-src 'unsafe-inline'; base-uri 'none'; form-action 'none'"><title>iFixAi · Business Policy Lab</title><style>+*{{box-sizing:border-box}}body{{margin:0;background:#0b1423;color:#e6edf5;font:16px/1.6 system-ui,sans-serif}}main{{max-width:1100px;margin:auto;padding:48px 24px}}h1{{font-size:42px;line-height:1.15}}h2{{margin-top:0}}small,.eyebrow{{color:#a8b8cd}}section{{padding:24px;margin:20px 0;background:#122239;border:1px solid #294461;border-radius:14px}}.result{{font-weight:700;color:#8ee2cc}}.grid{{display:grid;grid-template-columns:repeat(auto-fit,minmax(200px,1fr));gap:16px}}table{{width:100%;border-collapse:collapse}}th,td{{padding:12px;text-align:left;border-bottom:1px solid #294461;vertical-align:top;overflow-wrap:anywhere}}a{{color:#91caff}}pre{{white-space:pre-wrap;overflow-wrap:anywhere;font:13px/1.5 ui-monospace,monospace}}.path{{list-style:none;padding:0}}.path li{{border-left:3px solid #5cc3ab;padding:12px 18px;margin:10px 0;background:#0c1a2d}}.change{{color:#a0dfbb}}article{{padding:12px 0;margin:14px 0}}article:target{{outline:2px solid #91caff;padding:16px;border-radius:8px}}input{{width:100%;padding:12px;background:#0b1423;color:#fff;border:1px solid #7394b5}}details p{{overflow-wrap:anywhere}}+</style></head><body><main><p class="eyebrow">IFIXAI / PROSPECTIVE BUSINESS ASSURANCE</p><h1>Business Policy Lab</h1><p class="result">{escape(report.decision)}</p><p>{escape(bundle.model.name)} · policy {escape(bundle.policy.name)}</p><p>Searches explicit business contracts for harmful compositions of individually permitted actions. Conclusions apply to this declared model; implementation behavior is verified separately.</p>+<section class="grid"><div><b>{report.states_discovered} distinct states</b><p>{report.states_expanded} fully expanded</p></div><div><b>{report.transitions_examined} transition examinations</b><p>{report.transitions_enabled} enabled edges</p></div><div><b>{len(report.safety_witnesses)} harm witnesses</b><p>{len(report.goal_witnesses)} legitimate completion witnesses</p></div></section>+<section><h2>Safety properties</h2><table><thead><tr><th>Forbidden condition</th><th>Model verdict</th><th>Evidence</th></tr></thead><tbody>{''.join(properties)}</tbody></table></section>+<section><h2>Legitimate workflows</h2><p>Removing every permission is not an accepted repair. Each declared completion goal must remain reachable.</p><table><thead><tr><th>Goal</th><th>Reachability</th><th>Evidence</th></tr></thead><tbody>{''.join(goals)}</tbody></table></section>+{repair_section}{scenario}<section><h2>Shortest discovered paths</h2><input id="search" aria-label="Search witness paths" placeholder="Filter by actor, tool, action or fact"><div id="witnesses">{''.join(witness_card(w,index) for index,w in enumerate(witnesses))}</div></section>+<section><h2>Exploration coverage</h2><p>Frontier exhausted: <b>{str(report.frontier_exhausted).lower()}</b>. Limits: depth {bundle.config.max_depth}, states {bundle.config.max_states}, examinations {bundle.config.max_transitions}.</p><ul>{''.join('<li>'+escape(gap)+'</li>' for gap in gaps) or '<li>No retained search cutoff or coverage issue.</li>'}</ul><p>Actions not observed enabled: {escape(', '.join(report.not_observed_action_ids)) or 'none'}.</p></section>+<section><h2>Portable evidence and boundaries</h2><pre>Bundle SHA-256: {bundle.integrity_digest}\nModel SHA-256: {report.model_digest}\nPolicy SHA-256: {report.policy_digest}</pre><ul>{''.join('<li>'+escape(limit)+'</li>' for limit in report.limitations)}</ul><p>Effects, costs and actors are explicit declarations. A checksum provides integrity, not truth or completeness of the business model.</p></section></main><script>const search=document.getElementById('search');const paths=Array.from(document.querySelectorAll('.witness'));search.addEventListener('input',()=>{{const q=search.value.toLowerCase();paths.forEach(path=>path.hidden=!path.textContent.toLowerCase().includes(q));}});function revealWitness(hash){{const target=document.getElementById(hash.slice(1));if(target&&target.classList.contains('witness')){{target.hidden=false;target.scrollIntoView();}}}}window.addEventListener('hashchange',()=>revealWitness(location.hash));document.querySelectorAll('a[href^="#witness-"]').forEach(link=>link.addEventListener('click',()=>revealWitness(link.getAttribute('href'))));revealWitness(location.hash);</script></body></html>'''+++def render_policy_lab_markdown(bundle: ExplorationBundle) -> str:+ def safe(value: str) -> str:+ for char in ("\\", "`", "*", "_", "[", "]", "<", ">", "|", "#"):+ value = value.replace(char, "\\" + char)+ return value.replace("\n", " ").replace("\r", " ")++ report = bundle.report+ lines = ["# Business Policy Lab", "", f"Decision: **{report.decision}**", "",+ f"Model: {safe(bundle.model.name)}. Policy: {safe(bundle.policy.name)}.", "",+ f"States: {report.states_discovered}. Examinations: {report.transitions_examined}. Complete frontier: {report.frontier_exhausted}.", "",+ "## Safety and completion", ""]+ lines.extend(f"- Property {safe(key)}: {status}" for key, status in report.property_status.items())+ lines.extend(f"- Goal {safe(key)}: {status}" for key, status in report.goal_status.items())+ for witness in (*report.safety_witnesses, *report.goal_witnesses):+ lines.extend(["", f"## {safe(witness.kind)}: {safe(witness.objective_id)}", ""])+ lines.extend(f"{step.index}. {safe(step.actor_id)} → {safe(step.action_id)} ({safe(step.tool)})." for step in witness.steps)+ if not witness.steps:+ lines.append("Initial state already matches the objective.")+ lines.extend(["", "## Boundaries", ""])+ lines.extend(f"- {safe(limit)}" for limit in report.limitations)+ lines.extend(["", f"Bundle SHA-256: `{bundle.integrity_digest}`", ""])+ return "\n".join(lines)+++def write_policy_lab_reports(bundle: ExplorationBundle, output: str | Path, *, comparison: dict[str, Any] | None = None, repair: RepairReport | None = None, metadata: dict[str, Any] | None = None) -> Path:+ destination = Path(output)+ destination.mkdir(mode=0o700, exist_ok=False)+ files = {"bundle.json": _json(bundle.model_dump(mode="json")), "report.json": _json(bundle.report.model_dump(mode="json")),+ "report.html": render_policy_lab_html(bundle, comparison=comparison, repair=repair),+ "report.md": render_policy_lab_markdown(bundle)}+ if comparison is not None:+ files["comparison.json"] = _json(comparison)+ if repair is not None:+ files["repair.json"] = _json(repair.model_dump(mode="json"))+ if repair.selected_bundle is not None:+ files["selected-bundle.json"] = _json(repair.selected_bundle.model_dump(mode="json"))+ if metadata is not None:+ files["metadata.json"] = _json(metadata)+ for name, content in files.items():+ descriptor = os.open(destination / name, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)+ with os.fdopen(descriptor, "w", encoding="utf-8") as handle:+ handle.write(content)+ return destinationdiff --git a/ifixai/tests/policy_lab/conftest.py b/ifixai/tests/policy_lab/conftest.pynew file mode 100644index 0000000..cae4529--- /dev/null+++ b/ifixai/tests/policy_lab/conftest.py@@ -0,0 +1,15 @@+import importlib.util+from pathlib import Path++import pytest++_PATH = Path(__file__).resolve().parents[3] / "examples" / "business_policy_lab.py"+_SPEC = importlib.util.spec_from_file_location("business_lab_example", _PATH)+assert _SPEC is not None and _SPEC.loader is not None+example = importlib.util.module_from_spec(_SPEC)+_SPEC.loader.exec_module(example)+++def native_example():+ return examplediff --git a/ifixai/tests/policy_lab/test_evidence_and_repairs.py b/ifixai/tests/policy_lab/test_evidence_and_repairs.pynew file mode 100644index 0000000..8978506--- /dev/null+++ b/ifixai/tests/policy_lab/test_evidence_and_repairs.py@@ -0,0 +1,215 @@+"""Adversarial declarations, strict replay and goal-preserving repair oracles."""++import json++import pytest+from pydantic import ValidationError++from ifixai.policy_lab import (+ canonical_digest,+ create_exploration_bundle,+ explore_policy,+ load_exploration_bundle,+)+from ifixai.policy_lab.executor import execute_witness+from ifixai.policy_lab.models import (+ ActionContract,+ Actor,+ BudgetRule,+ BusinessGoal,+ BusinessModel,+ ExplorationPolicy,+ PermissionRule,+ SafetyProperty,+ SearchConfig,+)+from ifixai.policy_lab.repair import search_repairs++[email protected]("field,value", [("max_depth", True), ("max_states", 1.0), ("max_transitions", "10")])+def test_copied_config_cannot_coerce_strict_counters(native_example, field, value):+ config = SearchConfig().model_copy(update={field: value})+ with pytest.raises((ValidationError, ValueError)):+ explore_policy(native_example.demo_model(), native_example.demo_policy(), config)++[email protected]("mutation", ["fact", "cost", "budget"])+def test_copied_nested_declarations_revalidated(native_example, mutation):+ model, policy = native_example.demo_model(), native_example.demo_policy()+ if mutation == "fact":+ model = model.model_copy(update={"initial_facts": {**model.initial_facts, "paid": 1}})+ elif mutation == "cost":+ action = model.actions[0].model_copy(update={"cost_units": True})+ model = model.model_copy(update={"actions": (action, *model.actions[1:])})+ else:+ policy = policy.model_copy(update={"budgets": (policy.budgets[0].model_copy(update={"limit_units": True}),)})+ with pytest.raises((ValidationError, ValueError)):+ create_exploration_bundle(model, policy)++[email protected]("identifier", [" agent", "agent ", " ", ""])+def test_whitespace_identity_collisions_rejected(native_example, identifier):+ model = native_example.demo_model().model_dump(mode="python")+ model["actors"][identifier] = model["actors"]["agent"]+ with pytest.raises(ValidationError):+ type(native_example.demo_model()).model_validate(model)++[email protected]("scope", ["account:*:bad", "**", "account:**"])+def test_only_one_terminal_resource_wildcard(scope):+ with pytest.raises(ValidationError):+ PermissionRule(role="agent", tool="pay", resource_scope=scope)+++def test_tool_wildcards_never_grant_ambient_authority():+ with pytest.raises(ValidationError):+ PermissionRule(role="agent", tool="*")+++def test_exact_resource_scope_and_tenant_boundary(native_example):+ policy = native_example.demo_policy()+ denied = policy.model_copy(update={"permissions": tuple(rule.model_copy(update={"resource_scope": "account:other"}) for rule in policy.permissions)})+ report = explore_policy(native_example.demo_model(), denied)+ assert report.decision == "GOAL_BLOCKED" and report.transitions_enabled == 0+ with pytest.raises(ValueError):+ explore_policy(native_example.demo_model(), policy.model_copy(update={"tenant_id": "other"}))+++def test_duplicate_budget_constraints_equivalent_to_strictest(native_example):+ model, policy = native_example.demo_model(), native_example.demo_policy()+ duplicate = policy.model_copy(update={"budgets": (BudgetRule(scope="trace", limit_units=9), BudgetRule(scope="trace", limit_units=2), BudgetRule(scope="trace", limit_units=3))})+ single = policy.model_copy(update={"budgets": (BudgetRule(scope="trace", limit_units=2),)})+ left, right = explore_policy(model, duplicate), explore_policy(model, single)+ for field in ("decision", "states_discovered", "transitions_enabled", "goal_status", "property_status", "goal_witnesses"):+ assert getattr(left, field) == getattr(right, field)+++def test_exact_large_integer_cost_has_no_rounding(native_example):+ model, policy = native_example.demo_model(), native_example.demo_policy()+ amount = 2**63 - 1+ action = model.actions[0].model_copy(update={"cost_units": amount})+ model = model.model_copy(update={"actions": (action, *model.actions[1:])})+ policy = policy.model_copy(update={"budgets": (BudgetRule(scope="trace", limit_units=amount),)})+ report = explore_policy(model, policy)+ assert report.states_discovered == 2 and report.transitions_enabled == 1+ assert report.decision == "GOAL_BLOCKED"+++def test_actor_pools_charge_only_the_actor_and_share_trace_pool(native_example):+ model, policy = native_example.demo_model(), native_example.demo_policy()+ policy = policy.model_copy(update={"budgets": (BudgetRule(scope="trace", limit_units=3), BudgetRule(scope="actor", actor_id="agent", limit_units=2), BudgetRule(scope="actor", actor_id="reviewer", limit_units=1))})+ report = explore_policy(model, policy)+ assert report.decision == "VERIFIED" and not report.safety_witnesses+ witness = report.goal_witnesses[0]+ assert witness.terminal.budgets == {"trace": 3, "actor:agent": 2, "actor:reviewer": 1}+ assert witness.steps[1].after.budgets == {"trace": 2, "actor:agent": 1, "actor:reviewer": 1}+ unknown = policy.model_copy(update={"budgets": (BudgetRule(scope="actor", actor_id="unknown", limit_units=1),)})+ with pytest.raises(ValueError):+ explore_policy(model, unknown)+++def test_repair_removes_self_approval_but_preserves_native_goal(native_example):+ bundle = create_exploration_bundle(native_example.demo_model(), native_example.demo_policy())+ repair = search_repairs(bundle)+ assert repair.status == "FOUND" and repair.minimality_proven+ assert [candidate.removed_rule_indices for candidate in repair.candidates] == [(0,), (1,)]+ assert repair.candidates[0].report.decision == "GOAL_BLOCKED"+ selected = repair.selected_bundle+ assert selected is not None and selected.report.decision == "VERIFIED"+ assert selected.model == bundle.model and selected.policy.budgets == bundle.policy.budgets+ assert selected.report.goal_status == {"legitimate_refund": "REACHABLE"}+ store = native_example.LocalRefundStore()+ execution = execute_witness(selected, selected.report.goal_witnesses[0], store.bindings(), store.reset, store.observe)+ assert execution.decision == "MATCHED"+ assert store.calls == ["01_request", "03_independent_approve", "04_pay"]+ assert store.synthetic_paid_units == 60 and store.facts["independent"]+++def test_repair_never_accepts_block_everything_or_initial_harm(native_example):+ model, policy = native_example.demo_model(), native_example.demo_policy()+ model = model.model_copy(update={"initial_facts": {**model.initial_facts, "paid": True}})+ result = search_repairs(create_exploration_bundle(model, policy), max_removals=2)+ assert result.status == "EXHAUSTED" and result.selected_bundle is None+ assert all(candidate.report.decision == "COUNTEREXAMPLE" for candidate in result.candidates)++[email protected]("limits", [{"max_candidates": 1}, {"transition_budget": 1}])+def test_repair_global_limits_are_conservative(native_example, limits):+ bundle = create_exploration_bundle(native_example.demo_model(), native_example.demo_policy())+ result = search_repairs(bundle, **limits)+ assert result.status == "INCONCLUSIVE" and not result.minimality_proven+ assert result.selected_bundle is None and result.cutoff_reasons+ if "transition_budget" in limits:+ assert result.transitions_examined <= limits["transition_budget"]++[email protected]("mutation,rehash", [("report", False), ("report", True), ("witness", True), ("missing_default", True), ("extra", True)])+def test_replay_rejects_tamper_even_after_rehash(native_example, tmp_path, mutation, rehash):+ bundle = create_exploration_bundle(native_example.demo_model(), native_example.demo_policy())+ raw = bundle.model_dump(mode="json")+ if mutation == "report":+ raw["report"]["states_discovered"] += 1+ elif mutation == "witness":+ raw["report"]["safety_witnesses"][0]["steps"][0]["actor_id"] = "reviewer"+ elif mutation == "missing_default":+ del raw["model"]["schema_version"]+ else:+ raw["imports"] = ["os.system"]+ if rehash:+ raw["integrity_digest"] = canonical_digest({key: value for key, value in raw.items() if key != "integrity_digest"})+ path = tmp_path / "tamper.json"+ path.write_text(json.dumps(raw))+ with pytest.raises((ValueError, ValidationError)):+ load_exploration_bundle(path)++[email protected]("raw", ['{"schema_version":"1","schema_version":"1"}', '{"x":NaN}', '{"x":Infinity}', '\xff'])+def test_replay_rejects_ambiguous_or_nonfinite_json(tmp_path, raw):+ path = tmp_path / "malformed.json"+ path.write_bytes(raw.encode("latin-1"))+ with pytest.raises((ValueError, UnicodeError)):+ load_exploration_bundle(path)+++def test_native_replay_round_trip_exact(native_example, tmp_path):+ bundle = create_exploration_bundle(native_example.demo_model(), native_example.demo_policy())+ path = tmp_path / "bundle.json"+ path.write_text(bundle.model_dump_json())+ assert load_exploration_bundle(path) == bundle+++def test_repair_found_after_unknown_smaller_cardinality_never_claims_minimal():+ model = BusinessModel(name="Incomplete smaller repair", tenant_id="tenant", initial_facts={"bad": False, "stage": False, "stage2": False, "done": False}, actors={"actor": Actor(roles=("worker",))}, actions=(+ ActionContract(action_id="a_harm", actor_id="actor", tool="harm", resource="one", effects={"bad": True}),+ ActionContract(action_id="b_stage", actor_id="actor", tool="advance", resource="one", requires={"stage": False}, effects={"stage": True}),+ ActionContract(action_id="c_finish", actor_id="actor", tool="finish", resource="one", requires={"done": False}, effects={"done": True}),+ ActionContract(action_id="d_deep", actor_id="actor", tool="advance", resource="one", requires={"stage": True, "stage2": False}, effects={"stage2": True}),+ ), properties=(SafetyProperty(property_id="no_harm", forbidden={"bad": True}),), goals=(BusinessGoal(goal_id="completion", requires={"done": True}),))+ policy = ExplorationPolicy(name="Three permissions", tenant_id="tenant", permissions=tuple(PermissionRule(role="worker", tool=tool) for tool in ("harm", "advance", "finish")))+ bundle = create_exploration_bundle(model, policy, SearchConfig(max_depth=1))+ assert bundle.report.decision == "COUNTEREXAMPLE" and not bundle.report.frontier_exhausted+ result = search_repairs(bundle, max_removals=2)+ assert result.status == "FOUND" and not result.minimality_proven+ assert result.candidates[0].removed_rule_indices == (0,)+ assert result.candidates[0].report.decision == "INCONCLUSIVE"+ assert result.candidates[-1].removed_rule_indices == (0, 1)+ assert result.selected_bundle is not None and result.selected_bundle.report.decision == "VERIFIED"+++def test_repair_batch_retains_compact_summaries_without_candidate_witnesses():+ length = 24+ model = BusinessModel(name="Long multiobjective evidence", tenant_id="tenant", initial_facts={f"s{i}": False for i in range(length)}, actors={"actor": Actor(roles=("worker",))}, actions=tuple(+ ActionContract(action_id=f"a{i:02}", actor_id="actor", tool="advance", resource="chain", requires={f"s{i}": False, **({f"s{i-1}": True} if i else {})}, effects={f"s{i}": True}) for i in range(length)+ ), properties=tuple(SafetyProperty(property_id=f"p{i}", forbidden={f"s{length-1}": True}) for i in range(32)), goals=tuple(BusinessGoal(goal_id=f"g{i}", requires={"s0": True}) for i in range(32)))+ policy = ExplorationPolicy(name="Duplicate rule candidates", tenant_id="tenant", permissions=tuple(PermissionRule(role="worker", tool="advance") for _ in range(64)))+ original = create_exploration_bundle(model, policy, SearchConfig(max_depth=length))+ assert len(original.report.safety_witnesses) == 32+ assert all(len(w.steps) == length for w in original.report.safety_witnesses)+ repair = search_repairs(original, max_candidates=64)+ assert repair.status == "EXHAUSTED" and len(repair.candidates) == 64+ for candidate in repair.candidates:+ raw = candidate.report.model_dump(mode="json")+ assert "safety_witnesses" not in raw and "goal_witnesses" not in raw+ assert len(raw["report_digest"]) == len(raw["bundle_digest"]) == 64+ assert repair.selected_bundle is None+ assert len(repair.model_dump_json().encode()) < 300000diff --git a/ifixai/tests/policy_lab/test_exhaustive_oracle.py b/ifixai/tests/policy_lab/test_exhaustive_oracle.pynew file mode 100644index 0000000..663f5f3--- /dev/null+++ b/ifixai/tests/policy_lab/test_exhaustive_oracle.py@@ -0,0 +1,124 @@+"""Independent primitive-state oracle; never calls implementation helpers."""++from collections import deque+from itertools import product++import pytest++from ifixai.policy_lab import explore_policy+from ifixai.policy_lab.models import (+ ActionContract,+ Actor,+ BudgetRule,+ BusinessGoal,+ BusinessModel,+ ExplorationPolicy,+ PermissionRule,+ SafetyProperty,+ SearchConfig,+)+++def primitive_oracle(allowed, costs, limit):+ initial = (False, False, 0)+ queue = deque([(initial, 0)])+ seen = {initial}+ unsafe = goal = None+ enabled = 0+ # independent transition declarations (guard/effect lambdas), no engine APIs+ operations = (+ (lambda x, _y: not x, lambda _x, y: (True, y)),+ (lambda x, _y: x, lambda _x, y: (False, y)),+ (lambda x, y: x and not y, lambda x, _y: (x, True)),+ )+ while queue:+ (x, y, spent), depth = queue.popleft()+ if x and y and unsafe is None:+ unsafe = depth+ if y and goal is None:+ goal = depth+ for i, (guard, effect) in enumerate(operations):+ if not allowed[i] or not guard(x, y) or (limit is not None and spent + costs[i] > limit):+ continue+ enabled += 1+ next_x, next_y = effect(x, y)+ candidate = (next_x, next_y, spent + costs[i] if limit is not None else 0)+ if candidate not in seen:+ seen.add(candidate)+ queue.append((candidate, depth + 1))+ return len(seen), enabled, unsafe, goal+++def tiny_contract(allowed, costs, limit):+ model = BusinessModel(name="Tiny exhaustive oracle", tenant_id="tenant", initial_facts={"x": False, "y": False}, actors={"actor": Actor(roles=("worker",))}, actions=(+ ActionContract(action_id="a", actor_id="actor", tool="set_x", resource="object:one", requires={"x": False}, effects={"x": True}, cost_units=costs[0]),+ ActionContract(action_id="b", actor_id="actor", tool="clear_x", resource="object:one", requires={"x": True}, effects={"x": False}, cost_units=costs[1]),+ ActionContract(action_id="c", actor_id="actor", tool="set_y", resource="object:one", requires={"x": True, "y": False}, effects={"y": True}, cost_units=costs[2]),+ ), properties=(SafetyProperty(property_id="unsafe", forbidden={"x": True, "y": True}),), goals=(BusinessGoal(goal_id="goal", requires={"y": True}),))+ tools = ("set_x", "clear_x", "set_y")+ policy = ExplorationPolicy(name="Tiny policy", tenant_id="tenant", permissions=tuple(PermissionRule(role="worker", tool=tool) for enabled, tool in zip(allowed, tools, strict=True) if enabled), budgets=() if limit is None else (BudgetRule(scope="trace", limit_units=limit),))+ return model, policy+++def test_bfs_agrees_with_independent_exhaustive_oracle_across_120_policies():+ compared = 0+ for allowed, costs, limit in product(product((False, True), repeat=3), ((0, 0, 0), (1, 1, 1), (2, 1, 1)), (None, 0, 1, 2, 3)):+ expected_states, expected_edges, unsafe_depth, goal_depth = primitive_oracle(allowed, costs, limit)+ model, policy = tiny_contract(allowed, costs, limit)+ report = explore_policy(model, policy, SearchConfig(max_depth=64))+ assert report.frontier_exhausted+ assert report.states_discovered == expected_states+ assert report.transitions_enabled == expected_edges+ assert report.property_status["unsafe"] == ("COUNTEREXAMPLE" if unsafe_depth is not None else "VERIFIED")+ assert report.goal_status["goal"] == ("REACHABLE" if goal_depth is not None else "UNREACHABLE")+ assert [len(w.steps) for w in report.safety_witnesses] == ([] if unsafe_depth is None else [unsafe_depth])+ assert [len(w.steps) for w in report.goal_witnesses] == ([] if goal_depth is None else [goal_depth])+ if expected_edges == 0:+ assert report.decision != "VERIFIED"+ compared += 1+ assert compared == 120+++def test_budget_counters_are_part_of_visited_identity_even_when_facts_cycle():+ model, policy = tiny_contract((True, True, False), (1, 1, 0), 4)+ report = explore_policy(model, policy)+ assert report.states_discovered == 5+ assert report.transitions_enabled == 4+ assert report.frontier_exhausted++[email protected]("config", [SearchConfig(max_depth=1), SearchConfig(max_states=1), SearchConfig(max_transitions=1)])+def test_finite_limits_do_not_claim_complete_frontier(config):+ model, policy = tiny_contract((True, True, True), (0, 0, 0), None)+ report = explore_policy(model, policy, config)+ assert not report.frontier_exhausted+ assert report.cutoff_reasons+ assert report.decision == "INCONCLUSIVE"+ assert report.property_status["unsafe"] == "INCONCLUSIVE"+++def test_depth_boundary_existing_cycle_is_a_complete_fixed_point():+ model, policy = tiny_contract((True, True, False), (0, 0, 0), None)+ report = explore_policy(model, policy, SearchConfig(max_depth=1))+ assert report.frontier_exhausted+ assert report.states_discovered == 2+ assert not report.cutoff_reasons+ assert report.goal_status["goal"] == "UNREACHABLE"+ assert report.decision == "GOAL_BLOCKED"+++def test_initial_unsafe_has_zero_step_witness(native_example):+ model = native_example.demo_model()+ model = model.model_copy(update={"initial_facts": {**model.initial_facts, "paid": True}})+ report = explore_policy(model, native_example.demo_policy())+ assert report.decision == "COUNTEREXAMPLE"+ assert len(report.safety_witnesses[0].steps) == 0+++def test_simultaneous_effects_preserve_unchanged_facts(native_example):+ report = explore_policy(native_example.demo_model(), native_example.demo_policy())+ witness = report.goal_witnesses[0]+ approved = witness.steps[1]+ assert approved.action_id == "03_independent_approve"+ assert approved.before.facts["requested"] is True+ assert approved.after.facts == {"requested": True, "approved": True, "independent": True, "paid": False}diff --git a/ifixai/tests/policy_lab/test_native_executor.py b/ifixai/tests/policy_lab/test_native_executor.pynew file mode 100644index 0000000..ef3877b--- /dev/null+++ b/ifixai/tests/policy_lab/test_native_executor.py@@ -0,0 +1,176 @@+import pytest+from pydantic import ValidationError++from ifixai.policy_lab import create_exploration_bundle+from ifixai.policy_lab.executor import execute_witness+++def setup(native_example):+ bundle = create_exploration_bundle(native_example.demo_model(), native_example.demo_policy())+ return bundle, bundle.report.safety_witnesses[0], native_example.LocalRefundStore()+++def test_zero_step_initial_harm_is_observed_without_action_dispatch(native_example):+ model = native_example.demo_model()+ model = model.model_copy(update={"initial_facts": {**model.initial_facts, "paid": True}})+ bundle = create_exploration_bundle(model, native_example.demo_policy())+ witness = bundle.report.safety_witnesses[0]+ facts = {}+ def reset():+ facts.update(model.initial_facts)+ report = execute_witness(bundle, witness, {}, reset, lambda: dict(facts))+ assert not witness.steps and report.decision == "MATCHED"+ assert report.attempted_actions == report.completed_actions == 0+ assert report.initial_observed == model.initial_facts and not report.steps+++def test_three_legal_actions_have_real_observed_harm_and_immutable_context(native_example):+ bundle, witness, store = setup(native_example)+ contexts = []+ bindings = store.bindings()+ original = bindings["01_request"]+ def observed(context):+ contexts.append(context)+ with pytest.raises(ValidationError):+ context.actor_id = "different-actor"+ original(context)+ return "return value is not state authority"+ bindings["01_request"] = observed+ report = execute_witness(bundle, witness, bindings, store.reset, store.observe)+ assert report.decision == "MATCHED"+ assert report.attempted_actions == report.completed_actions == 3+ assert store.calls == ["01_request", "02_self_approve", "04_pay"]+ assert store.synthetic_paid_units == 60 and store.facts["independent"] is False+ assert contexts[0].cost_units == 1+ assert len(report.steps) == 3+ assert all(step.outcome == "completed" for step in report.steps)+++def test_after_mismatch_stops_before_any_later_binding(native_example):+ bundle, witness, store = setup(native_example)+ bindings = store.bindings()+ def wrong(context):+ store.calls.append(context.action_id)+ return "claimed success without changing independently observed state"+ bindings["01_request"] = wrong+ report = execute_witness(bundle, witness, bindings, store.reset, store.observe)+ assert report.decision == "MODEL_MISMATCH"+ assert report.attempted_actions == report.completed_actions == 1+ assert store.calls == ["01_request"]+ assert report.steps[0].outcome == "mismatch"++[email protected]("state", [+ {"requested": False},+ {"requested": 0, "approved": False, "independent": False, "paid": False},+ {"requested": False, "approved": False, "independent": False, "paid": False, "extra": False},+])+def test_initial_observer_must_report_all_exact_boolean_facts(native_example, state):+ bundle, witness, store = setup(native_example)+ report = execute_witness(bundle, witness, store.bindings(), store.reset, lambda: state)+ assert report.decision == "MODEL_MISMATCH"+ assert report.attempted_actions == report.completed_actions == 0+ assert not store.calls+++def test_before_observer_drift_stops_before_first_action(native_example):+ bundle, witness, store = setup(native_example)+ calls = 0+ def observer():+ nonlocal calls+ calls += 1+ return store.observe() if calls == 1 else {**store.observe(), "paid": True}+ report = execute_witness(bundle, witness, store.bindings(), store.reset, observer)+ assert report.decision == "MODEL_MISMATCH"+ assert report.attempted_actions == 0 and not store.calls+++def test_failure_retains_attempted_count_and_observed_side_effect_without_secret(native_example):+ bundle, witness, store = setup(native_example)+ bindings = store.bindings()+ def failed(context):+ store.request(context)+ raise RuntimeError("DO-NOT-EXPOSE-SECRET-KEY")+ bindings["01_request"] = failed+ report = execute_witness(bundle, witness, bindings, store.reset, store.observe)+ assert report.decision == "EXECUTION_ERROR"+ assert report.attempted_actions == 1 and report.completed_actions == 0+ assert store.calls == ["01_request"]+ assert report.steps[0].after["requested"] is True+ assert report.steps[0].error_type == "RuntimeError"+ assert "DO-NOT-EXPOSE" not in report.model_dump_json()++[email protected]("kind", ["coroutine", "generator", "async-generator"])+def test_sync_wrapper_deferred_result_is_never_completed(native_example, kind):+ bundle, witness, store = setup(native_example)+ async def coroutine():+ return 1+ def generator():+ yield 1+ async def async_generator():+ yield 1+ returned = {"coroutine": coroutine, "generator": generator, "async-generator": async_generator}[kind]+ bindings = store.bindings()+ bindings["01_request"] = lambda _context: returned()+ report = execute_witness(bundle, witness, bindings, store.reset, store.observe)+ assert report.decision == "EXECUTION_ERROR"+ assert report.attempted_actions == 1 and report.completed_actions == 0+ assert report.steps[0].outcome == "failed"+++def test_prevalidation_all_bindings_missing_or_async_never_resets(native_example):+ bundle, witness, store = setup(native_example)+ resets = []+ class AsyncCallable:+ async def __call__(self, _context):+ raise AssertionError("executed")+ for bindings in ({"01_request": store.request}, {**store.bindings(), "04_pay": AsyncCallable()}):+ report = execute_witness(bundle, witness, bindings, lambda: resets.append(1), store.observe)+ assert report.decision == "INCONCLUSIVE"+ assert report.attempted_actions == 0+ assert not resets and not store.calls++[email protected]("max_steps", [True, 0, 2, 65])+def test_explicit_step_budget_checked_before_reset(native_example, max_steps):+ bundle, witness, store = setup(native_example)+ resets = []+ report = execute_witness(bundle, witness, store.bindings(), lambda: resets.append(1), store.observe, max_steps=max_steps)+ assert report.decision == "INCONCLUSIVE" and not resets+++def test_tampered_bundle_and_unretained_witness_cannot_trigger_reset(native_example):+ bundle, witness, store = setup(native_example)+ resets = []+ tampered = bundle.model_copy(update={"integrity_digest": "0" * 64})+ report = execute_witness(tampered, witness, store.bindings(), lambda: resets.append(1), store.observe)+ assert report.decision == "INCONCLUSIVE"+ alien = witness.model_copy(update={"objective_id": "invented-objective"})+ report = execute_witness(bundle, alien, store.bindings(), lambda: resets.append(1), store.observe)+ assert report.decision == "INCONCLUSIVE"+ assert not resets+++def test_reset_return_value_and_observer_exception_cannot_claim_match(native_example):+ bundle, witness, store = setup(native_example)+ report = execute_witness(bundle, witness, store.bindings(), lambda: "unexpected reset return", store.observe)+ assert report.decision == "EXECUTION_ERROR" and report.attempted_actions == 0+ def failed_observer():+ raise RuntimeError("hidden observer payload")+ report = execute_witness(bundle, witness, store.bindings(), store.reset, failed_observer)+ assert report.decision == "EXECUTION_ERROR"+ assert "hidden observer" not in report.model_dump_json()[email protected]("mutation", ["index_bool", "cost_bool", "nonserializable"])+def test_unvalidated_witness_copy_never_hashes_or_resets(native_example, mutation):+ bundle = create_exploration_bundle(native_example.demo_model(), native_example.demo_policy())+ witness = bundle.report.safety_witnesses[0]+ if mutation == "nonserializable":+ witness = witness.model_copy(update={"objective_id": object()})+ else:+ step = witness.steps[0].model_copy(update={"index" if mutation == "index_bool" else "cost_units": True})+ witness = witness.model_copy(update={"steps": (step, *witness.steps[1:])})+ called = []+ report = execute_witness(bundle, witness, {}, lambda: called.append("reset"), lambda: {})+ assert report.decision == "INCONCLUSIVE" and not called+ assert report.attempted_actions == 0 and report.witness_digest == "0" * 64diff --git a/ifixai/tests/policy_lab/test_state_space_benchmark.py b/ifixai/tests/policy_lab/test_state_space_benchmark.pynew file mode 100644index 0000000..15be95c--- /dev/null+++ b/ifixai/tests/policy_lab/test_state_space_benchmark.py@@ -0,0 +1,75 @@+"""Exact synthetic binary state-space benchmark, independent integer oracle.++Monotone bit-set transitions produce every integer in [0, size). Explicit+Boolean prefix guards encode the upper bound; maximum path depth is bit width.+These local search measurements do not measure production agent latency.+"""++import json+import os+import time+import tracemalloc++import pytest++from ifixai.policy_lab import create_exploration_bundle+from ifixai.policy_lab.models import (+ ActionContract,+ Actor,+ BusinessGoal,+ BusinessModel,+ ExplorationPolicy,+ PermissionRule,+ SafetyProperty,+ SearchConfig,+)+++def binary_state_space(size):+ width = (size - 1).bit_length()+ facts = {f"b{i}": False for i in range(width)}+ facts["unsafe"] = False+ actions = []+ for bit in range(width):+ threshold = size - (1 << bit)+ # x < threshold is a disjoint union of equal-higher-prefix/zero-bit+ # clauses at each one-bit of threshold. Require target bit currently0.+ for pivot in range(width):+ if not threshold & (1 << pivot):+ continue+ guard = {f"b{i}": bool(threshold & (1 << i)) for i in range(pivot + 1, width)}+ guard[f"b{pivot}"] = False+ if guard.get(f"b{bit}") is True:+ continue+ guard[f"b{bit}"] = False+ actions.append(ActionContract(action_id=f"set_{bit:02}_{pivot:02}", actor_id="worker", tool="set_bit", resource="counter:synthetic", requires=guard, effects={f"b{bit}": True}))+ model = BusinessModel(name=f"Synthetic {size} state counter", tenant_id="benchmark", initial_facts=facts, actors={"worker": Actor(roles=("worker",))}, actions=tuple(actions), properties=(SafetyProperty(property_id="safe", forbidden={"unsafe": True}),), goals=(BusinessGoal(goal_id="last", requires={f"b{i}": bool((size - 1) & (1 << i)) for i in range(width)}),))+ policy = ExplorationPolicy(name="Declared bit operations", tenant_id="benchmark", permissions=(PermissionRule(role="worker", tool="set_bit", resource_scope="counter:synthetic"),))+ return model, policy, width++[email protected]("size", [1000, 10000])+def test_exhaustive_binary_state_count_and_edge_oracle(size):+ model, policy, width = binary_state_space(size)+ expected_edges = sum(1 for value in range(size) for bit in range(width) if not value & (1 << bit) and (value | (1 << bit)) < size)+ config = SearchConfig(max_depth=width, max_states=size, max_transitions=2000000)+ tracemalloc.start()+ started = time.perf_counter()+ bundle = create_exploration_bundle(model, policy, config)+ elapsed = time.perf_counter() - started+ _, peak_bytes = tracemalloc.get_traced_memory()+ tracemalloc.stop()+ report = bundle.report+ assert report.decision == "VERIFIED" and report.frontier_exhausted+ assert report.states_discovered == report.states_expanded == size+ assert report.transitions_enabled == expected_edges+ assert report.transitions_examined == size * len(model.actions)+ assert len(report.goal_witnesses[0].steps) == (size - 1).bit_count()+ encoded_bytes = len(bundle.model_dump_json().encode())+ assert encoded_bytes < 100000+ assert peak_bytes < 128 * 1024 * 1024+ destination = os.environ.get("IFIXAI_POLICY_LAB_BENCHMARK_DIR")+ if destination:+ from pathlib import Path+ path = Path(destination) / f"benchmark-{size}.json"+ path.write_text(json.dumps({"synthetic": True, "production_latency": False, "distinct_states": size, "enabled_edges": expected_edges, "action_contracts": len(model.actions), "transition_examinations": report.transitions_examined, "elapsed_seconds_with_tracemalloc": elapsed, "peak_traced_bytes": peak_bytes, "bundle_bytes": encoded_bytes, "frontier_exhausted": report.frontier_exhausted, "decision": report.decision}, indent=2))diff --git a/ifixai/tests/policy_lab/test_workflow.py b/ifixai/tests/policy_lab/test_workflow.pynew file mode 100644index 0000000..ae1d1d4--- /dev/null+++ b/ifixai/tests/policy_lab/test_workflow.py@@ -0,0 +1,140 @@+"""Real CLI, shipped fixture projection and private report boundary checks."""++import json+from pathlib import Path+from types import SimpleNamespace++import pytest+from click.testing import CliRunner++from ifixai.cli.policy_lab import policy_lab+from ifixai.core.fixture_loader import load_fixture+from ifixai.policy_lab import create_exploration_bundle, load_exploration_bundle+from ifixai.policy_lab.fixture import policy_from_fixture+from ifixai.policy_lab.models import ActionContract, Actor+from ifixai.reporting.policy_lab import render_policy_lab_html, write_policy_lab_reports+++def test_check_replay_repair_compare_cli_real_files(native_example, tmp_path):+ runner = CliRunner()+ model, policy = native_example.demo_model(), native_example.demo_policy()+ model_file, policy_file = tmp_path / "model.json", tmp_path / "policy.json"+ model_file.write_text(model.model_dump_json())+ policy_file.write_text(policy.model_dump_json())+ original = tmp_path / "check"+ checked = runner.invoke(policy_lab, ["check", str(model_file), str(policy_file), "--output", str(original)])+ assert checked.exit_code == 1, checked.output+ bundle = load_exploration_bundle(original / "bundle.json")+ assert bundle.report.decision == "COUNTEREXAMPLE"+ replayed = runner.invoke(policy_lab, ["replay", str(original / "bundle.json"), "--output", str(tmp_path / "replay")])+ assert replayed.exit_code == 1, replayed.output+ assert load_exploration_bundle(tmp_path / "replay" / "bundle.json") == bundle+ repaired = runner.invoke(policy_lab, ["repair", str(original / "bundle.json"), "--output", str(tmp_path / "repair")])+ assert repaired.exit_code == 0, repaired.output+ selected = load_exploration_bundle(tmp_path / "repair" / "selected-bundle.json")+ assert selected.report.decision == "VERIFIED"+ candidate_file = tmp_path / "candidate.json"+ candidate_file.write_text(selected.policy.model_dump_json())+ compared = runner.invoke(policy_lab, ["compare", str(original / "bundle.json"), str(candidate_file), "--output", str(tmp_path / "compare")])+ assert compared.exit_code == 0, compared.output+ comparison = json.loads((tmp_path / "compare" / "comparison.json").read_text())+ assert comparison["original_decision"] == "COUNTEREXAMPLE" and comparison["candidate_decision"] == "VERIFIED"+ assert comparison["goals"]["legitimate_refund"] == {"before": "REACHABLE", "after": "REACHABLE"}+ assert load_exploration_bundle(original / "bundle.json") == bundle+ for output in (original, tmp_path / "repair", tmp_path / "compare"):+ assert output.stat().st_mode & 0o777 == 0o700+ assert all(path.stat().st_mode & 0o777 == 0o600 for path in output.iterdir())+ duplicate = runner.invoke(policy_lab, ["replay", str(original / "bundle.json"), "--output", str(original)])+ assert duplicate.exit_code == 1 and "FileExistsError" in duplicate.output+++def fixture_model(native_example):+ fixture_path = Path(__file__).resolve().parents[2] / "fixtures" / "examples" / "customer_support.yaml"+ fixture = load_fixture(fixture_path)+ original = native_example.demo_model()+ actions = tuple(ActionContract.model_validate({**action.model_dump(mode="python"),+ "actor_id": "agent_jordan" if action.actor_id == "agent" else "sup_taylor",+ "tool": {"request_refund": "lookup_account", "approve_refund": "modify_contact", "pay_refund": "issue_refund"}[action.tool]}) for action in original.actions)+ model = original.model_copy(update={"actors": {user.user_id: Actor(roles=tuple(user.roles)) for user in fixture.users}, "actions": actions})+ return fixture_path, fixture, model+++def test_actual_customer_support_bridge_and_cli(native_example, tmp_path):+ fixture_path, fixture, model = fixture_model(native_example)+ policy = policy_from_fixture(fixture, model)+ assert policy.tenant_id == model.tenant_id+ assert all(rule.resource_scope == "*" for rule in policy.permissions)+ assert all(rule.tool in {action.tool for action in model.actions} for rule in policy.permissions)+ assert not any(rule.role == "agent" and rule.tool == "issue_refund" for rule in policy.permissions)+ assert create_exploration_bundle(model, policy).report.decision == "COUNTEREXAMPLE"+ model_file = tmp_path / "model.json"+ model_file.write_text(model.model_dump_json())+ output = tmp_path / "fixture"+ result = CliRunner().invoke(policy_lab, ["from-fixture", str(model_file), str(fixture_path), "--output", str(output)])+ assert result.exit_code == 1, result.output+ assert load_exploration_bundle(output / "bundle.json").policy == policy+ metadata = json.loads((output / "metadata.json").read_text())+ assert "resource scope is wildcard" in metadata["boundary"]++[email protected]("mutation", ["actor", "role", "tool"])+def test_fixture_bridge_rejects_unknown_or_invented_identity(native_example, mutation):+ _, fixture, model = fixture_model(native_example)+ if mutation == "actor":+ model = model.model_copy(update={"actors": {**model.actors, "fictional": Actor(roles=("supervisor",))}})+ elif mutation == "role":+ model = model.model_copy(update={"actors": {**model.actors, "agent_jordan": Actor(roles=("supervisor",))}})+ else:+ action = model.actions[0].model_copy(update={"tool": "python:os.system"})+ model = model.model_copy(update={"actions": (action, *model.actions[1:])})+ with pytest.raises(ValueError):+ policy_from_fixture(fixture, model)+++def test_html_escapes_untrusted_label_and_existing_destination_refused(native_example, tmp_path):+ model = native_example.demo_model()+ prop = model.properties[0].model_copy(update={"label": '<script>alert("model")</script>'})+ bundle = create_exploration_bundle(model.model_copy(update={"properties": (prop,)}), native_example.demo_policy())+ rendered = render_policy_lab_html(bundle)+ assert '<script>alert("model")</script>' not in rendered+ assert "<script>" in rendered+ destination = tmp_path / "private"+ write_policy_lab_reports(bundle, destination)+ existing = (destination / "bundle.json").read_bytes()+ with pytest.raises(FileExistsError):+ write_policy_lab_reports(bundle, destination)+ assert (destination / "bundle.json").read_bytes() == existing++[email protected]("raw", ['{"name":"x","name":"x"}', '{"amount":NaN}'])+def test_cli_strict_json_cannot_import_code(tmp_path, raw):+ input_file = tmp_path / "model.json"+ input_file.write_text(raw)+ result = CliRunner().invoke(policy_lab, ["check", str(input_file), str(input_file), "--output", str(tmp_path / "out")])+ assert result.exit_code == 1 and not (tmp_path / "out").exists()++[email protected]("mutation", ["duplicate_permission", "unknown_role", "unknown_tool", "governance_conflict"])+def test_fixture_authorization_contradictions_rejected(native_example, mutation):+ _, fixture, model = fixture_model(native_example)+ if mutation == "governance_conflict":+ fixture = fixture.model_copy(update={"governance": SimpleNamespace(policies=SimpleNamespace(authorization=[]))})+ else:+ rule = fixture.permissions[0]+ if mutation == "duplicate_permission":+ permissions = [*fixture.permissions, rule]+ elif mutation == "unknown_role":+ permissions = [rule.model_copy(update={"role": "invented"}), *fixture.permissions[1:]]+ else:+ permissions = [rule.model_copy(update={"tools": ["invented"]}), *fixture.permissions[1:]]+ fixture = fixture.model_copy(update={"permissions": permissions})+ with pytest.raises(ValueError):+ policy_from_fixture(fixture, model)+++def test_fixture_projection_revalidates_strict_copied_effects(native_example):+ _, fixture, model = fixture_model(native_example)+ bad_action = model.actions[0].model_copy(update={"cost_units": True})+ model = model.model_copy(update={"actions": (bad_action, *model.actions[1:])})+ with pytest.raises(ValueError):+ policy_from_fixture(fixture, model)