Skip to content

DeepTutor: Lifelong Personalized Tutoring. https://deeptutor.info/.

docs: record source-bound assessment verification and port boundaries

yepyhun3e383c80f473
diff --git a/docs/verification/mastery-semantic-assessment.md b/docs/verification/mastery-semantic-assessment.md
new file mode 100644
index 000000000..62aa63ae1
--- /dev/null
+++ b/docs/verification/mastery-semantic-assessment.md
@@ -0,0 +1,77 @@
+# Source-bound semantic assessment
+
+This contribution is stacked on `contrib/mastery-grouped-assessment` at
+`6532c8c2` (PR #1943). It ports selected assessment symbols from the local
+integration workspace, not whole shared service/tool files.
+
+## Guarantees
+
+- Exact verified choices/equivalents stay local. Other answered free-form
+ leaves share one bounded semantic request, using original authorized PDF
+ pixels when needed; image fallback is disabled.
+- Question contracts bind objective, question, options, key, rubric and fresh
+ source evidence. Citation quotes must exist. Original-exam provenance is
+ conservative; choice keys are not inferred from an isolated letter.
+- Await happens outside the store transaction. Commit rechecks the original
+ interaction/answer/objective fingerprint, lifecycle, source access and digest.
+- Provider errors write no grade. Uncertain answers are nonfinal and may be
+ explicitly revised; final siblings cannot be changed. Group children do not
+ create additional active-question rows or duplicate mastery credit.
+- Partial is distinct from incorrect in attempts, evidence, scheduler quality,
+ question-bank entries, native cards and tutor seeds. Corrections reassess the
+ saved raw learner answer, discard old key-derived aliases and repair receipts.
+- Qualitative assessment uses the actual runtime learner message, never the
+ model's passed flag. Native selected-model ContextVar routing is retained.
+- Single and grouped grading failures emit recovery events, preserving drafts.
+
+## Source dependencies
+
+`lesson_evidence.py` supplies actual preparation, bounded targeted repair,
+validated cached full/partial packets and selected-source scope. `source_pages.py`
+supplies physical locators, hashes, bounded rendering and legacy BM25 page
+sidecars. Four small LlamaIndex owner changes preserve physical PDF documents,
+persist page catalogs and scope retrieval before searching. No alternate
+renderer, fake verification flag, OCR migration or global model override exists.
+
+The explicit `mastery_prepare_lesson` tool is registered and mounted. The
+automatic cache-delivery pipeline and taught-state source guard are a separate
+stacked source-preparation follow-up, not silently copied into this slice.
+Tests specific to those unported owners and the separate legacy URL alias
+migration are intentionally not included. New references use native route
+parameters `source_path`, `source_hash`, `page`, `region`.
+
+## Validation
+
+All tests use temporary synthetic records and offline provider stubs. No learner
+data was changed; no backend restart or provider-cost call was made by this port.
+
+- Initial root RED: missing semantic implementation; longer correct paraphrase,
+ nonfinal revisions, partial outcomes and provider failure contracts failed.
+- Additional RED: single-card runtime failure variants produced zero recovery
+ events (3 failed, 3 passed); after fix runtime suite **20 passed**.
+- Additional RED: qualitative question-bank result argument was missing (2
+ failed); after fix root semantic suite **20 passed**.
+- Native backend: semantic assessment, source bridge, source pages, visual
+ workflow, mastery tools, group runtime and capability suites **280 passed**;
+ after adding the two qualitative-bank regressions the affected semantic/tools
+ suites **100 passed**.
+- Native DOM: `npm run test:unit -- tests/mastery-question-group.spec.tsx
+ tests/question-bank-picker.spec.tsx` **20 passed**. Covers uncertain leaf
+ revisions, final sibling locks, single clarification retry and typed badges.
+- Native `npm run typecheck`: passed.
+- Scoped Ruff and `git diff --check`: passed.
+
+Parent review separately ran two controlled model probes: four text decisions
+and one original synthetic-pixel table decision. Those integration observations
+are not substituted for the deterministic regression suite above.
+
+## Port boundaries
+
+Service: grade/evaluate/repair/qualitative paths, group merge/result/leaf paths,
+question registration/answer revision, evidence update and mastery calculation.
+Tools: prepare/quiz/grade/assess/repair and qualitative-bank projection only.
+Loop: raw-message injection for those tools and `_grade_seed` only. Runtime:
+`_grade_submitted_card_answer` failure projection only. UI: typed result helpers,
+cards, receipt/rejection projection, bank badges and six partial translations.
+Profile, intake, memory, broad teaching guidance and delivery recovery edits
+remain in their separate contributions.
diff --git a/tests/services/rag/test_mastery_source_pages.py b/tests/services/rag/test_mastery_source_pages.py
index 22864a547..4cc39bf79 100644
--- a/tests/services/rag/test_mastery_source_pages.py
+++ b/tests/services/rag/test_mastery_source_pages.py
@@ -674,5 +674,3 @@ def test_preparation_instructions_travel_in_actual_messages(material, monkeypatc
return '{"queries":["membrane"]}'
monkeypatch.setattr("deeptutor.services.llm.factory.complete_with_config", complete)
assert asyncio.run(lessons._ask(material.config, "Return only JSON", {"objective": "membrane"})) == {"queries": ["membrane"]}
-
-