Skip to content

DeepTutor: Lifelong Personalized Tutoring. https://deeptutor.info/.

fix(mastery): bind teaching records to visible delivered explanation

yepyhuncc079776c4dd
diff --git a/.github/tdd/mastery-teaching-delivery.md b/.github/tdd/mastery-teaching-delivery.md
new file mode 100644
index 000000000..b163f67a5
--- /dev/null
+++ b/.github/tdd/mastery-teaching-delivery.md
@@ -0,0 +1,60 @@
+# Teaching-delivery provenance: focused TDD evidence
+
+## Intent
+
+An explanation record must refer to assistant text already published in its
+runtime path, session and turn. A model's plan, private sources or reasoning
+must not stand in for delivered teaching. A short explanation does not prove
+that every part of a composite objective was taught or understood.
+
+The task was planned from the existing Mastery runtime and storage contracts.
+It introduces no database migration, extra model call or mastery-credit rule.
+
+## RED and GREEN
+
+Before implementation, running
+`python -m pytest tests/capabilities/test_mastery_teaching_delivery.py -q --tb=short`
+on the native v1.6.14 baseline executed the tests and produced **15 failed,
+1 passed**. In particular, the existing tool successfully persisted a note
+describing an explanation that had not yet been published.
+
+After implementation, running
+`python -m pytest tests/capabilities/test_mastery_teaching_delivery.py deeptutor/learning/tests/test_visual_learning_workflow.py tests/capabilities/test_mastery_loop_pipeline.py -q --tb=short`
+produced **38 passed in 9.37s** in an isolated contribution worktree after
+the independent visibility review. The added delivery suite has 18 cases
+(13 test functions with parametrization).
+
+The review found that optimistically streamed text can later be retracted by
+the native loop. Two additional regression cases initially failed before
+the snapshot was corrected. The final suite exercises the actual
+`AgentLoop._discard_deferred_output` correction and rejects that prose as
+teaching evidence, while preserving unrelated visible content.
+
+The tests use temporary learning stores and emitted stream events; no real
+learner path or inference provider is used.
+
+## Guarantees
+
+| Behavior | Proof |
+| --- | --- |
+| A future explanation cannot be recorded | The rejected call leaves isolated durable state unchanged |
+| Public fragments can supply a contiguous exact excerpt | A successful note survives store reload with event-range and turn provenance |
+| Thinking, tool results, sources and observations do not count | Each event type is independently rejected |
+| Another source, stage, path, session or turn cannot supply evidence | Scoped rejection tests preserve the explanation state |
+| Model-supplied private delivery arguments are overwritten | Capability and native-dispatch tests exercise server injection |
+| Retracted optimistic prose cannot authorize teaching records | Actual loop-retraction and matching-call visibility tests |
+| Snapshots do not expose mutable events or acquire future output | Snapshot-copy test |
+| A delivered slice is not whole-objective completion | Partial stage and unchanged mastery/evidence assertions |
+| Legacy self-reports are not retroactively certified | Legacy stage test |
+| Source-based practice and assistance still work | Existing visual-learning workflow tests |
+
+## Limits
+
+This proves publication and bounded provenance, not whether the learner read
+the text, understood it, or whether its semantic content fully teaches the
+named objective. Such interpretation remains the tutor's responsibility.
+
+Retrospective notes from an earlier turn are not supported without trusted
+delivery references. Legacy notes are preserved without changing assessment
+scores. Full-model evaluation, live UI testing and a global coverage run were
+not performed for this narrowly scoped correction.
diff --git a/deeptutor/agents/loop/pipeline.py b/deeptutor/agents/loop/pipeline.py
index fe4fb6c62..bbc25bda9 100644
--- a/deeptutor/agents/loop/pipeline.py
+++ b/deeptutor/agents/loop/pipeline.py
@@ -1126,6 +1126,12 @@ class AgenticLoopPipeline:
iteration_index: int,
stage: str = "exploring",
) -> DispatchOutcome:
+ # Capabilities may bind records to text the turn has actually published.
+ # Model messages also contain discarded/private prose and are not proof
+ # of delivery. The callback reads the existing bus; it adds no ledger.
+ context.extension("published_assistant_content")["snapshot"] = (
+ lambda: stream.public_content_snapshot(source=self.event_source, stage=stage)
+ )
too_many = None
if len(tool_calls) > MAX_PARALLEL_TOOL_CALLS:
too_many = self._t(
diff --git a/deeptutor/capabilities/mastery/loop.py b/deeptutor/capabilities/mastery/loop.py
index cfc870c0f..7df1e0118 100644
--- a/deeptutor/capabilities/mastery/loop.py
+++ b/deeptutor/capabilities/mastery/loop.py
@@ -277,6 +277,29 @@ class MasteryLoopCapability:
updated["_mastery_session_mode"] = context.metadata.get("mastery_session_mode")
updated["_session_id"] = str(context.session_id or "").strip()
updated["_turn_id"] = str(context.metadata.get("turn_id") or "").strip()
+ if tool_name == "mastery_note_explained":
+ session_id = str(context.session_id or "").strip()
+ turn_id = str(context.metadata.get("turn_id") or "").strip()
+
+ def teaching_delivery():
+ # A rebind must not let an earlier tool record teaching on
+ # the new path or under another session/turn's provenance.
+ if (
+ str(context.metadata.get("mastery_path_id") or "").strip() != path_id
+ or str(context.session_id or "").strip() != session_id
+ or str(context.metadata.get("turn_id") or "").strip() != turn_id
+ ):
+ return None
+ snapshot = context.extension("published_assistant_content").get("snapshot")
+ return {
+ "path_id": path_id,
+ "session_id": session_id,
+ "turn_id": turn_id,
+ "segments": snapshot() if callable(snapshot) else (),
+ }
+
+ # Always override model-supplied private kwargs.
+ updated["_teaching_delivery"] = teaching_delivery
if tool_name == "mastery_mode":
# The narrowest handle on the turn for a tool that changes what
# the rest of it may do — the same shape as ``_bind_active_path``
diff --git a/deeptutor/capabilities/mastery/prompts/en/mastery_loop.yaml b/deeptutor/capabilities/mastery/prompts/en/mastery_loop.yaml
index 519021a8d..6d27e8a8f 100644
--- a/deeptutor/capabilities/mastery/prompts/en/mastery_loop.yaml
+++ b/deeptutor/capabilities/mastery/prompts/en/mastery_loop.yaml
@@ -86,7 +86,7 @@ session:
# Teaching strategy. Kept separate from the loop protocol above: that one says
# how a round runs, this one says what to do about the objective in front of you.
playbook: |-
- **Source-grounded visual study:** Honor explanation, practice and review as explicit learner choices; explain untaught prerequisites at the requested level before assessment. Use `mastery_note_explained` after an explanation; this records coverage without mastery credit. An outline is never evidence of teaching. A concise explanation-only turn may end without a quiz; explicit deferral uses `mastery_defer_objective` and does not claim mastery.
+ **Source-grounded visual study:** Honor explanation, practice and review as explicit learner choices; explain untaught prerequisites at the requested level before assessment. Use `mastery_note_explained` only after the explanation was publicly delivered in this turn, with an exact `explanation_quote`; never record a future plan, private source text or reasoning. This records a partially explained slice, not complete-objective coverage, understanding or mastery credit. Historical teaching records without delivery references remain unverified, not evidence that every part was taught. An outline is never evidence of teaching. A concise explanation-only turn may end without a quiz; explicit deferral uses `mastery_defer_objective` and does not claim mastery.
When the goal includes figure understanding, declare the relevant objective's `required_visual_tasks` in the outline (`identification`, `relationship`, `table_graph`, `comparison`). Generic text scores do not satisfy these. Read `mastery_status.visual_requirements` and `learning_stages`; explain whether the next work addresses missing independent evidence, prior difficulty or due review.
Retrieve and inspect the original pixels with `rag` before a visual question. Pass `visual` to `mastery_quiz`, using exact source references and a supported reference quote, plus verified equivalent terminology (including the learner's language). Ask about the actual numbered label, visible relationship, plotted/table values or supported comparison. Preserve the original and source location; do not replace it with a generated diagram or invent a masked/segmented target. A key based only on uncertain pixel interpretation stays `unverified` and ungraded.
Describe visible answer cues honestly. A labeled image or direct table/page reading is guided practice; do not claim independent recall. Repeating recently assisted work is also weakened through the existing retention boundary. Unknown wording asks for clarification rather than fuzzy grading; use a well-grounded choice question when a supported wrong answer must be distinguished reliably. Explain mistakes against the original source and offer a different supported example, not an immediate loop of the same question.
diff --git a/deeptutor/capabilities/mastery/prompts/zh/mastery_loop.yaml b/deeptutor/capabilities/mastery/prompts/zh/mastery_loop.yaml
index 8ffcb6a95..71cf284d8 100644
--- a/deeptutor/capabilities/mastery/prompts/zh/mastery_loop.yaml
+++ b/deeptutor/capabilities/mastery/prompts/zh/mastery_loop.yaml
@@ -83,7 +83,7 @@ session:
# 教学策略。与 loop 协议分开:上面说的是「一轮怎么走」,这里说的是「针对这个知识点做什么」。
playbook: |-
- **来源扎实的视觉学习:** 尊重学习者明确选择的讲解、练习或复习,按要求的层次先解释必要前置知识。讲解后调用 `mastery_note_explained`,只记录已讲解覆盖,不授予掌握分;生成提纲不等于教学。仅需简洁讲解时可以不出题;明确暂缓用 `mastery_defer_objective`,不能假称已掌握。
+ **来源扎实的视觉学习:** 尊重学习者明确选择的讲解、练习或复习,按要求的层次先解释必要前置知识。仅在本轮已经公开讲解后调用 `mastery_note_explained`,提供逐字的 `explanation_quote`;不能记录未来计划、私有来源文本或推理。它只记录部分已讲解内容,不证明整个目标已覆盖、已理解或已掌握。没有公开交付引用的历史教学记录仍未核实,不能当作全部讲过的证据;生成提纲不等于教学。仅需简洁讲解时可以不出题;明确暂缓用 `mastery_defer_objective`,不能假称已掌握。
目标包含原图理解时,在对应提纲目标声明 `required_visual_tasks`(identification、relationship、table_graph、comparison)。通用文字题高分不能替代这些任务。查看 `mastery_status.visual_requirements` 和 `learning_stages`,解释下一步来自缺少独立证据、此前困难还是到期复习。
视觉题先通过 `rag` 读取并检查原始像素,再给 `mastery_quiz` 传 `visual`:精确来源、来源中的参考引文以及已核验的等价术语(包含学习者语言)。练习应判断真实编号标签、可见关系、图表数据或有证据支持的比较。保留原图及位置,不能用生成图替代,也不能虚构遮罩或分割目标。仅凭不确定图像理解得到的答案键标为 unverified,保留未评分。
如实记录可见答案线索:带答案标签或直接读取表格/页面是辅助练习,不是独立回忆;即时重复刚获帮助的题目也会按现有复习时间边界降权。未知表述应澄清而不是模糊打分;需要可靠区分实质性错误时使用来源明确的选择题。答错后对照原图讲解,提供其他有依据的例子,不要立即循环相同问题。
diff --git a/deeptutor/capabilities/mastery/tools.py b/deeptutor/capabilities/mastery/tools.py
index d988ccaf0..a3f031176 100644
--- a/deeptutor/capabilities/mastery/tools.py
+++ b/deeptutor/capabilities/mastery/tools.py
@@ -829,7 +829,7 @@ class MasteryNoteExplainedTool(BaseTool):
def get_definition(self) -> ToolDefinition:
return ToolDefinition(
name="mastery_note_explained",
- description="After explaining an objective at the requested level, record what was taught. This does not assess, grant mastery or schedule a review. Respect explanation-only requests; a quiz is optional until the learner wants practice.",
+ description="After publicly explaining part of an objective, record that delivered explanation with an exact quote. Never record a planned explanation or treat this as complete-objective coverage, assessment or mastery. A quiz is optional until the learner wants practice.",
parameters=[
ToolParameter(
name="knowledge_point_id",
@@ -841,6 +841,11 @@ class MasteryNoteExplainedTool(BaseTool):
type="string",
description="What was actually explained, including prerequisites and source limitations.",
),
+ ToolParameter(
+ name="explanation_quote",
+ type="string",
+ description="Exact 24-2000 character excerpt of explanation already publicly delivered in this turn; not source text, reasoning or a future plan.",
+ ),
],
)
@@ -855,6 +860,47 @@ class MasteryNoteExplainedTool(BaseTool):
summary = str(kwargs.get("summary") or "").strip()[:2000]
if not summary:
return ToolResult(content="A teaching summary is required.", success=False)
+ quote = kwargs.get("explanation_quote")
+ if not isinstance(quote, str) or not 24 <= len(quote.strip()) <= 2000:
+ return ToolResult(content="Provide an exact 24-2000 character excerpt of the explanation already delivered publicly in this turn.", success=False)
+ quote = quote.strip()
+ read_delivery = kwargs.get("_teaching_delivery")
+ delivery = read_delivery() if callable(read_delivery) else None
+ expected_scope = {
+ "path_id": path_id,
+ "session_id": _resolve_session_id(kwargs),
+ "turn_id": _resolve_turn_id(kwargs),
+ }
+ if (
+ not isinstance(delivery, dict)
+ or not all(expected_scope.values())
+ or any(delivery.get(key) != value for key, value in expected_scope.items())
+ ):
+ return ToolResult(content="Teaching must be bound to the active runtime path, session and turn; no explanation was recorded.", success=False)
+ segments = delivery.get("segments")
+ if not isinstance(segments, tuple) or any(
+ not isinstance(segment, tuple) or len(segment) != 3
+ or not isinstance(segment[0], int) or not isinstance(segment[1], (int, float))
+ or not isinstance(segment[2], str)
+ for segment in segments
+ ):
+ return ToolResult(content="Published explanation evidence is unavailable; no explanation was recorded.", success=False)
+ published = "".join(segment[2] for segment in segments)
+ start = published.rfind(quote)
+ if start < 0:
+ return ToolResult(content="That excerpt has not been publicly delivered in this turn. Explain first, then record only what was taught; do not record a future plan.", success=False)
+ offset = 0
+ matched = []
+ for segment in segments:
+ end = offset + len(segment[2])
+ if offset < start + len(quote) and end > start:
+ matched.append(segment)
+ offset = end
+ provenance = {
+ **expected_scope,
+ "event_seq_range": [matched[0][0], matched[-1][0]],
+ "event_timestamp_range": [matched[0][1], matched[-1][1]],
+ }
service = _new_service()
if _load_path(service, path_id) is None:
return _no_built_path_result("mastery_note_explained")
@@ -866,18 +912,24 @@ class MasteryNoteExplainedTool(BaseTool):
raise ValueError("Unknown objective; refresh mastery_status.")
tx.progress.explained_objectives[kp_id] = {
"summary": summary,
+ "explanation_quote": quote,
+ "delivery": provenance,
+ "coverage": "partial",
"timestamp": time.time(),
"session_id": _resolve_session_id(kwargs),
"turn_id": _resolve_turn_id(kwargs),
}
- tx.emit("objective.explained", {"knowledge_point_id": kp_id, "summary": summary})
+ tx.emit("objective.explained", {
+ "knowledge_point_id": kp_id, "summary": summary,
+ "explanation_quote": quote, "delivery": provenance, "coverage": "partial",
+ }, session_id=_resolve_session_id(kwargs), turn_id=_resolve_turn_id(kwargs))
try:
await asyncio.to_thread(service.store.mutate, path_id, note)
except (ValueError, FileNotFoundError) as exc:
return ToolResult(content=str(exc), success=False)
return _json_result(
- {"knowledge_point_id": kp_id, "stage": "explained", "mastery_credit": False},
+ {"knowledge_point_id": kp_id, "stage": "partially_explained", "coverage": "partial", "mastery_credit": False},
meta_key="mastery_note_explained",
)
diff --git a/deeptutor/learning/policy.py b/deeptutor/learning/policy.py
index 5989000b4..1980fd2b8 100644
--- a/deeptutor/learning/policy.py
+++ b/deeptutor/learning/policy.py
@@ -123,7 +123,18 @@ def learning_stage(progress: LearningProgress, kp_id: str) -> str:
return "practiced_with_help"
if attempts or evidence:
return "practiced_not_demonstrated"
- return "explained" if kp_id in progress.explained_objectives else "not_yet_covered"
+ explained = progress.explained_objectives.get(kp_id)
+ if not explained:
+ return "not_yet_covered"
+ if (
+ explained.get("coverage") == "partial"
+ and explained.get("explanation_quote")
+ and explained.get("delivery", {}).get("turn_id")
+ ):
+ return "partially_explained"
+ # Keep historical notes, but do not invent proof of published teaching or
+ # imply that a summary covered every part of a composite objective.
+ return "teaching_record_unverified"
def mastery_source(progress: LearningProgress, kp: KnowledgePoint) -> str:
diff --git a/deeptutor/learning/tests/test_visual_learning_workflow.py b/deeptutor/learning/tests/test_visual_learning_workflow.py
index 68c84d2f7..109ff0215 100644
--- a/deeptutor/learning/tests/test_visual_learning_workflow.py
+++ b/deeptutor/learning/tests/test_visual_learning_workflow.py
@@ -21,6 +21,7 @@ from deeptutor.learning.policy import is_assessed_mastered, visual_achievements
from deeptutor.learning.scheduler import SpacedRepetitionScheduler
from deeptutor.learning.service import LearningService
from deeptutor.learning.storage import LearningStore
+from deeptutor.runtime.stream_bus import StreamBus
from deeptutor.services.parsing.types import ParsedDocument
from deeptutor.services.rag.visual_assets import VisualAssetStore, collect_visual_assets
from deeptutor.services.session.sqlite_store import SQLiteSessionStore
@@ -149,10 +150,19 @@ async def pose(lesson, **overrides):
@pytest.mark.asyncio
async def test_teach_then_source_visual_practice_equivalent_answer_and_restart(lesson):
+ explanation = "An artery has a thicker wall than a vein; structure 1 is the artery in this source."
+ stream = StreamBus(assign_seq=True)
+ await stream.content(explanation, source="mastery_path", stage="responding")
explained = await tools.MasteryNoteExplainedTool().execute(
_mastery_path_id="visual",
+ _session_id="s", _turn_id="teaching",
knowledge_point_id="identify",
summary="Explained artery/vein prerequisites concisely at beginner level using the numbered source.",
+ explanation_quote=explanation,
+ _teaching_delivery=lambda: {
+ "path_id": "visual", "session_id": "s", "turn_id": "teaching",
+ "segments": stream.public_content_snapshot(source="mastery_path", stage="responding"),
+ },
)
assert explained.success
before = lesson.store.load("visual")
@@ -385,11 +395,21 @@ async def test_pdf_region_can_assess_relationship_without_displaying_the_referen
@pytest.mark.asyncio
async def test_help_after_question_is_assisted_and_deferral_never_claims_mastery(lesson):
card = await pose(lesson)
- await tools.MasteryNoteExplainedTool().execute(
+ explanation = "The thick-walled vessel is an artery; compare the two vessel walls in the source."
+ stream = StreamBus(assign_seq=True)
+ await stream.content(explanation, source="mastery_path", stage="responding")
+ explained = await tools.MasteryNoteExplainedTool().execute(
_mastery_path_id="visual",
+ _session_id="s", _turn_id="hint",
knowledge_point_id="identify",
summary="Gave a source-based hint after the learner asked for help with structure 1.",
+ explanation_quote=explanation,
+ _teaching_delivery=lambda: {
+ "path_id": "visual", "session_id": "s", "turn_id": "hint",
+ "segments": stream.public_content_snapshot(source="mastery_path", stage="responding"),
+ },
)
+ assert explained.success, explained.content
progress, _interaction, _ = lesson.service.grade_interaction(
"visual", answer="artery", question_id=card["question_id"]
)
diff --git a/deeptutor/runtime/stream_bus.py b/deeptutor/runtime/stream_bus.py
index 91a09cdd9..acc35d78e 100644
--- a/deeptutor/runtime/stream_bus.py
+++ b/deeptutor/runtime/stream_bus.py
@@ -59,6 +59,41 @@ class StreamBus:
"""Largest sequence issued by this bus, or zero before its first event."""
return self._next_seq - 1 if self._assign_seq else 0
+ def public_content_snapshot(
+ self, *, source: str, stage: str
+ ) -> tuple[tuple[int, float, str], ...]:
+ """Read already published answer text, without exposing mutable events.
+
+ Reasoning, sources and tool results are not assistant explanations.
+ Taking a snapshot never subscribes, waits for future output or emits an
+ event. Sequence zero remains zero for direct callers without sequencing.
+ """
+ visibility: dict[str, bool] = {}
+ for event in self._history:
+ if event.source != source or event.stage != stage:
+ continue
+ if event.type not in {StreamEventType.CONTENT, StreamEventType.PROGRESS}:
+ continue
+ call_id = event.metadata.get("call_id")
+ if not isinstance(call_id, str) or not call_id:
+ continue
+ if event.metadata.get("finish_rejected") is True:
+ visibility[call_id] = False
+ elif isinstance(event.metadata.get("answer_visible"), bool):
+ visibility[call_id] = event.metadata["answer_visible"]
+ return tuple(
+ (event.seq, event.timestamp, event.content)
+ for event in self._history
+ if event.type == StreamEventType.CONTENT
+ and event.source == source
+ and event.stage == stage
+ and event.content
+ and event.metadata.get("answer_visible") is not False
+ and event.metadata.get("finish_rejected") is not True
+ and (not isinstance(event.metadata.get("call_id"), str)
+ or visibility.get(event.metadata["call_id"], True))
+ )
+
async def emit(self, event: StreamEvent) -> None:
"""Push *event* to every active subscriber."""
if self._closed:
diff --git a/tests/capabilities/test_mastery_teaching_delivery.py b/tests/capabilities/test_mastery_teaching_delivery.py
new file mode 100644
index 000000000..c1bf008ed
--- /dev/null
+++ b/tests/capabilities/test_mastery_teaching_delivery.py
@@ -0,0 +1,227 @@
+"""Teaching records describe delivered explanations, not future model plans."""
+
+import pytest
+
+from types import SimpleNamespace
+
+from deeptutor.capabilities.mastery import tools
+from deeptutor.capabilities.mastery.loop import MasteryLoopCapability
+from deeptutor.agents.loop.pipeline import AgenticLoopPipeline
+from deeptutor.agents.loop.agent_loop import AgentLoop, LLMCallResult
+from deeptutor.core.context import UnifiedContext
+from deeptutor.core.stream import StreamEvent, StreamEventType
+from deeptutor.learning.models import KnowledgePoint, LearningModule, LearningProgress
+from deeptutor.learning.policy import learning_stage
+from deeptutor.learning.service import LearningService
+from deeptutor.learning.storage import LearningStore
+from deeptutor.runtime.stream_bus import StreamBus
+
+
+EXPLANATION = "A membrane is a boundary, separating the inside of the cell from its surroundings."
+
+
+def lesson(tmp_path, monkeypatch):
+ service = LearningService(LearningStore(tmp_path))
+ service.save(LearningProgress(book_id="p", modules=[LearningModule(
+ id="m", name="Module", order=0,
+ knowledge_points=[KnowledgePoint(id="kp", name="Boundary and transport", type="memory", module_id="m")],
+ )]))
+ monkeypatch.setattr(tools, "_new_service", lambda: service)
+ return service
+
+
+def delivery(bus, **overrides):
+ return lambda: {
+ "path_id": "p", "session_id": "s", "turn_id": "t",
+ "segments": bus.public_content_snapshot(source="mastery_path", stage="responding"),
+ **overrides,
+ }
+
+
+async def note(bus, **kwargs):
+ return await tools.MasteryNoteExplainedTool().execute(
+ _mastery_path_id="p", _session_id="s", _turn_id="t",
+ knowledge_point_id="kp", summary="Explained the membrane boundary, not all transport.",
+ explanation_quote=EXPLANATION, _teaching_delivery=delivery(bus), **kwargs,
+ )
+
+
+async def test_future_explanation_cannot_be_recorded(lesson):
+ before = lesson.store.load("p").model_dump()
+ result = await note(StreamBus(assign_seq=True))
+ assert not result.success
+ assert lesson.store.load("p").model_dump() == before
+
+
+async def test_public_fragmented_explanation_has_durable_partial_provenance(lesson):
+ bus = StreamBus(assign_seq=True)
+ await bus.content(EXPLANATION[:32], source="mastery_path", stage="responding")
+ await bus.content(EXPLANATION[32:], source="mastery_path", stage="responding")
+ result = await note(bus)
+ assert result.success, result.content
+ fresh = LearningStore(lesson.store._root).load("p")
+ recorded = fresh.explained_objectives["kp"]
+ assert recorded["explanation_quote"] == EXPLANATION
+ assert recorded["delivery"]["turn_id"] == "t"
+ assert recorded["delivery"]["event_seq_range"] == [1, 2]
+ assert recorded["coverage"] == "partial"
+ assert learning_stage(fresh, "kp") == "partially_explained"
+ assert fresh.mastery_levels.get("kp", 0) == 0
+ assert not fresh.learning_evidence and not fresh.quiz_attempts
+
+
[email protected]("event_type", [StreamEventType.THINKING, StreamEventType.TOOL_RESULT,
+ StreamEventType.SOURCES, StreamEventType.OBSERVATION])
+async def test_non_answer_events_are_not_teaching(lesson, event_type):
+ bus = StreamBus(assign_seq=True)
+ await bus.emit(StreamEvent(type=event_type, source="mastery_path", stage="responding",
+ content=EXPLANATION))
+ assert not (await note(bus)).success
+ assert not lesson.store.load("p").explained_objectives
+
+
[email protected]("scope", ["path_id", "session_id", "turn_id"])
+async def test_delivery_scope_cannot_cross_path_session_or_turn(lesson, scope):
+ bus = StreamBus(assign_seq=True)
+ await bus.content(EXPLANATION, source="mastery_path", stage="responding")
+ result = await tools.MasteryNoteExplainedTool().execute(
+ _mastery_path_id="p", _session_id="s", _turn_id="t", knowledge_point_id="kp",
+ summary="A real explanation with forged provenance.", explanation_quote=EXPLANATION,
+ _teaching_delivery=delivery(bus, **{scope: "other"}),
+ )
+ assert not result.success
+ assert not lesson.store.load("p").explained_objectives
+
+
+async def test_other_capability_content_and_unpublished_text_do_not_count(lesson):
+ bus = StreamBus(assign_seq=True)
+ await bus.content(EXPLANATION, source="rag", stage="responding")
+ await bus.content(EXPLANATION, source="mastery_path", stage="exploring")
+ assert not (await note(bus)).success
+ # Text discarded by the loop never reaches the stream and cannot supply evidence.
+ model_message_only = {"role": "assistant", "content": EXPLANATION}
+ assert model_message_only["content"] == EXPLANATION
+ assert not lesson.store.load("p").explained_objectives
+
+
+async def test_snapshot_is_immutable_and_does_not_gain_future_events():
+ bus = StreamBus(assign_seq=True)
+ await bus.content("first", source="mastery_path", stage="responding")
+ snapshot = bus.public_content_snapshot(source="mastery_path", stage="responding")
+ await bus.content("second", source="mastery_path", stage="responding")
+ assert isinstance(snapshot, tuple) and isinstance(snapshot[0], tuple)
+ assert len(snapshot) == 1 and snapshot[0][2] == "first"
+
+
+def test_legacy_self_report_is_not_verified_delivery(lesson):
+ progress = lesson.store.load("p")
+ progress.explained_objectives["kp"] = {"summary": "A legacy self-report", "timestamp": 1}
+ assert learning_stage(progress, "kp") == "teaching_record_unverified"
+
+
+def test_mastery_capability_uses_server_snapshot_and_rechecks_binding():
+ context = UnifiedContext(user_message="continue", session_id="s", metadata={
+ "mastery_mode": True, "mastery_path_id": "p", "mastery_session_mode": "study", "turn_id": "t",
+ })
+ context.extension("published_assistant_content")["snapshot"] = lambda: ((1, 1.0, EXPLANATION),)
+ bound = MasteryLoopCapability().augment_kwargs("mastery_note_explained", {
+ "_teaching_delivery": lambda: {"path_id": "FORGED"},
+ }, context)
+ assert bound["_teaching_delivery"]()["path_id"] == "p"
+ context.metadata["mastery_path_id"] = "other"
+ assert bound["_teaching_delivery"]() is None
+
+
+async def test_native_dispatch_binds_the_live_public_stream(lesson, monkeypatch):
+ context = UnifiedContext(user_message="continue", session_id="s", metadata={
+ "mastery_mode": True, "mastery_path_id": "p", "turn_id": "t",
+ })
+ bus = StreamBus(assign_seq=True)
+ await bus.content(EXPLANATION, source="mastery_path", stage="responding")
+ pipeline = object.__new__(AgenticLoopPipeline)
+ pipeline.event_source = "mastery_path"
+ pipeline._tool_view = None
+ pipeline.registry = SimpleNamespace()
+ pipeline._t = lambda *_args, **_kwargs: "test"
+ pipeline._capability_rebinding_tools = lambda _context: set()
+ pipeline._augment_tool_kwargs = lambda name, args, ctx: MasteryLoopCapability().augment_kwargs(name, args, ctx)
+
+ async def dispatch(**kwargs):
+ bound = kwargs["kwarg_augmenter"]("mastery_note_explained", {
+ "knowledge_point_id": "kp", "summary": "The delivered boundary explanation.",
+ "explanation_quote": EXPLANATION,
+ "_teaching_delivery": lambda: {"path_id": "forged"},
+ }, context)
+ return await tools.MasteryNoteExplainedTool().execute(**bound)
+
+ monkeypatch.setattr("deeptutor.agents.loop.pipeline.dispatch_tool_calls", dispatch)
+ result = await pipeline._dispatch_tool_calls(
+ tool_calls=[], context=context, stream=bus, iteration_index=0, stage="responding",
+ )
+ assert result.success, result.content
+ assert lesson.store.load("p").explained_objectives["kp"]["delivery"]["turn_id"] == "t"
+
+
+async def test_quote_must_be_exact_and_nontrivial(lesson):
+ bus = StreamBus(assign_seq=True)
+ await bus.content(EXPLANATION, source="mastery_path", stage="responding")
+ for quote in ("yes", "An invented explanation that was never emitted.", "a" * 2001):
+ result = await tools.MasteryNoteExplainedTool().execute(
+ _mastery_path_id="p", _session_id="s", _turn_id="t",
+ knowledge_point_id="kp", summary="A proposed teaching record.",
+ explanation_quote=quote, _teaching_delivery=delivery(bus),
+ )
+ assert not result.success
+ assert not lesson.store.load("p").explained_objectives
+
+
+async def test_unknown_objective_cannot_gain_delivered_record(lesson):
+ bus = StreamBus(assign_seq=True)
+ await bus.content(EXPLANATION, source="mastery_path", stage="responding")
+ result = await tools.MasteryNoteExplainedTool().execute(
+ _mastery_path_id="p", _session_id="s", _turn_id="t",
+ knowledge_point_id="other", summary="A delivered explanation.",
+ explanation_quote=EXPLANATION, _teaching_delivery=delivery(bus),
+ )
+ assert not result.success
+ assert not lesson.store.load("p").explained_objectives
+
+
+async def test_retracted_optimistic_prose_cannot_supply_teaching_evidence(lesson):
+ bus = StreamBus(assign_seq=True)
+ await bus.content(EXPLANATION, source="mastery_path", stage="responding",
+ metadata={"call_id": "rejected", "answer_visible": True})
+ runner = object.__new__(AgentLoop)
+ runner.stream = bus
+ runner.source = "mastery_path"
+ runner.stage = "responding"
+ await runner._discard_deferred_output(LLMCallResult(
+ text=EXPLANATION, completion_metadata={"call_id": "rejected", "answer_visible": True},
+ ))
+ assert not (await note(bus)).success
+ assert not lesson.store.load("p").explained_objectives
+
+
+async def test_visibility_corrections_are_scoped_to_the_matching_call():
+ bus = StreamBus(assign_seq=True)
+ await bus.content("discarded", source="mastery_path", stage="responding",
+ metadata={"call_id": "bad", "answer_visible": True})
+ await bus.content("accepted", source="mastery_path", stage="responding",
+ metadata={"call_id": "good", "answer_visible": True})
+ await bus.content("callless", source="mastery_path", stage="responding")
+ await bus.progress("", source="mastery_path", stage="responding",
+ metadata={"call_id": "bad", "answer_visible": False})
+ snapshot = bus.public_content_snapshot(source="mastery_path", stage="responding")
+ assert [part[2] for part in snapshot] == ["accepted", "callless"]