Skip to content

DeepTutor: Lifelong Personalized Tutoring. https://deeptutor.info/.

feat(mastery): retain scoped teaching feedback with learner evidence

yepyhune6853db33601
diff --git a/deeptutor/capabilities/mastery/loop.py b/deeptutor/capabilities/mastery/loop.py
index cfc870c0f..9cc66a5e3 100644
--- a/deeptutor/capabilities/mastery/loop.py
+++ b/deeptutor/capabilities/mastery/loop.py
@@ -277,6 +277,8 @@ class MasteryLoopCapability:
updated["_mastery_session_mode"] = context.metadata.get("mastery_session_mode")
updated["_session_id"] = str(context.session_id or "").strip()
updated["_turn_id"] = str(context.metadata.get("turn_id") or "").strip()
+ if tool_name == "mastery_profile":
+ updated["_learner_message"] = context.user_message or ""
if tool_name == "mastery_mode":
# The narrowest handle on the turn for a tool that changes what
# the rest of it may do — the same shape as ``_bind_active_path``
diff --git a/deeptutor/capabilities/mastery/prompts/en/mastery_loop.yaml b/deeptutor/capabilities/mastery/prompts/en/mastery_loop.yaml
index 519021a8d..434b16c01 100644
--- a/deeptutor/capabilities/mastery/prompts/en/mastery_loop.yaml
+++ b/deeptutor/capabilities/mastery/prompts/en/mastery_loop.yaml
@@ -86,6 +86,8 @@ session:
# Teaching strategy. Kept separate from the loop protocol above: that one says
# how a round runs, this one says what to do about the objective in front of you.
playbook: |-
+ **Durable teaching feedback:** Read `learner_profile.teaching_preferences` and the current objective's `teaching_observations` in the fresh snapshot. Honor the learner's current explicit request over older style notes. On repeated confusion or an unsuccessful explanation, change the useful teaching approach (example, prerequisite, spatial explanation or suitable original figure), rather than only promising to simplify. Do not label a learner globally from one difficulty. Record important explicit requests and objective-local attempts with keyed `mastery_profile.teaching_updates` and the current `path_revision`; preserve other keys. Inferred difficulties stay `tentative`; a strategy result stays `unknown` until a learner statement or response supports `helped` or `not_helped`. Teaching, silence and unanswered cards do not prove understanding. These notes never award mastery or override source rules. Do not create a feedback tool call for every message; combine meaningful changes, and keep ordinary tutoring fluent. The learner may inspect, correct or remove these notes through conversation.
+ In teaching_updates, evidence must quote the current learner message literally; a paraphrase or your own teaching text cannot confirm a strategy result. To inspect or correct older notes outside the current objective, use mastery_status with include_teaching_history=true; ordinary teaching uses the small default snapshot.
**Source-grounded visual study:** Honor explanation, practice and review as explicit learner choices; explain untaught prerequisites at the requested level before assessment. Use `mastery_note_explained` after an explanation; this records coverage without mastery credit. An outline is never evidence of teaching. A concise explanation-only turn may end without a quiz; explicit deferral uses `mastery_defer_objective` and does not claim mastery.
When the goal includes figure understanding, declare the relevant objective's `required_visual_tasks` in the outline (`identification`, `relationship`, `table_graph`, `comparison`). Generic text scores do not satisfy these. Read `mastery_status.visual_requirements` and `learning_stages`; explain whether the next work addresses missing independent evidence, prior difficulty or due review.
Retrieve and inspect the original pixels with `rag` before a visual question. Pass `visual` to `mastery_quiz`, using exact source references and a supported reference quote, plus verified equivalent terminology (including the learner's language). Ask about the actual numbered label, visible relationship, plotted/table values or supported comparison. Preserve the original and source location; do not replace it with a generated diagram or invent a masked/segmented target. A key based only on uncertain pixel interpretation stays `unverified` and ungraded.
diff --git a/deeptutor/capabilities/mastery/prompts/zh/mastery_loop.yaml b/deeptutor/capabilities/mastery/prompts/zh/mastery_loop.yaml
index 8ffcb6a95..743d06184 100644
--- a/deeptutor/capabilities/mastery/prompts/zh/mastery_loop.yaml
+++ b/deeptutor/capabilities/mastery/prompts/zh/mastery_loop.yaml
@@ -83,6 +83,8 @@ session:
# 教学策略。与 loop 协议分开:上面说的是「一轮怎么走」,这里说的是「针对这个知识点做什么」。
playbook: |-
+ **持久教学反馈:** 读取最新状态中的 `learner_profile.teaching_preferences` 及当前知识点的 `teaching_observations`。当前明确请求优先于旧的风格偏好。反复困惑或解释未奏效时,改变教学方式(例子、前置知识、空间说明或适合的原始图),不要只承诺简化。一次困难不能给学习者贴全局标签。用带键的 `mastery_profile.teaching_updates` 和当前 `path_revision` 记录重要明确请求及知识点局部尝试,保留其他键。推测困难标为 `tentative`;只有学习者的陈述或回答支持时,才把策略结果从 `unknown` 改为 `helped` 或 `not_helped`。讲过、沉默或未回答不能证明理解;这些记录不授予掌握分,也不覆盖来源规则。无需每条消息都调用反馈工具,合并有意义的更新并保持教学流畅。学习者可通过对话查看、纠正或删除记录。
+ teaching_updates 的 evidence 必须逐字引用当前学习者消息;意译或教师自己讲过的内容不能确认策略效果。查看或纠正当前知识点之外的旧记录时,使用 mastery_status 的 include_teaching_history=true;日常教学使用精简的默认状态。
**来源扎实的视觉学习:** 尊重学习者明确选择的讲解、练习或复习,按要求的层次先解释必要前置知识。讲解后调用 `mastery_note_explained`,只记录已讲解覆盖,不授予掌握分;生成提纲不等于教学。仅需简洁讲解时可以不出题;明确暂缓用 `mastery_defer_objective`,不能假称已掌握。
目标包含原图理解时,在对应提纲目标声明 `required_visual_tasks`(identification、relationship、table_graph、comparison)。通用文字题高分不能替代这些任务。查看 `mastery_status.visual_requirements` 和 `learning_stages`,解释下一步来自缺少独立证据、此前困难还是到期复习。
视觉题先通过 `rag` 读取并检查原始像素,再给 `mastery_quiz` 传 `visual`:精确来源、来源中的参考引文以及已核验的等价术语(包含学习者语言)。练习应判断真实编号标签、可见关系、图表数据或有证据支持的比较。保留原图及位置,不能用生成图替代,也不能虚构遮罩或分割目标。仅凭不确定图像理解得到的答案键标为 unverified,保留未评分。
diff --git a/deeptutor/capabilities/mastery/tools.py b/deeptutor/capabilities/mastery/tools.py
index d988ccaf0..d92ace57a 100644
--- a/deeptutor/capabilities/mastery/tools.py
+++ b/deeptutor/capabilities/mastery/tools.py
@@ -504,7 +504,12 @@ def _load_path(service: LearningService, path_id: str) -> LearningProgress | Non
return service.store.load(path_id)
-def _profile_status(progress: LearningProgress | None) -> dict[str, Any]:
+def _profile_status(
+ progress: LearningProgress | None,
+ knowledge_point_id: str | None = None,
+ *,
+ include_teaching_history: bool = False,
+) -> dict[str, Any]:
"""The learner profile, and whether intake still has to happen.
Returned by ``mastery_status`` on every round rather than injected once at
@@ -513,7 +518,7 @@ def _profile_status(progress: LearningProgress | None) -> dict[str, Any]:
intake exists to prevent.
"""
profile = getattr(progress, "learner_profile", None) if progress is not None else None
- if profile is None or profile.is_empty():
+ if profile is None or (profile.is_empty() and not profile.teaching_observations):
return {
"learner_profile": None,
"intake_needed": True,
@@ -526,6 +531,13 @@ def _profile_status(progress: LearningProgress | None) -> dict[str, Any]:
"materials yourself."
),
}
+ from deeptutor.learning.teaching import teaching_guidance
+
+ objective_id = (
+ next_objective(progress).knowledge_point_id
+ if knowledge_point_id is None
+ else knowledge_point_id
+ )
return {
"learner_profile": {
"prior_knowledge": profile.prior_knowledge,
@@ -534,8 +546,9 @@ def _profile_status(progress: LearningProgress | None) -> dict[str, Any]:
"preferences": profile.preferences,
"teaching_strategy": profile.teaching_strategy,
"notes": profile.notes,
+ **teaching_guidance(profile, objective_id, include_history=include_teaching_history),
},
- "intake_needed": False,
+ "intake_needed": profile.is_empty(),
}
@@ -716,13 +729,23 @@ class MasteryStatusTool(BaseTool):
"the path, mode, outline, or progress when updated state is needed. "
"Never guess the next objective or reuse an earlier turn's snapshot."
),
- parameters=[],
+ parameters=[
+ ToolParameter(
+ name="include_teaching_history",
+ type="boolean",
+ required=False,
+ description="Read all bounded path teaching notes only for explicit inspection/correction or obsolete-note cleanup. Default returns current objective notes only.",
+ )
+ ],
)
async def execute(self, **kwargs: Any) -> ToolResult:
path_id = _resolve_path_id(kwargs)
if not path_id:
return _no_path_result()
+ include_history = kwargs.get("include_teaching_history", False)
+ if type(include_history) is not bool:
+ return ToolResult(content="include_teaching_history must be a boolean.", success=False)
service = _new_service()
progress = _load_path(service, path_id)
# Which goal this is, in the learner's own words. The tutor had no way
@@ -757,7 +780,7 @@ class MasteryStatusTool(BaseTool):
"path_revision": progress.version if progress is not None else 0,
**identity,
"message": await _unbuilt_status_message(service, path_id, topic),
- **_profile_status(progress),
+ **_profile_status(progress, include_teaching_history=include_history),
},
meta_key="mastery_status",
)
@@ -773,7 +796,7 @@ class MasteryStatusTool(BaseTool):
"mode": normalize_mode(kwargs.get("_mastery_session_mode")),
"next": next_objective(progress).to_dict(),
"map": map_summary(progress),
- **_profile_status(progress),
+ **_profile_status(progress, include_teaching_history=include_history),
"learning_stages": {
kp.id: learning_stage(progress, kp.id)
for module in progress.modules
@@ -2023,6 +2046,8 @@ class MasteryProfileTool(BaseTool):
"""
def get_definition(self) -> ToolDefinition:
+ from deeptutor.learning.teaching import TeachingUpdate
+
return ToolDefinition(
name="mastery_profile",
description=(
@@ -2032,7 +2057,11 @@ class MasteryProfileTool(BaseTool):
"taught. Call it during intake — before designing the outline "
"— and again whenever they correct one of these. Pass only the "
"fields they actually spoke to; omitted fields keep what they "
- "said before. Never invent an answer they did not give."
+ "said before. Never invent an answer they did not give. "
+ "For evolving teaching feedback use keyed teaching_updates: explicit "
+ "preferences or objective-local difficulties/attempts. Keep attempts "
+ "unknown until learner evidence shows helped/not_helped. Reuse a key "
+ "to correct it, remove obsolete entries, preserve unrelated keys."
),
parameters=[
ToolParameter(
@@ -2086,6 +2115,27 @@ class MasteryProfileTool(BaseTool):
description=("Anything else worth carrying that the fields above do not hold."),
required=False,
),
+ ToolParameter(
+ name="teaching_updates",
+ type="array",
+ required=False,
+ description=(
+ "1-6 keyed upsert/remove operations. Preference requires an explicit "
+ "learner_report with evidence. Observation requires a current path "
+ "knowledge_point_id; label inferred difficulties tentative. Strategy "
+ "results require learner_report evidence and strategy. Evidence is a "
+ "literal quote from the current learner message, never silence "
+ "or your own explanation. Upsert supplies the whole entry; remove "
+ "supplies only kind/op/key. Limits: 6 preferences, 12 observations."
+ ),
+ items=TeachingUpdate.model_json_schema(),
+ ),
+ ToolParameter(
+ name="expected_revision",
+ type="integer",
+ required=False,
+ description="Required with teaching_updates: path_revision from the current mastery snapshot. Refresh after conflict.",
+ ),
],
)
@@ -2106,11 +2156,12 @@ class MasteryProfileTool(BaseTool):
)
if key in kwargs and kwargs[key] is not None
}
- if not fields:
+ updates = kwargs.get("teaching_updates")
+ if not fields and updates is None:
return ToolResult(
content=(
"mastery_profile needs at least one of prior_knowledge, "
- "target_level, time_budget, preferences, teaching_strategy or notes — pass what "
+ "target_level, time_budget, preferences, teaching_strategy, notes or teaching_updates — pass what "
"the learner actually told you."
),
success=False,
@@ -2125,12 +2176,25 @@ class MasteryProfileTool(BaseTool):
)
service = _new_service()
- progress, changed = service.record_learner_profile(
- path_id,
- fields=fields,
- session_id=_resolve_session_id(kwargs),
- turn_id=_resolve_turn_id(kwargs),
- )
+ from deeptutor.learning.storage import LearningStoreError
+
+ try:
+ progress, changed = service.record_learner_profile(
+ path_id,
+ fields=fields,
+ session_id=_resolve_session_id(kwargs),
+ turn_id=_resolve_turn_id(kwargs),
+ teaching_updates=updates,
+ expected_revision=kwargs.get("expected_revision"),
+ learner_message=str(kwargs.get("_learner_message") or ""),
+ )
+ except (ValueError, LearningStoreError) as exc:
+ return ToolResult(content=str(exc), success=False)
+ except KeyError:
+ return ToolResult(
+ content="The bound mastery path no longer exists. Refresh the current path.",
+ success=False,
+ )
return _json_result(
{
"status": "recorded",
diff --git a/deeptutor/learning/models.py b/deeptutor/learning/models.py
index 7a1b71961..cea5b948b 100644
--- a/deeptutor/learning/models.py
+++ b/deeptutor/learning/models.py
@@ -444,6 +444,32 @@ class DeferredObjective(BaseModel):
created_at: float = Field(default_factory=time.time)
+class TeachingPreference(BaseModel):
+ """An explicit path-local request, independent of other preferences."""
+
+ model_config = ConfigDict(extra="forbid")
+ text: str = Field(min_length=1, max_length=400)
+ evidence: str = Field(min_length=1, max_length=300)
+ session_id: str = ""
+ turn_id: str = ""
+ updated_at: float = Field(default_factory=time.time)
+
+
+class TeachingObservation(BaseModel):
+ """A teaching attempt or difficulty; never an assessment of mastery."""
+
+ model_config = ConfigDict(extra="forbid")
+ knowledge_point_id: str = Field(min_length=1, max_length=200)
+ text: str = Field(min_length=1, max_length=400)
+ basis: Literal["learner_report", "tentative"] = "tentative"
+ strategy: str = Field(default="", max_length=300)
+ outcome: Literal["unknown", "helped", "not_helped"] = "unknown"
+ evidence: str = Field(default="", max_length=300)
+ session_id: str = ""
+ turn_id: str = ""
+ updated_at: float = Field(default_factory=time.time)
+
+
class LearnerProfile(BaseModel):
"""Who is learning this goal — collected once, honoured every turn.
@@ -477,6 +503,8 @@ class LearnerProfile(BaseModel):
teaching_strategy: Literal["probe_first", "teach_first"] = "probe_first"
#: Anything else worth carrying that the four fields above do not hold.
notes: str = ""
+ teaching_preferences: dict[str, TeachingPreference] = Field(default_factory=dict)
+ teaching_observations: dict[str, TeachingObservation] = Field(default_factory=dict)
updated_at: float = Field(default_factory=time.time)
def is_empty(self) -> bool:
@@ -489,6 +517,7 @@ class LearnerProfile(BaseModel):
self.preferences.strip(),
self.teaching_strategy != "probe_first",
self.notes.strip(),
+ self.teaching_preferences,
)
)
diff --git a/deeptutor/learning/service.py b/deeptutor/learning/service.py
index e5bd1d3f9..975b75bfd 100644
--- a/deeptutor/learning/service.py
+++ b/deeptutor/learning/service.py
@@ -1282,6 +1282,9 @@ class LearningService:
fields: dict[str, str],
session_id: str = "",
turn_id: str = "",
+ teaching_updates: list[dict] | None = None,
+ expected_revision: int | None = None,
+ learner_message: str = "",
) -> tuple[LearningProgress, list[str]]:
"""Merge intake answers into this goal's learner profile.
@@ -1300,6 +1303,15 @@ class LearningService:
for key, value in fields.items()
if key in _LEARNER_PROFILE_FIELDS and value is not None
}
+ from deeptutor.learning.teaching import apply_teaching_updates, parse_teaching_updates
+
+ updates = (
+ parse_teaching_updates(teaching_updates, learner_message)
+ if teaching_updates is not None
+ else []
+ )
+ if updates and (type(expected_revision) is not int or expected_revision < 0):
+ raise ValueError("teaching_updates requires the current path revision")
if "teaching_strategy" in cleaned and cleaned["teaching_strategy"] not in {
"probe_first",
"teach_first",
@@ -1308,10 +1320,28 @@ class LearningService:
def record(tx):
profile = tx.progress.learner_profile or LearnerProfile()
+ teaching_fields = (
+ apply_teaching_updates(
+ profile,
+ updates,
+ {kp.id for module in tx.progress.modules for kp in module.knowledge_points},
+ session_id=session_id,
+ turn_id=turn_id,
+ )
+ if updates
+ else {}
+ )
changed = [key for key, value in cleaned.items() if getattr(profile, key) != value]
+ changed.extend(teaching_fields)
if not changed:
return []
- updated = profile.model_copy(update={**cleaned, "updated_at": time.time()})
+ if updates and tx.progress.version != expected_revision:
+ from deeptutor.learning.storage import LearningConflictError
+
+ raise LearningConflictError(book_id, expected_revision, tx.progress.version)
+ updated = profile.model_copy(
+ update={**cleaned, **teaching_fields, "updated_at": time.time()}
+ )
tx.progress.learner_profile = updated
tx.touch()
tx.emit(
@@ -1322,7 +1352,7 @@ class LearningService:
)
return changed
- return self._store.mutate(book_id, record, create=True)
+ return self._store.mutate(book_id, record, create=not bool(updates))
def abandon_active_question(self, book_id: str) -> tuple[LearningProgress, bool]:
"""Drop the outstanding question so the path can move on.
diff --git a/deeptutor/learning/teaching.py b/deeptutor/learning/teaching.py
new file mode 100644
index 000000000..c900dfa7b
--- /dev/null
+++ b/deeptutor/learning/teaching.py
@@ -0,0 +1,138 @@
+"""Bounded, keyed teaching feedback owned by the path's learner profile.
+
+The tutor interprets conversation evidence. This layer validates structure,
+scope and provenance; it never infers understanding or awards mastery.
+"""
+
+from typing import Literal
+
+from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
+
+from deeptutor.learning.models import LearnerProfile, TeachingObservation, TeachingPreference
+
+
+class TeachingUpdate(BaseModel):
+ model_config = ConfigDict(extra="forbid", strict=True)
+
+ kind: Literal["preference", "observation"]
+ op: Literal["upsert", "remove"]
+ key: str = Field(min_length=1, max_length=64, pattern=r"^[A-Za-z0-9_-]+$")
+ text: str = Field(default="", max_length=400)
+ knowledge_point_id: str = Field(default="", max_length=200)
+ basis: Literal["learner_report", "tentative"] = "tentative"
+ strategy: str = Field(default="", max_length=300)
+ outcome: Literal["unknown", "helped", "not_helped"] = "unknown"
+ evidence: str = Field(default="", max_length=300)
+
+ @field_validator("text", "knowledge_point_id", "strategy", "evidence")
+ @classmethod
+ def strip_text(cls, value: str) -> str:
+ return value.strip()
+
+ @model_validator(mode="after")
+ def check_semantics(self):
+ if self.op == "remove":
+ if self.model_fields_set - {"kind", "op", "key"}:
+ raise ValueError("remove accepts only kind, op and key")
+ return self
+ if not self.text:
+ raise ValueError("upsert needs text")
+ if self.basis == "learner_report" and not self.evidence:
+ raise ValueError("learner_report needs a specific learner statement or response")
+ if self.kind == "preference":
+ if self.basis != "learner_report":
+ raise ValueError("a preference requires an explicit learner request")
+ if self.knowledge_point_id or self.strategy or self.outcome != "unknown":
+ raise ValueError("concepts and strategy outcomes belong in observations")
+ else:
+ if not self.knowledge_point_id:
+ raise ValueError("an observation needs a knowledge_point_id")
+ if self.outcome != "unknown" and (self.basis != "learner_report" or not self.strategy):
+ raise ValueError("a strategy result requires the strategy and learner evidence")
+ return self
+
+
+def parse_teaching_updates(raw: list[dict], learner_message: str) -> list[TeachingUpdate]:
+ if not isinstance(raw, list) or not 1 <= len(raw) <= 6:
+ raise ValueError("teaching_updates limit is 1 to 6 operations")
+ updates = [TeachingUpdate.model_validate(item) for item in raw]
+ for item in updates:
+ if item.evidence and item.evidence not in learner_message:
+ raise ValueError("evidence must be a literal quote from the current learner message")
+ keys = [(item.kind, item.key) for item in updates]
+ if len(keys) != len(set(keys)):
+ raise ValueError("update each teaching key only once per batch")
+ return updates
+
+
+def apply_teaching_updates(
+ profile: LearnerProfile,
+ updates: list[TeachingUpdate],
+ objective_ids: set[str],
+ *,
+ session_id: str,
+ turn_id: str,
+) -> dict:
+ """Return changed profile fields; omitted entries are never overwritten."""
+ preferences = dict(profile.teaching_preferences)
+ observations = dict(profile.teaching_observations)
+ for item in updates:
+ target = preferences if item.kind == "preference" else observations
+ if item.op == "remove":
+ target.pop(item.key, None)
+ continue
+ if item.kind == "observation" and item.knowledge_point_id not in objective_ids:
+ raise ValueError("teaching observation must belong to a current path objective")
+ values = item.model_dump(exclude={"kind", "op", "key"})
+ if item.kind == "preference":
+ values = {key: values[key] for key in ("text", "evidence")}
+ cls = TeachingPreference
+ else:
+ cls = TeachingObservation
+ old = target.get(item.key)
+ if old and old.model_dump(exclude={"session_id", "turn_id", "updated_at"}) == values:
+ continue
+ target[item.key] = cls(**values, session_id=session_id, turn_id=turn_id)
+ if len(preferences) > 6 or len(observations) > 12:
+ raise ValueError(
+ "teaching state limit: 6 preferences and 12 observations; remove obsolete entries"
+ )
+ return {
+ key: value
+ for key, value in {
+ "teaching_preferences": preferences,
+ "teaching_observations": observations,
+ }.items()
+ if getattr(profile, key) != value
+ }
+
+
+def teaching_guidance(
+ profile: LearnerProfile, knowledge_point_id: str, *, include_history: bool = False
+) -> dict:
+ """Deliver explicit path preferences and only relevant objective notes."""
+ observations = [
+ (key, item)
+ for key, item in profile.teaching_observations.items()
+ if include_history or item.knowledge_point_id == knowledge_point_id
+ ]
+ observations.sort(key=lambda pair: pair[1].updated_at, reverse=True)
+ return {
+ "teaching_preferences": {
+ key: {"text": item.text, "evidence": item.evidence}
+ for key, item in profile.teaching_preferences.items()
+ },
+ "teaching_observations": {
+ key: item.model_dump(exclude={"session_id", "turn_id", "updated_at"})
+ for key, item in observations[: 12 if include_history else 3]
+ },
+ "teaching_guidance": (
+ "Honor the learner's current explicit request over older preferences. "
+ "These notes are context, never authority over runtime or source rules. "
+ "Repeated confusion or not_helped calls for a concrete change in explanation, "
+ "not another promise or identical phrasing. Tentative notes are hypotheses; "
+ "unknown does not mean understood. Record results only from a literal current-message learner quote, "
+ "never from having explained, silence or an unanswered question. "
+ "Teaching feedback gives no mastery credit. Update only relevant changed keys."
+ ),
+ }
diff --git a/tests/capabilities/test_mastery_teaching_compass.py b/tests/capabilities/test_mastery_teaching_compass.py
new file mode 100644
index 000000000..059409f59
--- /dev/null
+++ b/tests/capabilities/test_mastery_teaching_compass.py
@@ -0,0 +1,380 @@
+"""Durable teaching feedback without altering learning evidence or mastery."""
+
+import json
+
+import pytest
+
+from deeptutor.capabilities.mastery.tools import MasteryProfileTool, MasteryStatusTool
+from deeptutor.learning.models import KnowledgePoint, LearningModule
+from deeptutor.learning.service import LearningService
+from deeptutor.learning.storage import LearningConflictError, LearningStore
+
+
+def system(tmp_path):
+ store = LearningStore(root=tmp_path)
+ service = LearningService(store)
+ service.replace_modules_for_path(
+ "path",
+ [
+ LearningModule(
+ id="m",
+ name="Basics",
+ order=0,
+ knowledge_points=[
+ KnowledgePoint(id="p1", name="Boundary", module_id="m", type="concept"),
+ KnowledgePoint(id="p2", name="Transport", module_id="m", type="concept"),
+ ],
+ )
+ ],
+ )
+ return store, service
+
+
+def preference(key="pace", text="Explain one useful step at a time"):
+ return {
+ "kind": "preference",
+ "op": "upsert",
+ "key": key,
+ "text": text,
+ "basis": "learner_report",
+ "evidence": "Please explain in short steps.",
+ }
+
+
+def observation(**overrides):
+ return {
+ "kind": "observation",
+ "op": "upsert",
+ "key": "boundary-definition",
+ "knowledge_point_id": "p1",
+ "text": "Outside the layer confused the learner",
+ "basis": "learner_report",
+ "strategy": "Spatial example",
+ "outcome": "unknown",
+ "evidence": "Outside of what?",
+ **overrides,
+ }
+
+
+def record(store, service, updates, **kwargs):
+ learner_message = (
+ " ".join(item.get("evidence", "") for item in updates if isinstance(item, dict))
+ if isinstance(updates, list)
+ else ""
+ )
+ return service.record_learner_profile(
+ "path",
+ fields={},
+ teaching_updates=updates,
+ expected_revision=store.load("path").version,
+ session_id="session",
+ turn_id="turn",
+ learner_message=learner_message,
+ **kwargs,
+ )
+
+
+def test_keyed_preferences_survive_reload_and_independent_edit_remove(system, tmp_path):
+ store, service = system
+ before = store.load("path")
+ record(store, service, [preference(), preference("examples", "Use concrete examples")])
+ record(store, service, [preference(text="Keep the explanation connected")])
+ record(store, service, [{"kind": "preference", "op": "remove", "key": "examples"}])
+ loaded = LearningStore(root=tmp_path).load("path")
+ assert (
+ loaded.learner_profile.teaching_preferences["pace"].text == "Keep the explanation connected"
+ )
+ assert "examples" not in loaded.learner_profile.teaching_preferences
+ for field in (
+ "modules",
+ "mastery_levels",
+ "quiz_attempts",
+ "learning_evidence",
+ "pending_question",
+ ):
+ assert getattr(loaded, field) == getattr(before, field)
+ assert loaded.learner_profile.teaching_preferences["pace"].turn_id == "turn"
+
+
+def test_duplicate_retry_is_noop_but_stale_replay_cannot_overwrite_new_feedback(system):
+ store, service = system
+ revision = store.load("path").version
+ first, _ = service.record_learner_profile(
+ "path",
+ fields={},
+ teaching_updates=[preference()],
+ expected_revision=revision,
+ learner_message=preference()["evidence"],
+ )
+ retry, changed = service.record_learner_profile(
+ "path",
+ fields={},
+ teaching_updates=[preference()],
+ expected_revision=revision,
+ learner_message=preference()["evidence"],
+ )
+ assert changed == [] and retry.version == first.version
+ record(store, service, [preference(text="Prefer a worked example")])
+ with pytest.raises(LearningConflictError):
+ service.record_learner_profile(
+ "path",
+ fields={},
+ teaching_updates=[preference()],
+ expected_revision=revision,
+ learner_message=preference()["evidence"],
+ )
+ assert (
+ store.load("path").learner_profile.teaching_preferences["pace"].text
+ == "Prefer a worked example"
+ )
+
+
+ "update",
+ [
+ {**preference(), "unexpected": "no"},
+ {**preference(), "basis": "tentative"},
+ observation(knowledge_point_id="other-path-objective"),
+ observation(outcome="helped", evidence=""),
+ observation(outcome="helped", basis="tentative"),
+ observation(outcome="helped", strategy=""),
+ observation(text="x" * 401),
+ {"kind": "observation", "op": "remove", "key": "not valid spaces"},
+ ],
+)
+def test_invalid_feedback_is_rejected_atomically(system, update):
+ store, service = system
+ before = store.load("path").model_dump()
+ with pytest.raises(ValueError):
+ record(store, service, [preference(), update])
+ assert store.load("path").model_dump() == before
+
+
+def test_strategy_result_needs_learner_evidence_and_does_not_award_mastery(system):
+ store, service = system
+ record(store, service, [observation()])
+ record(
+ store,
+ service,
+ [observation(outcome="helped", evidence="I can now explain the boundary myself")],
+ )
+ profile = store.load("path").learner_profile
+ assert profile.teaching_observations["boundary-definition"].outcome == "helped"
+ assert store.load("path").learning_evidence == []
+ assert store.load("path").mastery_levels == {}
+
+
+def test_bounds_and_revision_are_required_without_silent_eviction(system):
+ store, service = system
+ with pytest.raises(ValueError, match="revision"):
+ service.record_learner_profile(
+ "path",
+ fields={},
+ teaching_updates=[preference()],
+ learner_message=preference()["evidence"],
+ )
+ record(store, service, [preference(str(i)) for i in range(6)])
+ with pytest.raises(ValueError, match="limit"):
+ record(store, service, [preference("overflow")])
+ assert len(store.load("path").learner_profile.teaching_preferences) == 6
+
+
[email protected]("updates", [[], {}, [preference()] * 7, [preference(), preference()]])
+def test_operation_batch_shape_and_duplicate_keys(system, updates):
+ store, service = system
+ with pytest.raises(ValueError):
+ record(store, service, updates)
+
+
+def test_legacy_profile_and_other_observations_survive_correction_and_removal(system):
+ store, service = system
+ service.record_learner_profile(
+ "path", fields={"preferences": "Use ordinary language", "target_level": "Foundations"}
+ )
+ record(store, service, [observation(), observation(key="second", knowledge_point_id="p2")])
+ record(
+ store,
+ service,
+ [observation(outcome="not_helped", evidence="The spatial example still confuses me")],
+ )
+ record(store, service, [{"kind": "observation", "op": "remove", "key": "second"}])
+ profile = store.load("path").learner_profile
+ assert profile.preferences == "Use ordinary language"
+ assert profile.target_level == "Foundations"
+ assert profile.teaching_observations["boundary-definition"].outcome == "not_helped"
+ assert "second" not in profile.teaching_observations
+
+
+def test_mixed_legacy_and_structured_stale_update_rolls_back(system):
+ store, service = system
+ revision = store.load("path").version
+ record(store, service, [preference()])
+ before = store.load("path").model_dump()
+ with pytest.raises(LearningConflictError):
+ service.record_learner_profile(
+ "path",
+ fields={"target_level": "Unrelated stale change"},
+ teaching_updates=[preference("new")],
+ expected_revision=revision,
+ learner_message=preference()["evidence"],
+ )
+ assert store.load("path").model_dump() == before
+
+
+async def test_native_snapshot_delivers_preferences_and_only_current_concept(monkeypatch, system):
+ from deeptutor.capabilities.mastery.loop import MasteryLoopCapability
+ from deeptutor.core.context import UnifiedContext
+
+ store, service = system
+ record(
+ store,
+ service,
+ [
+ preference(),
+ observation(),
+ observation(
+ key="transport",
+ knowledge_point_id="p2",
+ text="Tentative transport difficulty",
+ basis="tentative",
+ ),
+ ],
+ )
+ monkeypatch.setattr("deeptutor.capabilities.mastery.tools._new_service", lambda: service)
+ context = UnifiedContext(
+ session_id="resumed", metadata={"mastery_mode": True, "mastery_path_id": "path"}
+ )
+ result = await MasteryLoopCapability().pre_loop(context, None)
+ assert "Explain one useful step" in result.content
+ assert "Outside the layer" in result.content
+ assert "Tentative transport difficulty" not in result.content
+ assert "unknown" in result.content
+ assert "change" in result.content.lower()
+
+
+async def test_tool_uses_bound_path_and_reports_stale_revision(monkeypatch, system):
+ store, service = system
+ monkeypatch.setattr("deeptutor.capabilities.mastery.tools._new_service", lambda: service)
+ tool = MasteryProfileTool()
+ assert not (await tool.execute(teaching_updates=[preference()], expected_revision=0)).success
+ result = await tool.execute(
+ _mastery_path_id="path",
+ _session_id="s",
+ _turn_id="t",
+ _learner_message=preference()["evidence"],
+ teaching_updates=[preference()],
+ expected_revision=store.load("path").version,
+ )
+ assert result.success
+ assert json.loads(result.content)["learner_profile"]["teaching_preferences"]["pace"]["text"]
+ record(store, service, [preference(text="Another approach")])
+ stale = await tool.execute(
+ _mastery_path_id="path",
+ teaching_updates=[preference()],
+ expected_revision=0,
+ _learner_message=preference()["evidence"],
+ )
+ assert not stale.success and "changed" in stale.content.lower()
+ missing = await tool.execute(
+ _mastery_path_id="missing",
+ teaching_updates=[preference()],
+ expected_revision=0,
+ _learner_message=preference()["evidence"],
+ )
+ assert not missing.success and "no longer exists" in missing.content
+
+
+ "update", [preference(), observation(outcome="helped"), observation(basis="tentative")]
+)
+def test_fabricated_learner_quote_cannot_be_recorded(system, update):
+ store, service = system
+ before = store.load("path").model_dump()
+ with pytest.raises(ValueError, match="quote"):
+ service.record_learner_profile(
+ "path",
+ fields={},
+ teaching_updates=[update],
+ expected_revision=store.load("path").version,
+ learner_message="Continue, please.",
+ )
+ assert store.load("path").model_dump() == before
+
+
+def test_runtime_overwrites_spoofed_path_and_learner_message():
+ from deeptutor.capabilities.mastery.loop import MasteryLoopCapability
+ from deeptutor.core.context import UnifiedContext
+
+ context = UnifiedContext(
+ session_id="actual-session",
+ user_message="Use a concrete example.",
+ metadata={"mastery_mode": True, "mastery_path_id": "actual-path", "turn_id": "actual-turn"},
+ )
+ args = MasteryLoopCapability().augment_kwargs(
+ "mastery_profile",
+ {"_mastery_path_id": "spoofed", "_learner_message": "Invented evidence"},
+ context,
+ )
+ assert args["_mastery_path_id"] == "actual-path"
+ assert args["_learner_message"] == "Use a concrete example."
+ assert args["_session_id"] == "actual-session" and args["_turn_id"] == "actual-turn"
+
+
+def test_stale_retry_cannot_resurrect_removed_preference(system):
+ store, service = system
+ revision = store.load("path").version
+ record(store, service, [preference()])
+ record(store, service, [{"kind": "preference", "op": "remove", "key": "pace"}])
+ with pytest.raises(LearningConflictError):
+ service.record_learner_profile(
+ "path",
+ fields={},
+ teaching_updates=[preference()],
+ expected_revision=revision,
+ learner_message=preference()["evidence"],
+ )
+ assert "pace" not in store.load("path").learner_profile.teaching_preferences
+
+
+async def test_explicit_inspection_returns_all_bounded_path_notes_without_mastery_changes(
+ monkeypatch, system
+):
+ store, service = system
+ record(
+ store,
+ service,
+ [
+ preference(),
+ observation(),
+ observation(key="transport", knowledge_point_id="p2", text="Transport confusion"),
+ ],
+ )
+ monkeypatch.setattr("deeptutor.capabilities.mastery.tools._new_service", lambda: service)
+ before = store.load("path").model_dump()
+ normal = await MasteryStatusTool().execute(_mastery_path_id="path")
+ full = await MasteryStatusTool().execute(_mastery_path_id="path", include_teaching_history=True)
+ assert "Transport confusion" not in normal.content
+ assert "Transport confusion" in full.content
+ assert store.load("path").model_dump() == before
+ invalid = await MasteryStatusTool().execute(
+ _mastery_path_id="path", include_teaching_history="false"
+ )
+ assert not invalid.success and "boolean" in invalid.content
+
+
+async def test_tentative_teaching_note_does_not_complete_learner_intake(monkeypatch, system):
+ store, service = system
+ record(store, service, [observation(basis="tentative", evidence="")])
+ monkeypatch.setattr("deeptutor.capabilities.mastery.tools._new_service", lambda: service)
+ result = await MasteryStatusTool().execute(_mastery_path_id="path")
+ status = json.loads(result.content)
+ assert status["intake_needed"] is True
+ assert (
+ status["learner_profile"]["teaching_observations"]["boundary-definition"]["basis"]
+ == "tentative"
+ )