DeepTutor: Lifelong Personalized Tutoring. https://deeptutor.info/.
Python ai-agentsai-tutorclawbotcli-tooldeepresearchinteractive-learninglearningmulti-agent-systemsrag
feat(mastery): complete source-grounded visual practice and evidence (#1611)
Bingxi Zhao (Frank)2350ea2dba65
diff --git a/deeptutor/agents/loop/agent_loop.py b/deeptutor/agents/loop/agent_loop.pyindex 1bd08b2a6..7f1a76821 100644--- a/deeptutor/agents/loop/agent_loop.py+++ b/deeptutor/agents/loop/agent_loop.py@@ -49,6 +49,7 @@ from deeptutor.core.trace import build_trace_metadata, merge_trace_metadata, newfrom deeptutor.runtime.agentic.messages import (assistant_message_with_tool_calls,extend_transient_model_messages,+ image_content_hashes,with_transient_model_messages,)from deeptutor.runtime.agentic.think_stream import InlineThinkFilter@@ -298,6 +299,7 @@ class AgentLoop:# #1611: retrieved pixels belong to this turn's requests, never to# durable history. Both loop engines share the same ordering/budget.self._transient_model_messages: list[dict[str, Any]] = []+ context.extension("source_visual_evidence")["image_hashes"] = []self._request_fingerprint = (context.runtime.previous_model_turn or {}).get("request_fingerprint")@@ -1266,6 +1268,11 @@ class AgentLoop:self._create_response_stream(kwargs, trace_meta, stage),_STREAM_IDLE_TIMEOUT_SECONDS,)+ # A declared vision capability is not evidence of delivered+ # pixels when the provider actually falls back to text.+ self.context.extension("source_visual_evidence")["image_hashes"] = (+ image_content_hashes(kwargs["messages"])+ )# Retain an actual provider fallback for later rounds. The# request-only image deduplication must never overwrite history.if sent_had_images and not has_image_parts(kwargs["messages"]):diff --git a/deeptutor/capabilities/mastery/loop.py b/deeptutor/capabilities/mastery/loop.pyindex 65f45a7e6..cfc870c0f 100644--- a/deeptutor/capabilities/mastery/loop.py+++ b/deeptutor/capabilities/mastery/loop.py@@ -266,6 +266,10 @@ class MasteryLoopCapability:state["quiz_awaiting_grade"] = Falsestate["quiz_graded"] = Trueupdated["_mastery_path_id"] = path_id+ updated["_attached_kb_names"] = list(context.knowledge_bases)+ updated["_inspected_image_hashes"] = list(+ context.extension("source_visual_evidence").get("image_hashes", [])+ )# Raw, not normalised: "this conversation never recorded a mode"# has to survive down to the tools, or every pre-modes conversation# (and every CLI / SDK turn, which pass none) would be enforced asdiff --git a/deeptutor/capabilities/mastery/mode.py b/deeptutor/capabilities/mastery/mode.pyindex de1d07e79..60a99ac5f 100644--- a/deeptutor/capabilities/mastery/mode.py+++ b/deeptutor/capabilities/mastery/mode.py@@ -77,6 +77,7 @@ TOOL_MODES: dict[str, frozenset[str]] = {"mastery_skip_question": frozenset({STUDY, REVIEW}),"mastery_repair_question": frozenset({STUDY, REVIEW}),"mastery_defer_objective": frozenset({STUDY, REVIEW}),+ "mastery_note_explained": frozenset({STUDY, REVIEW}),}diff --git a/deeptutor/capabilities/mastery/prompts/en/mastery_loop.yaml b/deeptutor/capabilities/mastery/prompts/en/mastery_loop.yamlindex bc3fa186e..519021a8d 100644--- a/deeptutor/capabilities/mastery/prompts/en/mastery_loop.yaml+++ b/deeptutor/capabilities/mastery/prompts/en/mastery_loop.yaml@@ -86,6 +86,12 @@ session:# Teaching strategy. Kept separate from the loop protocol above: that one says# how a round runs, this one says what to do about the objective in front of you.playbook: |-+ **Source-grounded visual study:** Honor explanation, practice and review as explicit learner choices; explain untaught prerequisites at the requested level before assessment. Use `mastery_note_explained` after an explanation; this records coverage without mastery credit. An outline is never evidence of teaching. A concise explanation-only turn may end without a quiz; explicit deferral uses `mastery_defer_objective` and does not claim mastery.+ When the goal includes figure understanding, declare the relevant objective's `required_visual_tasks` in the outline (`identification`, `relationship`, `table_graph`, `comparison`). Generic text scores do not satisfy these. Read `mastery_status.visual_requirements` and `learning_stages`; explain whether the next work addresses missing independent evidence, prior difficulty or due review.+ Retrieve and inspect the original pixels with `rag` before a visual question. Pass `visual` to `mastery_quiz`, using exact source references and a supported reference quote, plus verified equivalent terminology (including the learner's language). Ask about the actual numbered label, visible relationship, plotted/table values or supported comparison. Preserve the original and source location; do not replace it with a generated diagram or invent a masked/segmented target. A key based only on uncertain pixel interpretation stays `unverified` and ungraded.+ Describe visible answer cues honestly. A labeled image or direct table/page reading is guided practice; do not claim independent recall. Repeating recently assisted work is also weakened through the existing retention boundary. Unknown wording asks for clarification rather than fuzzy grading; use a well-grounded choice question when a supported wrong answer must be distinguished reliably. Explain mistakes against the original source and offer a different supported example, not an immediate loop of the same question.+ The learner may challenge an assessment. Inspect its question ID, stored answer and source; use `mastery_repair_question` to invalidate or correct an unsupported assessment and recompute its effects. A challenge alone is not proof of mastery; a manual override has separate learner provenance. Changed/missing sources, ambiguity, unreadable labels and uncertain keys produce no negative evidence. Keep the source identity, objective, task and assistance together on every follow-up.+Once `mastery_status` has answered, act on the current objective:- **`intake_needed` is true — ask before you design.** This learner has never told you who they are, and an outline built without that is the same route for a second-year undergraduate and for an engineer with six free evenings. Read the materials first, then ask, in one short message, only what you cannot read off the materials yourself: what they can already do, what finishing looks like for them, how much time they have, and how they want it taught. Ask through `ask_user` — this is the clarification it exists for — and shape the questions around what the material actually turned out to be, rather than reciting the same four every time. Take whatever they give you: a learner who answers two of four has still told you something, and pressing for the rest is worse than starting. Record their answers with `mastery_profile`, inventing nothing they did not say, then design the outline against them. If they decline to answer at all, say you will start from a reasonable guess and adjust as you go — then do exactly that.diff --git a/deeptutor/capabilities/mastery/prompts/zh/mastery_loop.yaml b/deeptutor/capabilities/mastery/prompts/zh/mastery_loop.yamlindex b82895658..8ffcb6a95 100644--- a/deeptutor/capabilities/mastery/prompts/zh/mastery_loop.yaml+++ b/deeptutor/capabilities/mastery/prompts/zh/mastery_loop.yaml@@ -83,6 +83,12 @@ session:# 教学策略。与 loop 协议分开:上面说的是「一轮怎么走」,这里说的是「针对这个知识点做什么」。playbook: |-+ **来源扎实的视觉学习:** 尊重学习者明确选择的讲解、练习或复习,按要求的层次先解释必要前置知识。讲解后调用 `mastery_note_explained`,只记录已讲解覆盖,不授予掌握分;生成提纲不等于教学。仅需简洁讲解时可以不出题;明确暂缓用 `mastery_defer_objective`,不能假称已掌握。+ 目标包含原图理解时,在对应提纲目标声明 `required_visual_tasks`(identification、relationship、table_graph、comparison)。通用文字题高分不能替代这些任务。查看 `mastery_status.visual_requirements` 和 `learning_stages`,解释下一步来自缺少独立证据、此前困难还是到期复习。+ 视觉题先通过 `rag` 读取并检查原始像素,再给 `mastery_quiz` 传 `visual`:精确来源、来源中的参考引文以及已核验的等价术语(包含学习者语言)。练习应判断真实编号标签、可见关系、图表数据或有证据支持的比较。保留原图及位置,不能用生成图替代,也不能虚构遮罩或分割目标。仅凭不确定图像理解得到的答案键标为 unverified,保留未评分。+ 如实记录可见答案线索:带答案标签或直接读取表格/页面是辅助练习,不是独立回忆;即时重复刚获帮助的题目也会按现有复习时间边界降权。未知表述应澄清而不是模糊打分;需要可靠区分实质性错误时使用来源明确的选择题。答错后对照原图讲解,提供其他有依据的例子,不要立即循环相同问题。+ 学习者可申诉判定。核对题目 ID、保存的答案及来源后,用 `mastery_repair_question` 作废或纠正不受支持的判定,并重算学习状态。申诉本身不是掌握证明,手动覆盖有独立的学习者来源标记。来源变化、缺失、目标歧义、标签不可读或答案键不确定都不能形成负面掌握证据。追问中保持来源、目标、任务和辅助信息的关联。+拿到 `mastery_status` 之后,针对当前知识点行动:- **`intake_needed` 为 true——先问,再设计。** 这位学习者还从没告诉过你他是谁,而不问就设计出来的大纲,对一个大二学生和一个只有六个晚上的工程师是同一条路线。先把材料读了,然后用一条简短的消息只问你自己看不出来的那几件事:他已经会什么、对他来说学到什么程度算完成、打算花多少时间、希望怎么教。用 `ask_user` 问——这正是它在这里的用途——并且要根据材料实际是什么来组织问题,而不是每次背同样四条。他给什么你就收什么:四个问题答了两个,他也已经告诉你东西了,为了凑齐而追问比直接开始更糟。用 `mastery_profile` 如实记下他说的,不要替他编造,然后照着这些来设计大纲。他要是完全不想答,就说明你会先按一个合理的估计开始、边走边调——然后真的这么做。diff --git a/deeptutor/capabilities/mastery/tools.py b/deeptutor/capabilities/mastery/tools.pyindex 26106d631..d988ccaf0 100644--- a/deeptutor/capabilities/mastery/tools.py+++ b/deeptutor/capabilities/mastery/tools.py@@ -73,9 +73,11 @@ from deeptutor.learning.policy import (gate_kind,gate_threshold,is_mastered,+ learning_stage,map_summary,next_objective,path_display_name,+ visual_achievements,)from deeptutor.learning.question_card import (QUESTION_CARD_KEY,@@ -93,6 +95,7 @@ if TYPE_CHECKING:MASTERY_TOOL_NAMES: tuple[str, ...] = ("mastery_status","mastery_quiz",+ "mastery_note_explained","mastery_grade","mastery_skip_question","mastery_repair_question",@@ -392,6 +395,7 @@ async def _sync_mastery_attempt_to_question_bank(confidence=confidence,response_time=response_time,quality=quality,+ visual_context=pending.visual_context,)try:await asyncio.wait_for(record_assessment(record), timeout=5.0)@@ -770,6 +774,23 @@ class MasteryStatusTool(BaseTool):"next": next_objective(progress).to_dict(),"map": map_summary(progress),**_profile_status(progress),+ "learning_stages": {+ kp.id: learning_stage(progress, kp.id)+ for module in progress.modules+ for kp in module.knowledge_points+ }+ if progress+ else {},+ "visual_requirements": {+ kp.id: {+ "required": kp.required_visual_tasks,+ "demonstrated": visual_achievements(progress, kp.id),+ }+ for module in progress.modules+ for kp in module.knowledge_points+ }+ if progress+ else {},}interaction = service.store.get_active_interaction(path_id)if interaction is not None:@@ -802,6 +823,65 @@ class MasteryStatusTool(BaseTool):return _json_result(payload, meta_key="mastery_status")+class MasteryNoteExplainedTool(BaseTool):+ """Record teaching without pretending that explanation is an assessment."""++ def get_definition(self) -> ToolDefinition:+ return ToolDefinition(+ name="mastery_note_explained",+ description="After explaining an objective at the requested level, record what was taught. This does not assess, grant mastery or schedule a review. Respect explanation-only requests; a quiz is optional until the learner wants practice.",+ parameters=[+ ToolParameter(+ name="knowledge_point_id",+ type="string",+ description="Exact objective ID from mastery_status.",+ ),+ ToolParameter(+ name="summary",+ type="string",+ description="What was actually explained, including prerequisites and source limitations.",+ ),+ ],+ )++ async def execute(self, **kwargs: Any) -> ToolResult:+ refusal = _wrong_mode_result("mastery_note_explained", kwargs)+ if refusal is not None:+ return refusal+ path_id = _resolve_path_id(kwargs)+ if not path_id:+ return _no_path_result()+ kp_id = str(kwargs.get("knowledge_point_id") or "")+ summary = str(kwargs.get("summary") or "").strip()[:2000]+ if not summary:+ return ToolResult(content="A teaching summary is required.", success=False)+ service = _new_service()+ if _load_path(service, path_id) is None:+ return _no_built_path_result("mastery_note_explained")++ def note(tx):+ import time++ if find_knowledge_point(tx.progress, kp_id)[0] is None:+ raise ValueError("Unknown objective; refresh mastery_status.")+ tx.progress.explained_objectives[kp_id] = {+ "summary": summary,+ "timestamp": time.time(),+ "session_id": _resolve_session_id(kwargs),+ "turn_id": _resolve_turn_id(kwargs),+ }+ tx.emit("objective.explained", {"knowledge_point_id": kp_id, "summary": summary})++ try:+ await asyncio.to_thread(service.store.mutate, path_id, note)+ except (ValueError, FileNotFoundError) as exc:+ return ToolResult(content=str(exc), success=False)+ return _json_result(+ {"knowledge_point_id": kp_id, "stage": "explained", "mastery_credit": False},+ meta_key="mastery_note_explained",+ )++class MasteryQuizTool(BaseTool):"""Register an objective-type question; the engine holds the answer."""@@ -821,6 +901,13 @@ class MasteryQuizTool(BaseTool):"mastery_assess instead."),parameters=[+ ToolParameter(+ name="visual",+ type="object",+ required=False,+ sensitive=True,+ description="Optional source-grounded task: {task: identification|relationship|table_graph|comparison, sources:[{kb_name, asset_id OR source_path/page, source_hash?, region?}], reference_quote, accepted_answers: verified equivalent terms (including other languages), key_status: verified|unverified, answer_cues: none|visible|unverified, hints_used}. Retrieve and inspect original pixels first. reference_quote and expected_answer must use the source-grounded canonical terminology; accepted_answers may add verified translations for learner replies. Uncertain or conflicting keys stay ungraded. Visible or unknown cues mean guided practice, not independent mastery. Do not invent masks or regions.",+ ),ToolParameter(name="knowledge_point_id",type="string",@@ -961,6 +1048,34 @@ class MasteryQuizTool(BaseTool):content=f"Unknown objective {kp_id!r}; call mastery_status for valid ids.",success=False,)+ visual_context: dict[str, Any] = {}+ accepted_answers: list[str] = []+ if kwargs.get("visual") is not None:+ from deeptutor.learning.visual_practice import prepare_visual++ if not isinstance(kwargs["visual"], dict):+ return ToolResult(+ content="visual must be an object with exact source references.", success=False+ )+ try:+ visual_context, accepted_answers = await asyncio.to_thread(+ prepare_visual,+ kwargs["visual"],+ expected_answer=expected,+ options={option["label"]: option["body"] for option in options},+ attached_kbs=kwargs.get("_attached_kb_names"),+ inspected_image_hashes=kwargs.get("_inspected_image_hashes", []),+ )+ from deeptutor.learning.visual_practice import account_for_recent_assistance++ visual_context = account_for_recent_assistance(+ visual_context, progress.quiz_attempts, kp_id+ )+ except (OSError, ValueError, TypeError) as exc:+ return ToolResult(+ content=f"Visual practice is unavailable: {exc}. Inspect/clarify the source; no assessment was created.",+ success=False,+ )pending = PendingQuestion(question_id=uuid.uuid4().hex,knowledge_point_id=kp_id,@@ -971,6 +1086,8 @@ class MasteryQuizTool(BaseTool):options=options,explanation=str(kwargs.get("explanation") or "").strip()[:2000],difficulty=_normalize_difficulty(kwargs.get("difficulty")),+ visual_context=visual_context,+ accepted_answers=accepted_answers,)from deeptutor.learning.service import MasteryInteractionError@@ -1197,6 +1314,7 @@ class MasteryGradeTool(BaseTool):itemfor item in reversed(evidence_items)if getattr(item, "knowledge_point_id", "") == pending.knowledge_point_id+ and getattr(item, "question_id", "") == pending.question_id),None,)@@ -1220,6 +1338,7 @@ class MasteryGradeTool(BaseTool):if evidence is not Noneelse None,quality=getattr(evidence, "quality", None) if evidence is not None else None,+ result=str(interaction.result.get("result") or ""),)mastered = bool(kp and is_mastered(progress, kp))gate = gate_kind(kp) if kp else ""@@ -1248,6 +1367,8 @@ class MasteryGradeTool(BaseTool):"exposed, and when you pose the next question with ""mastery_quiz put that call last, since it ends the turn.")+ if interaction.result.get("result") == "ungraded":+ next_move = "going with source clarification or an explanation. This was ungraded, created no negative evidence and did not change mastery. Do not turn uncertainty into a wrong answer."payload = {"is_correct": is_correct,"replayed": replayed,@@ -1268,9 +1389,19 @@ class MasteryGradeTool(BaseTool):question_id=pending.question_id,is_correct=is_correct,learner_answer=interaction.user_answer or answer,- correct_label=expected_answer,+ correct_label=expected_answer+ if interaction.result.get("result") != "ungraded"+ else "",choice_options=choice_options,- explanation=pending.explanation,+ explanation=(+ str(interaction.result.get("diagnosis"))+ if interaction.result.get("result") == "ungraded"+ else pending.explanation+ ),+ result=str(interaction.result.get("result") or ""),+ independent=interaction.result.get("independent")+ if pending.visual_context+ else None,),# A graded card is not an answer to the learner. The verdict and# the explanation are now on it, so the model that stops here@@ -2579,6 +2710,7 @@ def _revise_points(module_id=module.id,prerequisite_ids=relation_values(raw, "prerequisite_ids", kp.prerequisite_ids),topic_source_ids=relation_values(raw, "topic_source_ids", kp.topic_source_ids),+ required_visual_tasks=raw.get("required_visual_tasks", kp.required_visual_tasks),))rewrite_ids[kp.id] = points[-1].id@@ -2609,6 +2741,9 @@ def _revise_points(if isinstance(raw, dict)else []),+ required_visual_tasks=(raw.get("required_visual_tasks") or [])+ if isinstance(raw, dict)+ else [],))@@ -2676,6 +2811,9 @@ def _parse_modules_with_refs(name=kp_name,type=KnowledgeType(kp_type),module_id=module_id,+ required_visual_tasks=(raw_kp.get("required_visual_tasks") or [])+ if isinstance(raw_kp, dict)+ else [],))relation_refs[kps[-1].id] = RelationRefs(@@ -2710,6 +2848,7 @@ def _parse_modules(MASTERY_TOOL_TYPES: tuple[type[BaseTool], ...] = (MasteryStatusTool,+ MasteryNoteExplainedTool,MasteryQuizTool,MasteryGradeTool,MasterySkipQuestionTool,@@ -2736,6 +2875,7 @@ __all__ = ["MasteryLeaveTool","MasteryPathsTool","MasteryModeTool",+ "MasteryNoteExplainedTool","MasteryProfileTool","MasteryQuizTool","MasteryRepairQuestionTool",diff --git a/deeptutor/learning/assessment.py b/deeptutor/learning/assessment.pyindex 96725691b..7dfb21097 100644--- a/deeptutor/learning/assessment.py+++ b/deeptutor/learning/assessment.py@@ -108,6 +108,7 @@ class AssessmentRecord(BaseModel):created_at: float = Field(default_factory=time.time)updated_at: float = Field(default_factory=time.time)bookmarked: bool = False+ visual_context: dict[str, Any] = Field(default_factory=dict)resolved: bool | None = Noneattempt_id: str = ""diff --git a/deeptutor/learning/models.py b/deeptutor/learning/models.pyindex 555f32d7c..7a1b71961 100644--- a/deeptutor/learning/models.py+++ b/deeptutor/learning/models.py@@ -89,6 +89,9 @@ class KnowledgePoint(BaseModel):# Selected non-goal sources that justify this objective. They are curriculum# provenance, not proof of factual correctness or retrieval permission.topic_source_ids: list[str] = Field(default_factory=list)+ required_visual_tasks: list[+ Literal["identification", "relationship", "table_graph", "comparison"]+ ] = Field(default_factory=list)class LearningModule(BaseModel):@@ -132,6 +135,8 @@ class QuizAttempt(BaseModel):# attempts are excluded from mastery, errors, and spaced repetition.voided: bool = Falsevoid_reason: str = ""+ independent: bool = True+ visual_context: dict[str, Any] = Field(default_factory=dict)class RetryAttempt(BaseModel):@@ -181,6 +186,7 @@ class LearningEvidence(BaseModel):response_time: float | None = Nonesession_id: str = ""turn_id: str = ""+ visual_context: dict[str, Any] = Field(default_factory=dict)class RepetitionState(BaseModel):@@ -267,6 +273,8 @@ class PendingQuestion(BaseModel):explanation: str = ""difficulty: str = ""created_at: float = Field(default_factory=time.time)+ visual_context: dict[str, Any] = Field(default_factory=dict)+ accepted_answers: list[str] = Field(default_factory=list)@field_validator("options", mode="before")@classmethod@@ -570,6 +578,7 @@ class LearningProgress(BaseModel):# Temporarily skipped objectives. These never count as mastered; routing# just prefers any other eligible waypoint until only deferred ones remain.deferred_objectives: dict[str, DeferredObjective] = Field(default_factory=dict)+ explained_objectives: dict[str, dict[str, Any]] = Field(default_factory=dict)# A single outstanding question; grading reads its expected answer so the# model never has to recall it across turns.pending_question: PendingQuestion | None = Nonediff --git a/deeptutor/learning/policy.py b/deeptutor/learning/policy.pyindex de69c1dd1..5989000b4 100644--- a/deeptutor/learning/policy.py+++ b/deeptutor/learning/policy.py@@ -85,11 +85,47 @@ def is_assessed_mastered(progress: LearningProgress, kp: KnowledgePoint) -> bool* MEMORY / PROCEDURE: recency-weighted accuracy ≥ the type's threshold.* CONCEPT / DESIGN: a recorded qualitative pass (``mastery_assess``)."""+ if not set(kp.required_visual_tasks).issubset(visual_achievements(progress, kp.id)):+ return Falseif kp.type in QUALITATIVE_TYPES:return bool(progress.qualitative_mastery.get(kp.id, False))return progress.mastery_levels.get(kp.id, 0.0) >= gate_threshold(kp.type)+def visual_achievements(progress: LearningProgress, kp_id: str) -> list[str]:+ """Independent source-verified visual tasks, separate from generic text scores."""+ return sorted(+ {+ event.visual_context["task"]+ for event in progress.learning_evidence+ if event.knowledge_point_id == kp_id+ and event.result == "correct"+ and event.visual_context.get("task")+ and event.visual_context.get("key_status") == "verified"+ and not event.hints_used+ }+ )+++def learning_stage(progress: LearningProgress, kp_id: str) -> str:+ attempts = [+ item+ for item in progress.quiz_attempts+ if item.knowledge_point_id == kp_id and not item.voided+ ]+ evidence = [item for item in progress.learning_evidence if item.knowledge_point_id == kp_id]+ if any(item.independent and item.is_correct for item in attempts) or any(+ item.assessment_type == "qualitative" and item.result == "correct" and not item.hints_used+ for item in evidence+ ):+ return "independently_demonstrated"+ if any(not item.independent for item in attempts) or any(item.hints_used for item in evidence):+ return "practiced_with_help"+ if attempts or evidence:+ return "practiced_not_demonstrated"+ return "explained" if kp_id in progress.explained_objectives else "not_yet_covered"++def mastery_source(progress: LearningProgress, kp: KnowledgePoint) -> str:"""Return ``system`` or ``learner`` provenance for a cleared waypoint."""@@ -444,6 +480,7 @@ def objective_report({"question_id": attempt.question_id,"is_correct": attempt.is_correct,+ "independent": attempt.independent,"answer": str(attempt.user_answer or ""),"error_type": attempt.error_type.value if attempt.error_type else "","at": attempt.timestamp,@@ -479,6 +516,9 @@ def objective_report("correct_count": sum(1 for attempt in attempts if attempt["is_correct"]),# The learner's own words, kept as the evidence behind a qualitative pass."explanation": progress.feynman_explanations.get(kp_id, ""),+ "explained": progress.explained_objectives.get(kp_id),+ "required_visual_tasks": kp.required_visual_tasks,+ "demonstrated_visual_tasks": visual_achievements(progress, kp_id),"review": _review_report(progress, kp_id, state, due_at, now=moment),"errors": [{diff --git a/deeptutor/learning/question_card.py b/deeptutor/learning/question_card.pyindex ecb0fa512..7afc0ad2c 100644--- a/deeptutor/learning/question_card.py+++ b/deeptutor/learning/question_card.py@@ -66,7 +66,9 @@ def build_question_card(until the answer is committed, and this payload is rendered to the learner."""public = public_pending_question(pending)- return {+ from deeptutor.learning.visual_practice import public_visual++ payload = {"question_id": public.question_id,"prompt": public.prompt,"question_type": public.question_type,@@ -81,6 +83,9 @@ def build_question_card(# tap a letter cannot say "I don't know" or show their work."allow_free_text": True,}+ if pending.visual_context:+ payload["visual"] = public_visual(pending.visual_context)+ return payloaddef build_grade_result(@@ -91,6 +96,8 @@ def build_grade_result(correct_label: str,choice_options: dict[str, str],explanation: str,+ result: str = "",+ independent: bool | None = None,) -> dict[str, Any]:"""What the answered card shows once the gate has ruled on it.@@ -99,7 +106,7 @@ def build_grade_result(it is graded — which is exactly when they are worth reading."""label = str(correct_label or "").strip()- return {+ payload = {"question_id": str(question_id or ""),"is_correct": bool(is_correct),"learner_answer": str(learner_answer or ""),@@ -107,6 +114,11 @@ def build_grade_result("correct_body": str(choice_options.get(label) or "") if label else "","explanation": str(explanation or ""),}+ if result:+ payload["result"] = result+ if independent is not None:+ payload["independent"] = independent+ return payload__all__ = [diff --git a/deeptutor/learning/service.py b/deeptutor/learning/service.pyindex 19d4e1c97..e5bd1d3f9 100644--- a/deeptutor/learning/service.py+++ b/deeptutor/learning/service.py@@ -2,7 +2,7 @@ from __future__ import annotationsimport loggingimport time-from typing import TYPE_CHECKING, Literal+from typing import TYPE_CHECKING, Any, Literalimport uuidfrom deeptutor.learning.grading import classify_error, grade_answer@@ -82,6 +82,7 @@ def _reserved_objective_ids(progress: LearningProgress) -> set[str]:ids.update(progress.feynman_retries)ids.update(progress.feynman_explanations)ids.update(progress.deferred_objectives)+ ids.update(progress.explained_objectives)ids.update(attempt.knowledge_point_id for attempt in progress.quiz_attempts)ids.update(record.knowledge_point_id for record in progress.error_records)ids.update(task.knowledge_point_id for task in progress.review_queue)@@ -173,6 +174,8 @@ def _update_quiz_evidence_for_attempts(attempt = unmatched.pop(match_index)event.result = "correct" if attempt.is_correct else "incorrect"event.quality = 1.0 if attempt.is_correct else 0.0+ if event.hints_used and attempt.is_correct:+ event.quality = min(event.quality, 0.6)def assign_objective_identities(@@ -351,6 +354,9 @@ class LearningService:for key in list(progress.deferred_objectives.keys()):if key not in new_kp_ids:del progress.deferred_objectives[key]+ for key in list(progress.explained_objectives):+ if key not in new_kp_ids:+ del progress.explained_objectives[key]progress.error_records = [r for r in progress.error_records if r.knowledge_point_id in new_kp_ids]@@ -459,7 +465,7 @@ class LearningService:correctness = [a.is_correctfor a in progress.quiz_attempts- if a.knowledge_point_id == kp_id and not a.voided+ if a.knowledge_point_id == kp_id and not a.voided and a.independent]return compute_mastery(correctness)@@ -520,6 +526,7 @@ class LearningService:scheduler: SpacedRepetitionScheduler | None = None,session_id: str = "",turn_id: str = "",+ visual_context: dict[str, Any] | None = None,) -> bool:"""Mutate one aggregate with a grade without performing I/O."""is_correct = bool(expected_answer) and grade_answer(@@ -544,6 +551,8 @@ class LearningService:user_answer=user_answer,self_attribution=self_attribution,error_type=None if is_correct else classify_error(user_answer),+ independent=not visual_context or not visual_context.get("hints_used"),+ visual_context=visual_context or {},),)evidence = None@@ -559,6 +568,11 @@ class LearningService:turn_id=turn_id,assessment_type="review" if already_scheduled else "quiz",)+ if visual_context:+ evidence.visual_context = visual_context+ evidence.hints_used = int(visual_context.get("hints_used") or 0)+ if evidence.hints_used and is_correct:+ evidence.quality = min(evidence.quality or 1.0, 0.6)self.update_mastery(progress, knowledge_point_id, self.calculate_mastery(progress, knowledge_point_id))@@ -920,17 +934,46 @@ class LearningService:]pending.expected_answer = authoritative_answerinteraction.question = pending- is_correct = self._apply_grade(- tx.progress,- question_id=pending.question_id,- knowledge_point_id=pending.knowledge_point_id,- module_id=pending.module_id,- user_answer=graded_answer,- expected_answer=authoritative_answer,- question_type=pending.question_type,- scheduler=scheduler,- session_id=session_id,- turn_id=turn_id,+ visual_result = ""+ visual_diagnosis = ""+ if pending.visual_context:+ from deeptutor.learning.visual_practice import (+ account_for_recent_assistance,+ evaluate_visual,+ )++ pending.visual_context = account_for_recent_assistance(+ pending.visual_context, tx.progress.quiz_attempts, pending.knowledge_point_id+ )+ visual_result, visual_diagnosis = evaluate_visual(pending, raw_answer)+ if (+ tx.progress.explained_objectives.get(pending.knowledge_point_id, {}).get(+ "timestamp", 0+ )+ > pending.created_at+ ):+ pending.visual_context["hints_used"] = max(+ 1, pending.visual_context.get("hints_used", 0)+ )+ pending.visual_context["teaching_after_question"] = True+ if visual_result == "correct":+ graded_answer = authoritative_answer+ is_correct = (+ None+ if visual_result == "ungraded"+ else self._apply_grade(+ tx.progress,+ question_id=pending.question_id,+ knowledge_point_id=pending.knowledge_point_id,+ module_id=pending.module_id,+ user_answer=graded_answer,+ expected_answer=authoritative_answer,+ question_type=pending.question_type,+ scheduler=scheduler,+ session_id=session_id,+ turn_id=turn_id,+ visual_context=pending.visual_context,+ ))if (tx.progress.pending_question is not None@@ -945,9 +988,18 @@ class LearningService:"is_correct": is_correct,"knowledge_point_id": pending.knowledge_point_id,}+ if pending.visual_context:+ interaction.result.update(+ {+ "result": visual_result,+ "diagnosis": visual_diagnosis,+ "visual_context": pending.visual_context,+ "independent": not bool(pending.visual_context.get("hints_used")),+ }+ )tx.put_interaction(interaction)tx.emit(- "attempt.recorded",+ "practice.ungraded" if visual_result == "ungraded" else "attempt.recorded",{"interaction_id": interaction.interaction_id,"knowledge_point_id": pending.knowledge_point_id,@@ -956,7 +1008,7 @@ class LearningService:session_id=interaction.session_id,turn_id=interaction.turn_id,)- if tx.progress.learning_evidence:+ if visual_result != "ungraded" and tx.progress.learning_evidence:latest = tx.progress.learning_evidence[-1]if latest.knowledge_point_id == pending.knowledge_point_id:tx.emit(@@ -1317,6 +1369,7 @@ class LearningService:progress.review_queue = []progress.learner_mastery_overrides = {}progress.deferred_objectives = {}+ progress.explained_objectives = {}progress.pending_question = Noneprogress.feynman_retries = {}progress.feynman_explanations = {}@@ -1494,6 +1547,34 @@ class LearningService:question_type = question.question_type if question is not None else "short"previous_expected = question.expected_answer if question is not None else ""previous_is_correct = attempts[0].is_correct if attempts else None+ if action_name == "correct" and question is not None and question.visual_context:+ from deeptutor.learning.visual_practice import prepare_visual++ candidate_answer = str(expected_answer or question.expected_answer)+ if not question.visual_context.get("pixels_inspected"):+ raise MasteryInteractionError(+ "This visual practice had no verified pixel inspection. Void it and pose a supported replacement; a challenge cannot create mastery evidence."+ )+ try:+ corrected_visual, corrected_aliases = prepare_visual(+ {+ **question.visual_context,+ "accepted_answers": [],+ "key_status": "verified",+ },+ expected_answer=candidate_answer,+ options=question.choice_map,+ )+ except (OSError, ValueError, TypeError) as exc:+ raise MasteryInteractionError(+ "Visual correction lacks current source support; void the assessment and pose a supported replacement."+ ) from exc+ if corrected_visual["key_status"] != "verified":+ raise MasteryInteractionError(+ "Visual correction lacks a source-supported key; void the assessment instead of granting unsupported mastery."+ )+ question.accepted_answers = corrected_aliases+ question.visual_context = corrected_visualuser_answer = (interaction.user_answer if interaction is not None else "") or (str(attempts[0].user_answer or "") if attempts else "")diff --git a/deeptutor/learning/tests/test_visual_learning_workflow.py b/deeptutor/learning/tests/test_visual_learning_workflow.pynew file mode 100644index 000000000..68c84d2f7--- /dev/null+++ b/deeptutor/learning/tests/test_visual_learning_workflow.py@@ -0,0 +1,440 @@+"""Inspectable source-to-question-to-durable-evidence workflows (#1611)."""++from __future__ import annotations++from hashlib import sha256+import json+from types import SimpleNamespace++from PIL import Image, ImageDraw+import pymupdf+import pytest++from deeptutor.capabilities.mastery import tools+from deeptutor.learning.models import (+ KnowledgePoint,+ KnowledgeType,+ LearningModule,+ LearningProgress,+)+from deeptutor.learning.policy import is_assessed_mastered, visual_achievements+from deeptutor.learning.scheduler import SpacedRepetitionScheduler+from deeptutor.learning.service import LearningService+from deeptutor.learning.storage import LearningStore+from deeptutor.services.parsing.types import ParsedDocument+from deeptutor.services.rag.visual_assets import VisualAssetStore, collect_visual_assets+from deeptutor.services.session.sqlite_store import SQLiteSessionStore+++def lesson(tmp_path, monkeypatch):+ kb = tmp_path / "kb"+ (kb / "raw").mkdir(parents=True)+ source = kb / "raw" / "anatomy.pdf"+ image = tmp_path / "images" / "numbered.png"+ image.parent.mkdir()+ picture = Image.new("RGB", (400, 250), "white")+ draw = ImageDraw.Draw(picture)+ draw.ellipse((40, 40, 160, 200), outline="red", width=4)+ draw.text((95, 110), "1", fill="black")+ draw.ellipse((220, 40, 340, 200), outline="blue", width=4)+ draw.text((270, 110), "2", fill="black")+ picture.save(image)+ pdf = pymupdf.open()+ page = pdf.new_page()+ page.insert_text(+ (50, 50), "Figure 1: Number 1 refers to an artery. The artery lies left of the vein."+ )+ page.insert_image(pymupdf.Rect(50, 80, 450, 330), filename=str(image))+ page.insert_text((50, 360), "Table 1: speed (m/s), row A: 12, row B: 20. Row B is larger.")+ pdf.save(source, deflate=True)+ pdf.close()+ parsed = ParsedDocument(+ markdown="",+ asset_dir=image.parent,+ parser_signature="fixture",+ blocks=[+ {+ "type": "image",+ "img_path": str(image),+ "page_idx": 0,+ "image_caption": ["Figure 1: numbered structures"],+ "text": "Number 1 refers to an artery. The artery lies left of the vein.",+ }+ ],+ )+ candidate = collect_visual_assets(parsed, source, kb)[0]+ VisualAssetStore(kb).publish([candidate])+ monkeypatch.setattr(+ "deeptutor.multi_user.knowledge_access.resolve_for_rag",+ lambda name: SimpleNamespace(base_dir=kb.parent, name=kb.name) if name == "kb" else None,+ )+ store = LearningStore(tmp_path / "learning")+ service = LearningService(store)+ progress = LearningProgress(book_id="visual")+ kp = KnowledgePoint(+ id="identify",+ name="Identify and interpret the original source",+ type=KnowledgeType.MEMORY,+ module_id="m1",+ required_visual_tasks=["identification"],+ )+ progress.modules = [+ LearningModule(id="m1", name="Source practice", order=0, knowledge_points=[kp])+ ]+ progress.knowledge_types[kp.id] = kp.type+ service.save(progress)+ monkeypatch.setattr(tools, "_new_service", lambda: service)+ session_store = SQLiteSessionStore(tmp_path / "chat.db")+ monkeypatch.setattr(+ "deeptutor.services.session.get_sqlite_session_store", lambda: session_store+ )+ return SimpleNamespace(+ kb=kb,+ source=source,+ asset=candidate.record["asset_id"],+ image_hash=candidate.record["image_sha256"],+ service=service,+ store=store,+ session_store=session_store,+ kp=kp,+ )+++def visual(lesson, *, task="identification", cues="none", quote="Number 1 refers to an artery."):+ return {+ "task": task,+ "sources": [{"kb_name": "kb", "asset_id": lesson.asset}],+ "reference_quote": quote,+ "accepted_answers": ["artery", "an artery", "动脉"],+ "key_status": "verified",+ "answer_cues": cues,+ }+++async def pose(lesson, **overrides):+ kwargs = {+ "_mastery_path_id": "visual",+ "_attached_kb_names": ["kb"],+ "_inspected_image_hashes": [lesson.image_hash],+ "knowledge_point_id": "identify",+ "question": "Identify numbered structure 1 in the original figure.",+ "expected_answer": "artery",+ "explanation": "The source explains that structure 1 is an artery; the original numbered figure supplies the target.",+ "visual": visual(lesson),+ **overrides,+ }+ if "_inspected_image_hashes" not in overrides:+ from deeptutor.services.rag.source_visuals import retrieve_visual++ kwargs["_inspected_image_hashes"] = [+ sha256(+ retrieve_visual(+ lesson.kb,+ "kb",+ **{+ key: ref[key]+ for key in ("asset_id", "source_path", "page", "source_hash", "region")+ if ref.get(key) is not None+ },+ ).images[0][1]+ ).hexdigest()+ for ref in kwargs["visual"]["sources"]+ ]+ result = await tools.MasteryQuizTool().execute(**kwargs)+ assert result.success, result.content+ card = result.metadata["mastery_question"]+ return card+++async def test_teach_then_source_visual_practice_equivalent_answer_and_restart(lesson):+ explained = await tools.MasteryNoteExplainedTool().execute(+ _mastery_path_id="visual",+ knowledge_point_id="identify",+ summary="Explained artery/vein prerequisites concisely at beginner level using the numbered source.",+ )+ assert explained.success+ before = lesson.store.load("visual")+ assert before.explained_objectives["identify"]["summary"]+ assert before.mastery_levels.get("identify", 0) == 0+ card = await pose(lesson)+ assert card["visual"]["sources"][0]["image_url"].endswith(lesson.asset + "?dt_workspace=")+ assert "reference_quote" not in json.dumps(card) and "accepted_answers" not in json.dumps(card)+ result = await tools.MasteryGradeTool().execute(+ _mastery_path_id="visual", question_id=card["question_id"], answer="动脉"+ )+ assert result.success+ grade = result.metadata["mastery_grade"]+ assert grade["result"]["is_correct"]+ fresh = LearningService(LearningStore(lesson.store._root))+ progress = fresh.store.load("visual")+ assert progress.quiz_attempts[0].visual_context["sources"][0]["asset_id"] == lesson.asset+ assert progress.quiz_attempts[0].independent+ assert visual_achievements(progress, "identify") == ["identification"]+++async def test_assistance_immediate_repeat_and_delayed_independence_are_distinct(lesson):+ first = await pose(lesson, visual=visual(lesson, cues="visible"))+ lesson.service.grade_interaction(+ "visual",+ answer="artery",+ question_id=first["question_id"],+ scheduler=SpacedRepetitionScheduler(),+ )+ second = await pose(lesson)+ progress, _, _ = lesson.service.grade_interaction(+ "visual",+ answer="artery",+ question_id=second["question_id"],+ scheduler=SpacedRepetitionScheduler(),+ )+ assert all(not attempt.independent for attempt in progress.quiz_attempts)+ assert not is_assessed_mastered(progress, lesson.kp)+ assert progress.mastery_levels["identify"] == 0+ assert all(e.hints_used and e.quality <= 0.6 for e in progress.learning_evidence)+ for attempt in progress.quiz_attempts:+ attempt.timestamp -= 86400+ lesson.service.save(progress)+ third = await pose(lesson)+ progress, _, _ = lesson.service.grade_interaction(+ "visual",+ answer="an artery",+ question_id=third["question_id"],+ scheduler=SpacedRepetitionScheduler(),+ )+ assert progress.quiz_attempts[-1].independent+ assert visual_achievements(progress, "identify") == ["identification"]+++async def test_supported_wrong_choice_is_graded_and_faulty_assessment_repair_removes_effect(lesson):+ card = await pose(+ lesson,+ question_type="choice",+ options=[{"label": "A", "body": "artery"}, {"label": "B", "body": "vein"}],+ expected_answer="A",+ )+ wrong = next(option["label"] for option in card["options"] if option["body"] == "vein")+ progress, interaction, _ = lesson.service.grade_interaction(+ "visual",+ answer=wrong,+ question_id=card["question_id"],+ scheduler=SpacedRepetitionScheduler(),+ )+ assert interaction.result["result"] == "incorrect"+ assert progress.learning_evidence[0].result == "incorrect"+ from deeptutor.learning.service import MasteryInteractionError++ with pytest.raises(MasteryInteractionError, match="source-supported"):+ lesson.service.repair_question(+ "visual",+ question_id=card["question_id"],+ action="correct",+ expected_answer=wrong,+ reason="Learner says their answer should be accepted.",+ )+ progress, details = lesson.service.repair_question(+ "visual",+ question_id=card["question_id"],+ action="void",+ reason="Independent review found the target labeling ambiguous.",+ scheduler=SpacedRepetitionScheduler(),+ )+ assert progress.quiz_attempts[0].voided+ assert not progress.learning_evidence+ assert "identify" not in progress.repetition_states+ assert progress.mastery_levels.get("identify", 0) == 0++[email protected]("case", ["key", "wording", "changed", "conflicting_key"])+async def test_ambiguous_unverified_or_changed_evidence_creates_no_negative_mastery(lesson, case):+ v = visual(lesson)+ if case == "key":+ v["reference_quote"] = "Invented unsupported key"+ card = await pose(+ lesson, visual=v, expected_answer="vein" if case == "conflicting_key" else "artery"+ )+ if case == "changed":+ lesson.source.write_bytes(lesson.source.read_bytes() + b"changed source")+ answer = "some unverified terminology" if case == "wording" else "artery"+ progress, interaction, _ = lesson.service.grade_interaction(+ "visual",+ answer=answer,+ question_id=card["question_id"],+ scheduler=SpacedRepetitionScheduler(),+ )+ assert interaction.result["result"] == "ungraded"+ assert interaction.result["is_correct"] is None+ assert (+ not progress.quiz_attempts and not progress.learning_evidence and not progress.error_records+ )+ assert not progress.repetition_states+++async def test_retry_is_one_attempt_but_new_question_is_new_evidence_and_outline_cannot_transfer_it(+ lesson,+):+ first = await pose(lesson)+ for _ in range(2):+ lesson.service.grade_interaction(+ "visual", answer="artery", question_id=first["question_id"]+ )+ second = await pose(lesson)+ progress, _, _ = lesson.service.grade_interaction(+ "visual", answer="artery", question_id=second["question_id"]+ )+ assert len(progress.quiz_attempts) == len(progress.learning_evidence) == 2+ old_id = progress.modules[0].knowledge_points[0].id+ replacement = [+ LearningModule(+ id="m1",+ name="Different subject",+ order=0,+ knowledge_points=[+ KnowledgePoint(+ id=old_id,+ name="Unrelated geometry theorem",+ type=KnowledgeType.MEMORY,+ module_id="m1",+ )+ ],+ )+ ]+ progress = lesson.service.replace_modules_for_path("visual", replacement)+ assert progress.modules[0].knowledge_points[0].id != old_id+ assert not is_assessed_mastered(progress, progress.modules[0].knowledge_points[0])+++async def test_relationship_table_and_comparison_use_both_original_sources(lesson):+ for task, expected, quote in [+ ("relationship", "left", "The artery lies left of the vein."),+ ("table_graph", "Row B", "Row B is larger."),+ ]:+ refs = (+ [{"kb_name": "kb", "asset_id": lesson.asset}]+ if task == "relationship"+ else [{"kb_name": "kb", "source_path": "anatomy.pdf", "page": 1}]+ )+ card = await pose(+ lesson,+ expected_answer=expected,+ question="Interpret the relationship/value in the original source.",+ visual={+ "task": task,+ "sources": refs,+ "reference_quote": quote,+ "key_status": "verified",+ "accepted_answers": [expected],+ "answer_cues": "none",+ },+ )+ progress, interaction, _ = lesson.service.grade_interaction(+ "visual", answer=expected, question_id=card["question_id"]+ )+ assert interaction.result["result"] == "correct"+ assert progress.learning_evidence[-1].visual_context["task"] == task+ v = visual(lesson, task="comparison")+ v["sources"] = [v["sources"][0], {"kb_name": "kb", "source_path": "anatomy.pdf", "page": 1}]+ card = await pose(lesson, visual=v)+ assert len(card["visual"]["sources"]) == 2+++def test_generic_text_success_does_not_satisfy_required_visual_task(lesson):+ progress = lesson.store.load("visual")+ progress.mastery_levels["identify"] = 1.0+ assert not is_assessed_mastered(progress, lesson.kp)+++async def test_pdf_region_can_assess_relationship_without_displaying_the_reference_key(lesson):+ card = await pose(+ lesson,+ expected_answer="left",+ question="Describe where numbered structure 1 is relative to structure 2.",+ visual={+ "task": "relationship",+ "sources": [+ {+ "kb_name": "kb",+ "source_path": "anatomy.pdf",+ "page": 1,+ "region": [0.07, 0.08, 0.8, 0.4],+ }+ ],+ "reference_quote": "The artery lies left of the vein.",+ "key_status": "verified",+ "accepted_answers": ["left", "左侧"],+ "answer_cues": "none",+ },+ )+ assert card["visual"]["hints_used"] == 0+ assert card["visual"]["key_status"] == "verified"+ progress, interaction, _ = lesson.service.grade_interaction(+ "visual", answer="左侧", question_id=card["question_id"]+ )+ assert interaction.result["result"] == "correct"+ assert progress.quiz_attempts[-1].independent+ assert "relationship" in visual_achievements(progress, "identify")+++async def test_help_after_question_is_assisted_and_deferral_never_claims_mastery(lesson):+ card = await pose(lesson)+ await tools.MasteryNoteExplainedTool().execute(+ _mastery_path_id="visual",+ knowledge_point_id="identify",+ summary="Gave a source-based hint after the learner asked for help with structure 1.",+ )+ progress, _interaction, _ = lesson.service.grade_interaction(+ "visual", answer="artery", question_id=card["question_id"]+ )+ assert not progress.quiz_attempts[0].independent+ assert progress.learning_evidence[0].hints_used+ deferred = await tools.MasteryDeferObjectiveTool().execute(+ _mastery_path_id="visual", knowledge_point_id="identify", reason="Move on for now"+ )+ assert deferred.success+ fresh = lesson.store.load("visual")+ assert "identify" in fresh.deferred_objectives+ assert not is_assessed_mastered(fresh, lesson.kp)+++async def test_declared_vision_or_a_url_without_matching_accepted_pixels_is_guided(lesson):+ card = await pose(lesson, _inspected_image_hashes=["some_other_image"])+ assert card["visual"]["answer_cues"] == "unverified"+ progress, interaction, _replayed = lesson.service.grade_interaction(+ "visual", answer="artery", question_id=card["question_id"]+ )+ assert interaction.result["result"] == "ungraded"+ assert not progress.quiz_attempts+ assert not visual_achievements(progress, "identify")+++async def test_canonical_writer_keeps_visual_context_and_ungraded_attempt_once(lesson):+ session = await lesson.session_store.create_session()+ v = visual(lesson)+ v["key_status"] = "unverified"+ card = await pose(lesson, visual=v)+ for _ in range(2):+ result = await tools.MasteryGradeTool().execute(+ _mastery_path_id="visual",+ _session_id=session["id"],+ _turn_id="turn1",+ question_id=card["question_id"],+ answer="artery",+ )+ assert result.success+ assert result.metadata["mastery_grade"]["result"]["result"] == "ungraded"+ with lesson.session_store._connect() as connection:+ row = connection.execute("SELECT assessment_json FROM assessment_attempts").fetchall()+ assert len(row) == 1+ payload = json.loads(row[0][0])+ assert payload["result"] == "ungraded"+ assert payload["visual_context"]["sources"][0]["asset_id"] == lesson.assetdiff --git a/deeptutor/learning/visual_practice.py b/deeptutor/learning/visual_practice.pynew file mode 100644index 000000000..33aec6cbb--- /dev/null+++ b/deeptutor/learning/visual_practice.py@@ -0,0 +1,250 @@+"""Source-bound visual questions, conservative keys and explicit assistance (#1611)."""++from __future__ import annotations++from hashlib import sha256+import re+from typing import Any+import unicodedata+++def normalized_answer(value: str) -> str:+ text = unicodedata.normalize("NFKC", str(value)).casefold().strip()+ return re.sub(r"\s+", " ", text).rstrip(".。!!")+++def _answer_in_quote(answer: str, quote: str) -> bool:+ needle = normalized_answer(answer)+ if not needle:+ return False+ if needle.isascii():+ return re.search(r"(?<![a-z0-9])" + re.escape(needle) + r"(?![a-z0-9])", quote) is not None+ return needle in quote+++def prepare_visual(+ request: dict[str, Any],+ *,+ expected_answer: str,+ options: dict[str, str],+ attached_kbs: list[str] | None = None,+ inspected_image_hashes: list[str] | None = None,+) -> tuple[dict[str, Any], list[str]]:+ from deeptutor.multi_user.knowledge_access import resolve_for_rag+ from deeptutor.services.rag.kb_paths import resolve_kb_dir+ from deeptutor.services.rag.source_visuals import retrieve_visual++ task = str(request.get("task") or "identification")+ if task not in {"identification", "relationship", "table_graph", "comparison"}:+ raise ValueError(+ "Visual tasks support identification, relationship, table_graph and comparison."+ )+ refs = request.get("sources")+ if not isinstance(refs, list) or not 1 <= len(refs) <= 2:+ raise ValueError("Select one or two exact source figures/pages; comparisons require two.")+ if task == "comparison" and len(refs) != 2:+ raise ValueError("A visual comparison requires both original sources.")+ resolved = []+ source_text = []+ for ref in refs:+ if not isinstance(ref, dict):+ raise ValueError("Each visual source must be an object.")+ kb_name = str(ref.get("kb_name") or "")+ if attached_kbs is not None and kb_name not in attached_kbs:+ raise ValueError(+ "Visual practice must use a knowledge base attached to this topic/turn."+ )+ resource = resolve_for_rag(kb_name)+ if resource is None:+ raise ValueError("The selected source knowledge base is not accessible.")+ selectors = {+ key: ref[key]+ for key in ("asset_id", "source_path", "page", "region", "source_hash", "figure")+ if ref.get(key) is not None+ }+ evidence = retrieve_visual(+ resolve_kb_dir(resource.base_dir, resource.name), kb_name, **selectors+ )+ if evidence.error or not evidence.images:+ raise ValueError(f"Cannot establish original visual evidence: {evidence.content}")+ record = evidence.images[0][0]+ resolved.append(+ {+ "kb_name": kb_name,+ "asset_id": record.get("asset_id") or "",+ "source_path": record["source_path"],+ "page": record.get("page_number"),+ "source_hash": record["source_document_id"],+ "region": record.get("region"),+ "image_url": evidence.sources[0]["visual_asset_url"],+ "url": evidence.sources[0]["url"],+ "image_sha256": sha256(evidence.images[0][1]).hexdigest(),+ "visible_answer_text": _answer_in_quote(+ options.get(expected_answer, expected_answer),+ normalized_answer(record.get("text", "")),+ ),+ }+ )+ source_text.append(+ "\n".join(+ str(record.get(key) or "")+ for key in ("caption", "context", "text", "page_context", "table_html", "notes")+ )+ )+ quote = str(request.get("reference_quote") or "").strip()[:3000]+ # Answer equivalences are explicit and source-specific, never fuzzy+ # substring/keyword overlap. Unknown prose is clarified, not penalized.+ correct_body = options.get(expected_answer, expected_answer)+ raw_aliases = request.get("accepted_answers") or []+ if not isinstance(raw_aliases, list) or len(raw_aliases) > 12:+ raise ValueError("accepted_answers must contain at most 12 verified equivalents.")+ aliases = list(+ dict.fromkeys(+ [correct_body, *[str(x).strip()[:200] for x in raw_aliases if str(x).strip()]]+ )+ )+ text = normalized_answer(re.sub(r"<[^>]+>", " ", "\n".join(source_text)))+ quote_normalized = normalized_answer(re.sub(r"<[^>]+>", " ", quote))+ verified = bool(+ quote_normalized+ and quote_normalized in text+ and _answer_in_quote(correct_body, quote_normalized)+ and request.get("key_status") == "verified"+ )+ cues = str(request.get("answer_cues") or "unverified")+ if cues not in {"none", "visible", "unverified"}:+ raise ValueError(+ "answer_cues must be none, visible or unverified; masking requires independently verified regions."+ )+ # Page text can reveal a table value directly. Treat it as guided reading,+ # even when the caller mistakenly declares no visible cue.+ inspected = (+ all(ref.get("image_sha256") in inspected_image_hashes for ref in resolved)+ if inspected_image_hashes is not None+ else bool(request.get("pixels_inspected"))+ )+ if not inspected:+ verified = False+ cues = "unverified"+ elif any(+ ref.get("page")+ and not ref.get("asset_id")+ and (not ref.get("region") or ref.get("visible_answer_text"))+ for ref in resolved+ ):+ cues = "visible"+ return {+ "task": task,+ "sources": resolved,+ "answer_cues": cues,+ "key_status": "verified" if verified else "unverified",+ "pixels_inspected": inspected,+ "reference_quote": quote,+ "reference_answer": correct_body,+ "hints_used": max(int(request.get("hints_used") or 0), int(cues != "none")),+ }, aliases+++def public_visual(context: dict[str, Any]) -> dict[str, Any]:+ return {+ key: context[key]+ for key in (+ "task",+ "sources",+ "answer_cues",+ "key_status",+ "hints_used",+ "pixels_inspected",+ )+ if key in context+ }+++def account_for_recent_assistance(context: dict[str, Any], attempts, kp_id: str) -> dict[str, Any]:+ """Use the existing retention session boundary, not a visual scheduler."""+ import time++ from deeptutor.learning.scheduler import _SAME_SESSION_DAYS++ def locations(visual):+ return {(ref.get("source_hash"), ref.get("page")) for ref in visual.get("sources", [])}++ current = locations(context)+ for attempt in reversed(attempts):+ prior = attempt.visual_context+ if (+ not attempt.voided+ and attempt.knowledge_point_id == kp_id+ and (+ prior.get("hints_used")+ or (+ normalized_answer(prior.get("reference_quote", ""))+ == normalized_answer(context.get("reference_quote", ""))+ and normalized_answer(prior.get("reference_answer", ""))+ == normalized_answer(context.get("reference_answer", ""))+ )+ )+ and prior.get("task") == context.get("task")+ and locations(prior) == current+ and time.time() - attempt.timestamp < _SAME_SESSION_DAYS * 86400+ ):+ return {+ **context,+ "hints_used": max(1, context.get("hints_used", 0)),+ "recently_assisted": True,+ }+ return context+++def evaluate_visual(pending, answer: str) -> tuple[str, str]:+ """Return (result, diagnosis); uncertainty produces no mastery evidence."""+ from deeptutor.multi_user.knowledge_access import resolve_for_rag+ from deeptutor.services.rag.kb_paths import resolve_kb_dir+ from deeptutor.services.rag.source_visuals import retrieve_visual++ visual = pending.visual_context+ if not visual.get("pixels_inspected"):+ return (+ "ungraded",+ "No matching source pixels were verified in the tutor's model input. Re-retrieve the original image with a vision-capable model; this attempt does not lower mastery.",+ )+ if visual.get("key_status") != "verified" or not visual.get("sources"):+ return (+ "ungraded",+ "The reference key is not independently supported by source text; clarify it before grading.",+ )+ for ref in visual.get("sources", []):+ resource = resolve_for_rag(ref["kb_name"])+ if resource is None:+ return "ungraded", "Source access is unavailable; this attempt does not lower mastery."+ selectors = {+ key: ref[key]+ for key in ("asset_id", "source_path", "page", "region", "source_hash")+ if ref.get(key) not in (None, "")+ }+ evidence = retrieve_visual(+ resolve_kb_dir(resource.base_dir, resource.name), ref["kb_name"], **selectors+ )+ if evidence.error or not evidence.images:+ return (+ "ungraded",+ "Original source evidence changed or is unavailable; clarify the source before assessment.",+ )+ normalized = normalized_answer(answer)+ aliases = {normalized_answer(alias) for alias in pending.accepted_answers}+ if normalized in aliases:+ return "correct", "Accepted source-specific equivalent terminology."+ if pending.question_type == "choice":+ from deeptutor.learning.pending import is_readable_choice_answer, resolve_choice_submission++ if not is_readable_choice_answer(answer, pending.choice_map):+ return "ungraded", "Clarify the selected answer before assessing it."+ label = resolve_choice_submission(answer, pending.choice_map)+ if label:+ return (+ "correct" if label == pending.expected_answer else "incorrect"+ ), "Compare the chosen answer with the original source evidence."+ return (+ "ungraded",+ "This wording is not a verified equivalent. Ask for clarification or repair the reference; do not infer a wrong answer from surface similarity.",+ )diff --git a/deeptutor/runtime/agentic/messages.py b/deeptutor/runtime/agentic/messages.pyindex 649a38f6b..792bb909b 100644--- a/deeptutor/runtime/agentic/messages.py+++ b/deeptutor/runtime/agentic/messages.py@@ -5,6 +5,42 @@ from __future__ import annotationsfrom typing import Any+def image_content_hashes(messages: list[dict]) -> list[str]:+ """Hashes of bounded image bytes on the accepted provider wire (#1611)."""+ from base64 import b64decode+ from hashlib import sha256++ hashes = set()+ for message in messages:+ content = message.get("content")+ for part in content if isinstance(content, list) else []:+ if not isinstance(part, dict):+ continue+ value = part.get("image_url")+ url = value.get("url") if isinstance(value, dict) else value+ source = part.get("source")+ if (+ part.get("type") == "image"+ and isinstance(source, dict)+ and source.get("type") == "base64"+ ):+ url = f"data:{source.get('media_type', '')};base64,{source.get('data', '')}"+ if (+ not isinstance(url, str)+ or not url.startswith("data:image/")+ or ";base64," not in url+ ):+ continue+ encoded = url.split(";base64,", 1)[1]+ if len(encoded) > 7 * 1024 * 1024:+ continue+ try:+ hashes.add(sha256(b64decode(encoded, validate=True)).hexdigest())+ except ValueError:+ continue+ return sorted(hashes)++def assistant_message_with_tool_calls(content: str,tool_calls: list[dict[str, Any]],diff --git a/deeptutor/runtime/bootstrap/builtin_capabilities.py b/deeptutor/runtime/bootstrap/builtin_capabilities.pyindex 11a4ec989..cf9552e38 100644--- a/deeptutor/runtime/bootstrap/builtin_capabilities.py+++ b/deeptutor/runtime/bootstrap/builtin_capabilities.py@@ -166,6 +166,7 @@ BUILTIN_CAPABILITY_SPECS: dict[str, BuiltinCapabilitySpec] = {tools_used=["mastery_status","mastery_quiz",+ "mastery_note_explained","mastery_grade","mastery_skip_question","mastery_repair_question",diff --git a/deeptutor/services/rag/source_visuals.py b/deeptutor/services/rag/source_visuals.pyindex a276e435b..f742d6311 100644--- a/deeptutor/services/rag/source_visuals.py+++ b/deeptutor/services/rag/source_visuals.py@@ -128,6 +128,7 @@ def page_image("Page image exceeds the model byte budget. Select a smaller region.")text = sheet.get_text("text", clip=clip)[:6000]+ page_context = sheet.get_text("text")[:6000]except ValueError:raiseexcept Exception as exc:@@ -143,6 +144,7 @@ def page_image("region": region,"mime_type": mime,"text": text,+ "page_context": page_context,"render_scale": scale,"kind": "source_page","source_locator": f"page:{page}",diff --git a/deeptutor/tools/builtin_specs.py b/deeptutor/tools/builtin_specs.pyindex 6f32ec015..3b9223e9c 100644--- a/deeptutor/tools/builtin_specs.py+++ b/deeptutor/tools/builtin_specs.py@@ -105,6 +105,7 @@ BUILTIN_TOOL_SPECS: tuple[BuiltinToolSpec, ...] = ((("mastery_status", "MasteryStatusTool"),("mastery_quiz", "MasteryQuizTool"),+ ("mastery_note_explained", "MasteryNoteExplainedTool"),("mastery_grade", "MasteryGradeTool"),("mastery_skip_question", "MasterySkipQuestionTool"),("mastery_repair_question", "MasteryRepairQuestionTool"),diff --git a/deeptutor/tools/mastery_tool.py b/deeptutor/tools/mastery_tool.pyindex 17d51fc51..412c6102b 100644--- a/deeptutor/tools/mastery_tool.py+++ b/deeptutor/tools/mastery_tool.py@@ -15,6 +15,7 @@ from deeptutor.capabilities.mastery.tools import (MasteryGradeTool,MasteryLeaveTool,MasteryModeTool,+ MasteryNoteExplainedTool,MasteryPathsTool,MasteryProfileTool,MasteryQuizTool,@@ -34,6 +35,7 @@ __all__ = ["MasteryGradeTool","MasteryLeaveTool","MasteryModeTool",+ "MasteryNoteExplainedTool","MasteryPathsTool","MasteryProfileTool","MasteryQuizTool",diff --git a/docs-for-user/SOURCE_VISUALS.md b/docs-for-user/SOURCE_VISUALS.mdindex b37e8f52b..63e2d9bff 100644--- a/docs-for-user/SOURCE_VISUALS.md+++ b/docs-for-user/SOURCE_VISUALS.md@@ -79,6 +79,61 @@ unavailable pixels and changed-source abstentions are checked independently oflanguage-model fluency.+## Source-grounded practice and learning evidence++In Guided Learning, request explanation, practice or review explicitly. An+explanation-only turn need not pose a question; `mastery_note_explained` records+what was taught without granting mastery. Deferring an objective preserves it+as unfinished. For goals that require visual understanding, outline objectives+can declare `required_visual_tasks`: `identification`, `relationship`,+`table_graph` or `comparison`. Generic text scores cannot satisfy those visual+requirements. The objective view shows the remaining independent evidence.++Ask for practice on an exact original figure or page from an attached KB. The+tutor retrieves its pixels, then poses a `mastery_quiz` with a `visual` context.+The question card displays the original evidence and document/page links.+Numbered-label identification, source-backed relationship questions, table or+graph interpretation and bounded two-source comparisons are supported. No+automatic segmentation, invented label masks or generated replacement diagram+is used. A source image that fails to load disables submission while still+allowing the learner to skip or ask for help.++The reference key uses canonical source terminology and a supporting caption,+explanation or page quote. Explicitly verified equivalent terms, including+other languages, can answer the same question. Choice questions distinguish+supported wrong answers; unknown free-text wording asks for clarification+instead of fuzzy keyword grading. Uncertain/conflicting keys, inaccessible or+changed sources, and missing model-input pixels remain ungraded and create no+negative mastery or retention evidence. A figure URL or a declared vision+capability is insufficient: the active loop checks matching image bytes in the+accepted provider request, including actual text-only fallback.++Visible/unknown answer cues are guided practice, with assistance retained and+weaker review evidence. They do not demonstrate independent mastery. A full+page that exposes the reference answer stays guided; a verified original-page+region can support independent interpretation when the answer cue is outside+the view, with the full source retained for reference verification. Recent+assistance, repeated answer-key exposure and help given after a question also+remain distinguishable through the existing retention session boundary.++Use **Review or challenge this assessment** to request source review. Existing+`mastery_repair_question` invalidates a faulty assessment and recomputes its+mastery/error/review effects. Correcting a visual key also requires source+support; a challenge alone never grants mastery. Manual learner overrides retain+their separate provenance. Source/figure identity, objective, task, cues/help,+answer and attempt identity survive restart. A transport retry records one+attempt; a new question records new evidence. Outline replacement preserves+trusted objective identity rather than transferring an achievement by title or+position. Unassessed, explained, assisted and independently demonstrated work+remain distinct.++Regression workflows use independently specified synthetic numbered figures,+relationships and table values, including equivalent-language answers, wrong+choices, ungraded ambiguity, repair, deferral, repeat submissions, outline+replacement, source changes and restart. This supported set provides explicit+recovery for uncertainty rather than promising unrestricted grading of every+image or arbitrary prose answer.+## Token estimates for image requestsWhen the provider returns usage, DeepTutor uses those reported counters. Whendiff --git a/tests/agents/chat/test_agent_loop.py b/tests/agents/chat/test_agent_loop.pyindex d55096d88..bdd1840e4 100644--- a/tests/agents/chat/test_agent_loop.py+++ b/tests/agents/chat/test_agent_loop.py@@ -305,6 +305,9 @@ async def test_active_mastery_retrieves_verified_pixels_without_persisting_them(]if evidence == "available":assert len(images) == 1+ assert context.extension("source_visual_evidence")["image_hashes"] == [+ candidate.record["image_sha256"]+ ]assert (base64.b64decode(images[0]["image_url"]["url"].split(",", 1)[1]) == image.read_bytes())@@ -318,6 +321,7 @@ async def test_active_mastery_retrieves_verified_pixels_without_persisting_them(assert candidate.record["asset_id"] in str(request[image_position])else:assert images == []+ assert context.extension("source_visual_evidence")["image_hashes"] == []tool_text = next(msg["content"] for msg in request if msg.get("tool_call_id") == "figure")assert ("cannot inspect source image pixels"@@ -3824,4 +3828,5 @@ async def test_image_fallback_strips_all_canonical_copies_after_wire_deduplicatiassert not has_image_parts(attempted[1])assert not has_image_parts(attempted[2])assert not has_image_parts(messages)+ assert context.extension("source_visual_evidence")["image_hashes"] == []assert "Repeated image" not in str(messages)diff --git a/web/components/chat/home/MasteryQuestionCard.tsx b/web/components/chat/home/MasteryQuestionCard.tsxindex e1c9678cb..a100da28a 100644--- a/web/components/chat/home/MasteryQuestionCard.tsx+++ b/web/components/chat/home/MasteryQuestionCard.tsx@@ -100,6 +100,7 @@ export const MasteryQuestionCard = memo(function MasteryQuestionCard({submittedAnswer,onSubmit,onSkip,+ onChallenge,}: {question: MasteryQuestion;/** The gate's ruling, once it has been made. */@@ -117,10 +118,12 @@ export const MasteryQuestionCard = memo(function MasteryQuestionCard({/** Drop this question and let the tutor move on. Absent where the surface* cannot start a turn of its own, which hides the control entirely. */onSkip?: (questionId: string) => void | boolean | Promise<void | boolean>;+ onChallenge?: (questionId: string) => void | boolean | Promise<void | boolean>;}) {const { t } = useTranslation();const [picked, setPicked] = useState<string>("");const [skipping, setSkipping] = useState(false);+ const [sourceUnavailable, setSourceUnavailable] = useState(false);const [freeText, setFreeText] = useState("");const [freeSelected, setFreeSelected] = useState(false);const { sending, failed: sendFailed, submit } = useCardSubmission(onSubmit);@@ -136,6 +139,7 @@ export const MasteryQuestionCard = memo(function MasteryQuestionCard({// Skipping settles the card exactly as answering does: the engine closed the// question, so there is nothing left on it to send.const settled = answered || skipped === true;+ const ungraded = grade?.result === "ungraded";const locked = settled || sending || skipping;const handleSkip = useCallback(() => {@@ -148,12 +152,12 @@ export const MasteryQuestionCard = memo(function MasteryQuestionCard({}, [locked, onSkip, question.questionId]);const handleSubmit = useCallback(() => {- if (locked || !answer) return;+ if (locked || !answer || sourceUnavailable) return;void submit({text: answer,answers: [{ questionId: question.questionId, text: answer }],});- }, [answer, locked, submit, question.questionId]);+ }, [answer, locked, sourceUnavailable, submit, question.questionId]);// Which answer to treat as the learner's, preferring what the graded record// committed over what this browser happens to remember.@@ -168,7 +172,7 @@ export const MasteryQuestionCard = memo(function MasteryQuestionCard({].filter(Boolean);// The rule down the left margin is the whole status indicator.- const rule = grade+ const rule = ungraded ? "border-dashed border-[var(--border)]" : grade? grade.isCorrect? "border-[var(--primary)]": "border-[var(--destructive)]"@@ -180,6 +184,7 @@ export const MasteryQuestionCard = memo(function MasteryQuestionCard({const optionState = (option: { label: string }): OptionState => {if (grade) {+ if (ungraded) return "idle";if (option.label && option.label === grade.correctLabel) return "correct";if (option.label && option.label === learnerAnswer && !grade.isCorrect)return "wrong";@@ -200,6 +205,17 @@ export const MasteryQuestionCard = memo(function MasteryQuestionCard({<div className="mt-0.5 font-serif text-[15.5px] font-semibold leading-relaxed tracking-[-0.01em] text-[var(--foreground)]"><InlineMarkdown content={question.prompt} /></div>+ {question.visual && <div className="mt-3 space-y-2 text-xs text-[var(--muted-foreground)]">+ <p>{grade?.independent === false || question.visual.hintsUsed || question.visual.answerCues !== "none" ? t("Guided visual practice — this does not demonstrate independent mastery.") : t("Visual practice with the original source.")}</p>+ {sourceUnavailable && <p role="alert">{t("The source image is unavailable. Open the source or ask your tutor to inspect its current version.")}</p>}+ {question.visual.keyStatus !== "verified" && <p>{t("The reference key is uncertain. This practice will remain ungraded.")}</p>}+ {question.visual.pixelsInspected === false && <p>{t("Current model input has no verified source pixels. Retrieve the image again with a vision-capable model before grading.")}</p>}+ {question.visual.sources.map((source, index) => <figure key={`${source.imageUrl}-${index}`}>+ {/* Authenticated source endpoints need the browser's own cookies. */}+ <a href={source.imageUrl} target="_blank" rel="noreferrer"><img src={source.imageUrl} alt={t("Original source evidence")} onError={() => setSourceUnavailable(true)} className="max-h-80 max-w-full object-contain" /></a>+ <figcaption><a className="underline" href={source.url} target="_blank" rel="noreferrer">{source.sourcePath}{source.page ? ` · ${t("Page")} ${source.page}` : ""}</a></figcaption>+ </figure>)}+ </div>}{hasChoices ? (<div className="mt-2.5 flex flex-col gap-0.5">@@ -254,14 +270,14 @@ export const MasteryQuestionCard = memo(function MasteryQuestionCard({<spanclassName={"font-semibold " +- (grade.isCorrect+ (ungraded ? "text-[var(--muted-foreground)]" : grade.isCorrect? "text-[var(--primary)]": "text-[var(--destructive)]")}>- {grade.isCorrect ? t("Correct") : t("Not quite")}+ {ungraded ? t("Ungraded — mastery unchanged") : grade.isCorrect ? t("Correct") : t("Not quite")}</span>- {!grade.isCorrect && grade.correctLabel ? (+ {!ungraded && !grade.isCorrect && grade.correctLabel ? (<span className="text-[var(--foreground)]">{" · "}{t("Answer: {{label}}", { label: grade.correctLabel })}@@ -273,6 +289,7 @@ export const MasteryQuestionCard = memo(function MasteryQuestionCard({<InlineMarkdown content={grade.explanation} /></div>) : null}+ {onChallenge && <button type="button" className="mt-2 text-xs underline" onClick={() => onChallenge(question.questionId)}>{t("Review or challenge this assessment")}</button>}</div>) : null}@@ -312,7 +329,7 @@ export const MasteryQuestionCard = memo(function MasteryQuestionCard({<buttontype="button"onClick={handleSubmit}- disabled={locked || !answer}+ disabled={locked || !answer || sourceUnavailable}className="rounded-md bg-[var(--primary)] px-3 py-1.5 text-[12px] font-medium text-[var(--primary-foreground)] transition-opacity hover:opacity-90 disabled:cursor-not-allowed disabled:opacity-40">{t("Submit")}diff --git a/web/components/space/learning/MasteryStudy.tsx b/web/components/space/learning/MasteryStudy.tsxindex 555f13395..ffdb96c57 100644--- a/web/components/space/learning/MasteryStudy.tsx+++ b/web/components/space/learning/MasteryStudy.tsx@@ -809,6 +809,11 @@ export function MasteryStudy({onSubmitUserReply={submitUserReply}onAnswerMasteryQuestion={answerMasteryQuestion}onSkipMasteryQuestion={skipMasteryQuestion}+ onChallengeMasteryQuestion={(questionId) => {+ if (state.isStreaming || sessionLoading || sessionError) return false;+ sendMessage(t('Please review assessment {{questionId}} against its original source evidence. If it is unsupported, invalidate it and recompute learning state. Do not grant mastery merely because I challenged it.', { questionId }));+ return true;+ }}onPreviewAttachment={handlePreviewMessageAttachment}onConfirmOutline={confirmResearchOutline}onLoadMessageTrace={(messageId) =>diff --git a/web/components/space/learning/ObjectiveDetail.tsx b/web/components/space/learning/ObjectiveDetail.tsxindex c4f14360d..4d5b7cdb2 100644--- a/web/components/space/learning/ObjectiveDetail.tsx+++ b/web/components/space/learning/ObjectiveDetail.tsx@@ -115,6 +115,10 @@ export function ObjectiveDetail({</Row>)}+ {report.explained && <Row label={t("Explained")}>{report.explained.summary}</Row>}+ {!!report.required_visual_tasks?.length && <Row label={t("Independent visual evidence")}>{report.demonstrated_visual_tasks?.length ?? 0} / {report.required_visual_tasks.length}</Row>}+ {!!report.required_visual_tasks?.some((task) => !report.demonstrated_visual_tasks?.includes(task)) && <p className="text-xs text-[var(--muted-foreground)]">{t("Independent visual evidence is still required for this objective.")}</p>}+ {report.attempts.some((attempt) => attempt.independent === false) && <p className="text-xs text-[var(--muted-foreground)]">{t("Assisted practice is recorded separately from independent mastery.")}</p>}{evidence.length > 0 && (<div><div className="text-xs text-[var(--muted-foreground)]">@@ -130,6 +134,7 @@ export function ObjectiveDetail({><span>{t(evidence.result)} · {t(evidence.assessment_type)}+ {evidence.visual_context?.task && ` · ${t("Visual practice")}${evidence.hints_used ? ` · ${t("Assisted")}` : ""}`}{evidence.quality !== null &&` · ${Math.round(evidence.quality * 100)}%`}</span>diff --git a/web/features/chat/components/ChatWorkspace.tsx b/web/features/chat/components/ChatWorkspace.tsxindex b63bf33d8..9cae05f61 100644--- a/web/features/chat/components/ChatWorkspace.tsx+++ b/web/features/chat/components/ChatWorkspace.tsx@@ -2585,6 +2585,7 @@ export default function ChatWorkspace() {onSubmitUserReply={submitUserReply}onAnswerMasteryQuestion={answerMasteryQuestion}onSkipMasteryQuestion={skipMasteryQuestion}+ onChallengeMasteryQuestion={(questionId) => handleSend(t('Please review assessment {{questionId}} against its original source evidence. If it is unsupported, invalidate it and recompute learning state. Do not grant mastery merely because I challenged it.', { questionId }))}onLoadMessageTrace={(messageId) =>state.sessionId? loadMessageTrace(state.sessionId, messageId)diff --git a/web/features/chat/messages/ChatMessageList.tsx b/web/features/chat/messages/ChatMessageList.tsxindex 4e4454304..88150b741 100644--- a/web/features/chat/messages/ChatMessageList.tsx+++ b/web/features/chat/messages/ChatMessageList.tsx@@ -643,6 +643,7 @@ export const AssistantMessage = memo(function AssistantMessage({onSubmitUserReply,onAnswerMasteryQuestion,onSkipMasteryQuestion,+ onChallengeMasteryQuestion,researchRequestSnapshot,onTraceToggle,masteryGrades,@@ -710,6 +711,7 @@ export const AssistantMessage = memo(function AssistantMessage({onSkipMasteryQuestion?: (questionId: string,) => void | boolean | Promise<void | boolean>;+ onChallengeMasteryQuestion?: (questionId: string) => void | boolean | Promise<void | boolean>;}) {const { t } = useTranslation();const events = useMemo(() => msg.events ?? [], [msg.events]);@@ -819,6 +821,7 @@ export const AssistantMessage = memo(function AssistantMessage({: false}onSkip={onSkipMasteryQuestion}+ onChallenge={onChallengeMasteryQuestion}/>);},@@ -827,6 +830,7 @@ export const AssistantMessage = memo(function AssistantMessage({masterySkips,onAnswerMasteryQuestion,onSkipMasteryQuestion,+ onChallengeMasteryQuestion,],);// Set by ``request_credential`` when a configuration step needs a secret the@@ -1923,6 +1927,7 @@ export const ChatMessageList = memo(function ChatMessageList({onSubmitUserReply,onAnswerMasteryQuestion,onSkipMasteryQuestion,+ onChallengeMasteryQuestion,showModeBadge = true,onLoadMessageTrace,onReleaseMessageTrace,@@ -1979,6 +1984,7 @@ export const ChatMessageList = memo(function ChatMessageList({onSkipMasteryQuestion?: (questionId: string,) => void | boolean | Promise<void | boolean>;+ onChallengeMasteryQuestion?: (questionId: string) => void | boolean | Promise<void | boolean>;/** Names of KBs confirmed to exist. Omitted when the KB list is unavailable. */availableKbNames?: Set<string>;/** Qualified KB ref -> display name, from the same catalog the composer@@ -2314,6 +2320,7 @@ export const ChatMessageList = memo(function ChatMessageList({onSubmitUserReply={onSubmitUserReply}onAnswerMasteryQuestion={onAnswerMasteryQuestion}onSkipMasteryQuestion={onSkipMasteryQuestion}+ onChallengeMasteryQuestion={onChallengeMasteryQuestion}researchRequestSnapshot={pairedUserMessage?.requestSnapshot ?? null}diff --git a/web/lib/learning-api.ts b/web/lib/learning-api.tsindex 354b7d353..c0b70e8bf 100644--- a/web/lib/learning-api.ts+++ b/web/lib/learning-api.ts@@ -23,6 +23,7 @@ export interface LearningKnowledgePoint {id: string;name: string;type: string;+ required_visual_tasks?: string[];}export interface LearningModule {@@ -233,6 +234,7 @@ export async function fetchProgressEvents(// Mirrors deeptutor/learning/policy.py objective_report.export interface ObjectiveAttempt {+ independent?: boolean;question_id: string;prompt: string;answer: string;@@ -268,6 +270,7 @@ export interface LearningEvidence {response_time: number | null;session_id: string;turn_id: string;+ visual_context?: { task?: string; sources?: { image_url: string; url: string; source_path: string }[]; hints_used?: number };}export interface ObjectiveErrorRecord {@@ -288,6 +291,9 @@ export interface ObjectiveReport {gate: "quantitative" | "qualitative";mastered: boolean;assessed_mastered: boolean;+ explained?: { summary: string; timestamp: number };+ required_visual_tasks?: string[];+ demonstrated_visual_tasks?: string[];mastery_source: "system" | "learner" | "";override_note: string;mastery: number;diff --git a/web/lib/mastery-question.ts b/web/lib/mastery-question.tsindex 1b5c1432d..48a2a53c4 100644--- a/web/lib/mastery-question.ts+++ b/web/lib/mastery-question.ts@@ -44,6 +44,7 @@ export interface MasteryQuestion {attempt: number;options: MasteryQuestionOption[];allowFreeText: boolean;+ visual?: { task: string; answerCues: string; keyStatus: string; hintsUsed: number; pixelsInspected?: boolean; sources: { imageUrl: string; url: string; sourcePath: string; page: number | null }[] };}export interface MasteryGradeResult {@@ -53,6 +54,8 @@ export interface MasteryGradeResult {correctLabel: string;correctBody: string;explanation: string;+ result?: string;+ independent?: boolean;}function asRecord(value: unknown): Record<string, unknown> | null {@@ -82,6 +85,10 @@ function parseQuestion(value: unknown): MasteryQuestion | null {.filter((option) => option.label): [];const attempt = Number(raw.attempt);+ const visual = asRecord(raw.visual);+ const visualSources = Array.isArray(visual?.sources) ? visual.sources.map(asRecord).filter((ref): ref is Record<string, unknown> => ref !== null)+ .filter((ref) => text(ref.image_url).startsWith("/api/knowledge-bases/") && text(ref.url).startsWith("/api/knowledge-bases/"))+ .map((ref) => ({ imageUrl: text(ref.image_url), url: text(ref.url), sourcePath: text(ref.source_path), page: typeof ref.page === "number" ? ref.page : null })) : [];return {questionId,prompt: text(raw.prompt),@@ -91,6 +98,7 @@ function parseQuestion(value: unknown): MasteryQuestion | null {attempt: Number.isFinite(attempt) && attempt > 0 ? Math.floor(attempt) : 1,options,allowFreeText: raw.allow_free_text !== false,+ ...(visual ? { visual: { task: text(visual.task), answerCues: text(visual.answer_cues), keyStatus: text(visual.key_status), hintsUsed: Number(visual.hints_used) || 0, pixelsInspected: visual.pixels_inspected === true, sources: visualSources } } : {}),};}@@ -106,6 +114,8 @@ function parseGrade(value: unknown): MasteryGradeResult | null {correctLabel: text(raw.correct_label),correctBody: text(raw.correct_body),explanation: text(raw.explanation),+ ...(text(raw.result) ? { result: text(raw.result) } : {}),+ ...(typeof raw.independent === "boolean" ? { independent: raw.independent } : {}),};}diff --git a/web/locales/de/app.json b/web/locales/de/app.jsonindex 51387ef5a..410c675b7 100644--- a/web/locales/de/app.json+++ b/web/locales/de/app.json@@ -5472,5 +5472,20 @@"visualCoverage.asset_read_failed": "Abbildung konnte nicht gelesen werden; prüfe die Originalseite","visualCoverage.table_structure_requires_source_verification": "Tabellenstruktur muss anhand der Originalseite geprüft werden","visualCoverage.referenced_image_not_retained": "Referenzierte Abbildung wurde nicht gespeichert; prüfe die Quellseite",- "visualCoverage.extraction_coverage_unknown": "Extraktionsabdeckung ist noch ungeprüft"+ "visualCoverage.extraction_coverage_unknown": "Extraktionsabdeckung ist noch ungeprüft",+ "Guided visual practice — this does not demonstrate independent mastery.": "Angeleitete visuelle Übung – kein Nachweis eigenständiger Beherrschung.",+ "Visual practice with the original source.": "Visuelle Übung mit der Originalquelle.",+ "The reference key is uncertain. This practice will remain ungraded.": "Der Lösungsschlüssel ist unsicher. Diese Übung bleibt unbewertet.",+ "Original source evidence": "Originaler Quellenbeleg",+ "Ungraded — mastery unchanged": "Unbewertet – Lernstand unverändert",+ "Review or challenge this assessment": "Bewertung prüfen oder anfechten",+ "The source image is unavailable. Open the source or ask your tutor to inspect its current version.": "Die Quellabbildung ist nicht verfügbar. Öffne die Quelle oder bitte deinen Tutor, die aktuelle Version zu prüfen.",+ "Independent visual evidence": "Eigenständiger visueller Nachweis",+ "Explained": "Erklärt",+ "Visual practice": "Visuelle Übung",+ "Assisted": "Mit Hilfe",+ "Independent visual evidence is still required for this objective.": "Für dieses Lernziel ist noch ein eigenständiger visueller Nachweis erforderlich.",+ "Assisted practice is recorded separately from independent mastery.": "Angeleitete Übung wird getrennt von eigenständiger Beherrschung erfasst.",+ "Please review assessment {{questionId}} against its original source evidence. If it is unsupported, invalidate it and recompute learning state. Do not grant mastery merely because I challenged it.": "Bitte prüfe Bewertung {{questionId}} anhand der Originalquelle. Wenn sie unbelegt ist, annulliere sie und berechne den Lernstand neu. Vergib keinen Beherrschungsnachweis allein wegen meines Einspruchs.",+ "Current model input has no verified source pixels. Retrieve the image again with a vision-capable model before grading.": "In der aktuellen Modelleingabe sind keine verifizierten Quellbilder enthalten. Lade das Bild vor der Bewertung mit einem visuellen Modell erneut."}diff --git a/web/locales/en/app.json b/web/locales/en/app.jsonindex fd7cc4cb6..869989547 100644--- a/web/locales/en/app.json+++ b/web/locales/en/app.json@@ -5472,5 +5472,20 @@"visualCoverage.asset_read_failed": "Image read failed; inspect the original page","visualCoverage.table_structure_requires_source_verification": "Verify table structure against the original page","visualCoverage.referenced_image_not_retained": "Referenced image not retained; inspect its source page",- "visualCoverage.extraction_coverage_unknown": "Extraction coverage has not been verified"+ "visualCoverage.extraction_coverage_unknown": "Extraction coverage has not been verified",+ "Guided visual practice — this does not demonstrate independent mastery.": "Guided visual practice — this does not demonstrate independent mastery.",+ "Visual practice with the original source.": "Visual practice with the original source.",+ "The reference key is uncertain. This practice will remain ungraded.": "The reference key is uncertain. This practice will remain ungraded.",+ "Original source evidence": "Original source evidence",+ "Ungraded — mastery unchanged": "Ungraded — mastery unchanged",+ "Review or challenge this assessment": "Review or challenge this assessment",+ "The source image is unavailable. Open the source or ask your tutor to inspect its current version.": "The source image is unavailable. Open the source or ask your tutor to inspect its current version.",+ "Independent visual evidence": "Independent visual evidence",+ "Explained": "Explained",+ "Visual practice": "Visual practice",+ "Assisted": "Assisted",+ "Independent visual evidence is still required for this objective.": "Independent visual evidence is still required for this objective.",+ "Assisted practice is recorded separately from independent mastery.": "Assisted practice is recorded separately from independent mastery.",+ "Please review assessment {{questionId}} against its original source evidence. If it is unsupported, invalidate it and recompute learning state. Do not grant mastery merely because I challenged it.": "Please review assessment {{questionId}} against its original source evidence. If it is unsupported, invalidate it and recompute learning state. Do not grant mastery merely because I challenged it.",+ "Current model input has no verified source pixels. Retrieve the image again with a vision-capable model before grading.": "Current model input has no verified source pixels. Retrieve the image again with a vision-capable model before grading."}diff --git a/web/locales/pl/app.json b/web/locales/pl/app.jsonindex d8001a5ef..f89767b9f 100644--- a/web/locales/pl/app.json+++ b/web/locales/pl/app.json@@ -5488,5 +5488,20 @@"visualCoverage.asset_read_failed": "Nie udało się odczytać obrazu; sprawdź oryginalną stronę","visualCoverage.table_structure_requires_source_verification": "Struktura tabeli wymaga sprawdzenia na oryginalnej stronie","visualCoverage.referenced_image_not_retained": "Nie zachowano wskazanego obrazu; sprawdź stronę źródłową",- "visualCoverage.extraction_coverage_unknown": "Pokrycie wyodrębniania nie zostało zweryfikowane"+ "visualCoverage.extraction_coverage_unknown": "Pokrycie wyodrębniania nie zostało zweryfikowane",+ "Guided visual practice — this does not demonstrate independent mastery.": "Ćwiczenie wizualne ze wsparciem — nie dowodzi samodzielnego opanowania.",+ "Visual practice with the original source.": "Ćwiczenie wizualne z oryginalnym źródłem.",+ "The reference key is uncertain. This practice will remain ungraded.": "Klucz odpowiedzi jest niepewny. Ćwiczenie pozostanie bez oceny.",+ "Original source evidence": "Oryginalny materiał źródłowy",+ "Ungraded — mastery unchanged": "Bez oceny — poziom opanowania bez zmian",+ "Review or challenge this assessment": "Sprawdź lub zakwestionuj tę ocenę",+ "The source image is unavailable. Open the source or ask your tutor to inspect its current version.": "Obraz źródłowy jest niedostępny. Otwórz źródło lub poproś tutora o sprawdzenie bieżącej wersji.",+ "Independent visual evidence": "Samodzielny dowód rozumienia obrazu",+ "Explained": "Wyjaśnione",+ "Visual practice": "Ćwiczenie wizualne",+ "Assisted": "Ze wsparciem",+ "Independent visual evidence is still required for this objective.": "Ten cel nadal wymaga samodzielnego dowodu rozumienia obrazu.",+ "Assisted practice is recorded separately from independent mastery.": "Ćwiczenia ze wsparciem są zapisywane oddzielnie od samodzielnego opanowania.",+ "Please review assessment {{questionId}} against its original source evidence. If it is unsupported, invalidate it and recompute learning state. Do not grant mastery merely because I challenged it.": "Sprawdź ocenę {{questionId}} na podstawie oryginalnego źródła. Jeśli jest nieuzasadniona, unieważnij ją i przelicz stan nauki. Samo zakwestionowanie nie dowodzi opanowania.",+ "Current model input has no verified source pixels. Retrieve the image again with a vision-capable model before grading.": "Bieżące dane wejściowe modelu nie zawierają zweryfikowanego obrazu źródłowego. Przed oceną pobierz go ponownie modelem obsługującym obrazy."}diff --git a/web/locales/zh/app.json b/web/locales/zh/app.jsonindex ba624a7df..a0232ad34 100644--- a/web/locales/zh/app.json+++ b/web/locales/zh/app.json@@ -5472,5 +5472,20 @@"visualCoverage.asset_read_failed": "读取图像失败;请检查原始页面","visualCoverage.table_structure_requires_source_verification": "表格结构需要对照原始页面核验","visualCoverage.referenced_image_not_retained": "引用的图像未保留;请检查该来源页面",- "visualCoverage.extraction_coverage_unknown": "提取覆盖情况尚未核验"+ "visualCoverage.extraction_coverage_unknown": "提取覆盖情况尚未核验",+ "Guided visual practice — this does not demonstrate independent mastery.": "辅助视觉练习,不代表已独立掌握。",+ "Visual practice with the original source.": "使用原始来源的视觉练习。",+ "The reference key is uncertain. This practice will remain ungraded.": "参考答案尚不确定,本次练习将保留为未评分。",+ "Original source evidence": "原始来源证据",+ "Ungraded — mastery unchanged": "未评分,掌握状态不变",+ "Review or challenge this assessment": "复核或申诉本次判定",+ "The source image is unavailable. Open the source or ask your tutor to inspect its current version.": "来源图像不可用。请打开原始来源,或让导师检查当前版本。",+ "Independent visual evidence": "独立视觉证据",+ "Explained": "已讲解",+ "Visual practice": "视觉练习",+ "Assisted": "有辅助",+ "Independent visual evidence is still required for this objective.": "此目标仍需要独立视觉证据。",+ "Assisted practice is recorded separately from independent mastery.": "辅助练习与独立掌握分开记录。",+ "Please review assessment {{questionId}} against its original source evidence. If it is unsupported, invalidate it and recompute learning state. Do not grant mastery merely because I challenged it.": "请对照原始来源复核判定 {{questionId}}。如果没有可靠依据,请作废并重算学习状态;不能只因我提出申诉就授予掌握状态。",+ "Current model input has no verified source pixels. Retrieve the image again with a vision-capable model before grading.": "当前模型输入没有已核验的来源像素。请使用支持视觉的模型重新读取原图,再进行评分。"}diff --git a/web/tests/mastery-question-card.spec.tsx b/web/tests/mastery-question-card.spec.tsxindex 48a89c2dd..29390f68b 100644--- a/web/tests/mastery-question-card.spec.tsx+++ b/web/tests/mastery-question-card.spec.tsx@@ -1,4 +1,4 @@-import { render, screen } from "@testing-library/react";+import { fireEvent, render, screen } from "@testing-library/react";import userEvent from "@testing-library/user-event";import { describe, expect, it, vi } from "vitest";@@ -21,6 +21,23 @@ const question = (overrides: Partial<MasteryQuestion> = {}): MasteryQuestion =>});describe("MasteryQuestionCard", () => {+ it("shows original evidence and assisted meaning without exposing a reference key", () => {+ render(<MasteryQuestionCard question={question({ visual: { task: "identification", answerCues: "visible", keyStatus: "unverified", hintsUsed: 1, sources: [{ imageUrl: "/api/knowledge-bases/kb/visual-assets/known", url: "/api/knowledge-bases/kb/files/book.pdf#page=4", sourcePath: "book.pdf", page: 4 }] } })} grade={null} answered={false} submittedAnswer="" onSubmit={() => true} />);+ expect(screen.getByRole("img", { name: "Original source evidence" })).toHaveAttribute("src", "/api/knowledge-bases/kb/visual-assets/known");+ expect(screen.getByText("Guided visual practice — this does not demonstrate independent mastery.")).toBeInTheDocument();+ expect(screen.getByText("The reference key is uncertain. This practice will remain ungraded.")).toBeInTheDocument();+ fireEvent.error(screen.getByRole("img"));+ expect(screen.getByRole("alert")).toHaveTextContent("The source image is unavailable");+ });+ it("keeps uncertain grades neutral and sends a source-review challenge", async () => {+ const user = userEvent.setup();+ const onChallenge = vi.fn(() => true);+ render(<MasteryQuestionCard question={question()} grade={{ questionId: "q-free", isCorrect: false, result: "ungraded", learnerAnswer: "ambiguous", correctLabel: "", correctBody: "", explanation: "Source evidence needs clarification." }} answered submittedAnswer="" onSubmit={() => true} onChallenge={onChallenge} />);+ expect(screen.getByText("Ungraded — mastery unchanged")).toBeInTheDocument();+ expect(screen.queryByText("Not quite")).toBeNull();+ await user.click(screen.getByRole("button", { name: "Review or challenge this assessment" }));+ expect(onChallenge).toHaveBeenCalledWith("q-free");+ });it("enables submit from a free-text-only question", async () => {const user = userEvent.setup();const onSubmit = vi.fn(() => true);