Skip to content

DeepTutor: Lifelong Personalized Tutoring. https://deeptutor.info/.

feat(mastery): complete source-grounded visual practice and evidence (#1611)

Bingxi Zhao (Frank)2350ea2dba65
diff --git a/deeptutor/agents/loop/agent_loop.py b/deeptutor/agents/loop/agent_loop.py
index 1bd08b2a6..7f1a76821 100644
--- a/deeptutor/agents/loop/agent_loop.py
+++ b/deeptutor/agents/loop/agent_loop.py
@@ -49,6 +49,7 @@ from deeptutor.core.trace import build_trace_metadata, merge_trace_metadata, new
from deeptutor.runtime.agentic.messages import (
assistant_message_with_tool_calls,
extend_transient_model_messages,
+ image_content_hashes,
with_transient_model_messages,
)
from deeptutor.runtime.agentic.think_stream import InlineThinkFilter
@@ -298,6 +299,7 @@ class AgentLoop:
# #1611: retrieved pixels belong to this turn's requests, never to
# durable history. Both loop engines share the same ordering/budget.
self._transient_model_messages: list[dict[str, Any]] = []
+ context.extension("source_visual_evidence")["image_hashes"] = []
self._request_fingerprint = (context.runtime.previous_model_turn or {}).get(
"request_fingerprint"
)
@@ -1266,6 +1268,11 @@ class AgentLoop:
self._create_response_stream(kwargs, trace_meta, stage),
_STREAM_IDLE_TIMEOUT_SECONDS,
)
+ # A declared vision capability is not evidence of delivered
+ # pixels when the provider actually falls back to text.
+ self.context.extension("source_visual_evidence")["image_hashes"] = (
+ image_content_hashes(kwargs["messages"])
+ )
# Retain an actual provider fallback for later rounds. The
# request-only image deduplication must never overwrite history.
if sent_had_images and not has_image_parts(kwargs["messages"]):
diff --git a/deeptutor/capabilities/mastery/loop.py b/deeptutor/capabilities/mastery/loop.py
index 65f45a7e6..cfc870c0f 100644
--- a/deeptutor/capabilities/mastery/loop.py
+++ b/deeptutor/capabilities/mastery/loop.py
@@ -266,6 +266,10 @@ class MasteryLoopCapability:
state["quiz_awaiting_grade"] = False
state["quiz_graded"] = True
updated["_mastery_path_id"] = path_id
+ updated["_attached_kb_names"] = list(context.knowledge_bases)
+ updated["_inspected_image_hashes"] = list(
+ context.extension("source_visual_evidence").get("image_hashes", [])
+ )
# Raw, not normalised: "this conversation never recorded a mode"
# has to survive down to the tools, or every pre-modes conversation
# (and every CLI / SDK turn, which pass none) would be enforced as
diff --git a/deeptutor/capabilities/mastery/mode.py b/deeptutor/capabilities/mastery/mode.py
index de1d07e79..60a99ac5f 100644
--- a/deeptutor/capabilities/mastery/mode.py
+++ b/deeptutor/capabilities/mastery/mode.py
@@ -77,6 +77,7 @@ TOOL_MODES: dict[str, frozenset[str]] = {
"mastery_skip_question": frozenset({STUDY, REVIEW}),
"mastery_repair_question": frozenset({STUDY, REVIEW}),
"mastery_defer_objective": frozenset({STUDY, REVIEW}),
+ "mastery_note_explained": frozenset({STUDY, REVIEW}),
}
diff --git a/deeptutor/capabilities/mastery/prompts/en/mastery_loop.yaml b/deeptutor/capabilities/mastery/prompts/en/mastery_loop.yaml
index bc3fa186e..519021a8d 100644
--- a/deeptutor/capabilities/mastery/prompts/en/mastery_loop.yaml
+++ b/deeptutor/capabilities/mastery/prompts/en/mastery_loop.yaml
@@ -86,6 +86,12 @@ session:
# Teaching strategy. Kept separate from the loop protocol above: that one says
# how a round runs, this one says what to do about the objective in front of you.
playbook: |-
+ **Source-grounded visual study:** Honor explanation, practice and review as explicit learner choices; explain untaught prerequisites at the requested level before assessment. Use `mastery_note_explained` after an explanation; this records coverage without mastery credit. An outline is never evidence of teaching. A concise explanation-only turn may end without a quiz; explicit deferral uses `mastery_defer_objective` and does not claim mastery.
+ When the goal includes figure understanding, declare the relevant objective's `required_visual_tasks` in the outline (`identification`, `relationship`, `table_graph`, `comparison`). Generic text scores do not satisfy these. Read `mastery_status.visual_requirements` and `learning_stages`; explain whether the next work addresses missing independent evidence, prior difficulty or due review.
+ Retrieve and inspect the original pixels with `rag` before a visual question. Pass `visual` to `mastery_quiz`, using exact source references and a supported reference quote, plus verified equivalent terminology (including the learner's language). Ask about the actual numbered label, visible relationship, plotted/table values or supported comparison. Preserve the original and source location; do not replace it with a generated diagram or invent a masked/segmented target. A key based only on uncertain pixel interpretation stays `unverified` and ungraded.
+ Describe visible answer cues honestly. A labeled image or direct table/page reading is guided practice; do not claim independent recall. Repeating recently assisted work is also weakened through the existing retention boundary. Unknown wording asks for clarification rather than fuzzy grading; use a well-grounded choice question when a supported wrong answer must be distinguished reliably. Explain mistakes against the original source and offer a different supported example, not an immediate loop of the same question.
+ The learner may challenge an assessment. Inspect its question ID, stored answer and source; use `mastery_repair_question` to invalidate or correct an unsupported assessment and recompute its effects. A challenge alone is not proof of mastery; a manual override has separate learner provenance. Changed/missing sources, ambiguity, unreadable labels and uncertain keys produce no negative evidence. Keep the source identity, objective, task and assistance together on every follow-up.
+
Once `mastery_status` has answered, act on the current objective:
- **`intake_needed` is true — ask before you design.** This learner has never told you who they are, and an outline built without that is the same route for a second-year undergraduate and for an engineer with six free evenings. Read the materials first, then ask, in one short message, only what you cannot read off the materials yourself: what they can already do, what finishing looks like for them, how much time they have, and how they want it taught. Ask through `ask_user` — this is the clarification it exists for — and shape the questions around what the material actually turned out to be, rather than reciting the same four every time. Take whatever they give you: a learner who answers two of four has still told you something, and pressing for the rest is worse than starting. Record their answers with `mastery_profile`, inventing nothing they did not say, then design the outline against them. If they decline to answer at all, say you will start from a reasonable guess and adjust as you go — then do exactly that.
diff --git a/deeptutor/capabilities/mastery/prompts/zh/mastery_loop.yaml b/deeptutor/capabilities/mastery/prompts/zh/mastery_loop.yaml
index b82895658..8ffcb6a95 100644
--- a/deeptutor/capabilities/mastery/prompts/zh/mastery_loop.yaml
+++ b/deeptutor/capabilities/mastery/prompts/zh/mastery_loop.yaml
@@ -83,6 +83,12 @@ session:
# 教学策略。与 loop 协议分开:上面说的是「一轮怎么走」,这里说的是「针对这个知识点做什么」。
playbook: |-
+ **来源扎实的视觉学习:** 尊重学习者明确选择的讲解、练习或复习,按要求的层次先解释必要前置知识。讲解后调用 `mastery_note_explained`,只记录已讲解覆盖,不授予掌握分;生成提纲不等于教学。仅需简洁讲解时可以不出题;明确暂缓用 `mastery_defer_objective`,不能假称已掌握。
+ 目标包含原图理解时,在对应提纲目标声明 `required_visual_tasks`(identification、relationship、table_graph、comparison)。通用文字题高分不能替代这些任务。查看 `mastery_status.visual_requirements` 和 `learning_stages`,解释下一步来自缺少独立证据、此前困难还是到期复习。
+ 视觉题先通过 `rag` 读取并检查原始像素,再给 `mastery_quiz` 传 `visual`:精确来源、来源中的参考引文以及已核验的等价术语(包含学习者语言)。练习应判断真实编号标签、可见关系、图表数据或有证据支持的比较。保留原图及位置,不能用生成图替代,也不能虚构遮罩或分割目标。仅凭不确定图像理解得到的答案键标为 unverified,保留未评分。
+ 如实记录可见答案线索:带答案标签或直接读取表格/页面是辅助练习,不是独立回忆;即时重复刚获帮助的题目也会按现有复习时间边界降权。未知表述应澄清而不是模糊打分;需要可靠区分实质性错误时使用来源明确的选择题。答错后对照原图讲解,提供其他有依据的例子,不要立即循环相同问题。
+ 学习者可申诉判定。核对题目 ID、保存的答案及来源后,用 `mastery_repair_question` 作废或纠正不受支持的判定,并重算学习状态。申诉本身不是掌握证明,手动覆盖有独立的学习者来源标记。来源变化、缺失、目标歧义、标签不可读或答案键不确定都不能形成负面掌握证据。追问中保持来源、目标、任务和辅助信息的关联。
+
拿到 `mastery_status` 之后,针对当前知识点行动:
- **`intake_needed` 为 true——先问,再设计。** 这位学习者还从没告诉过你他是谁,而不问就设计出来的大纲,对一个大二学生和一个只有六个晚上的工程师是同一条路线。先把材料读了,然后用一条简短的消息只问你自己看不出来的那几件事:他已经会什么、对他来说学到什么程度算完成、打算花多少时间、希望怎么教。用 `ask_user` 问——这正是它在这里的用途——并且要根据材料实际是什么来组织问题,而不是每次背同样四条。他给什么你就收什么:四个问题答了两个,他也已经告诉你东西了,为了凑齐而追问比直接开始更糟。用 `mastery_profile` 如实记下他说的,不要替他编造,然后照着这些来设计大纲。他要是完全不想答,就说明你会先按一个合理的估计开始、边走边调——然后真的这么做。
diff --git a/deeptutor/capabilities/mastery/tools.py b/deeptutor/capabilities/mastery/tools.py
index 26106d631..d988ccaf0 100644
--- a/deeptutor/capabilities/mastery/tools.py
+++ b/deeptutor/capabilities/mastery/tools.py
@@ -73,9 +73,11 @@ from deeptutor.learning.policy import (
gate_kind,
gate_threshold,
is_mastered,
+ learning_stage,
map_summary,
next_objective,
path_display_name,
+ visual_achievements,
)
from deeptutor.learning.question_card import (
QUESTION_CARD_KEY,
@@ -93,6 +95,7 @@ if TYPE_CHECKING:
MASTERY_TOOL_NAMES: tuple[str, ...] = (
"mastery_status",
"mastery_quiz",
+ "mastery_note_explained",
"mastery_grade",
"mastery_skip_question",
"mastery_repair_question",
@@ -392,6 +395,7 @@ async def _sync_mastery_attempt_to_question_bank(
confidence=confidence,
response_time=response_time,
quality=quality,
+ visual_context=pending.visual_context,
)
try:
await asyncio.wait_for(record_assessment(record), timeout=5.0)
@@ -770,6 +774,23 @@ class MasteryStatusTool(BaseTool):
"next": next_objective(progress).to_dict(),
"map": map_summary(progress),
**_profile_status(progress),
+ "learning_stages": {
+ kp.id: learning_stage(progress, kp.id)
+ for module in progress.modules
+ for kp in module.knowledge_points
+ }
+ if progress
+ else {},
+ "visual_requirements": {
+ kp.id: {
+ "required": kp.required_visual_tasks,
+ "demonstrated": visual_achievements(progress, kp.id),
+ }
+ for module in progress.modules
+ for kp in module.knowledge_points
+ }
+ if progress
+ else {},
}
interaction = service.store.get_active_interaction(path_id)
if interaction is not None:
@@ -802,6 +823,65 @@ class MasteryStatusTool(BaseTool):
return _json_result(payload, meta_key="mastery_status")
+class MasteryNoteExplainedTool(BaseTool):
+ """Record teaching without pretending that explanation is an assessment."""
+
+ def get_definition(self) -> ToolDefinition:
+ return ToolDefinition(
+ name="mastery_note_explained",
+ description="After explaining an objective at the requested level, record what was taught. This does not assess, grant mastery or schedule a review. Respect explanation-only requests; a quiz is optional until the learner wants practice.",
+ parameters=[
+ ToolParameter(
+ name="knowledge_point_id",
+ type="string",
+ description="Exact objective ID from mastery_status.",
+ ),
+ ToolParameter(
+ name="summary",
+ type="string",
+ description="What was actually explained, including prerequisites and source limitations.",
+ ),
+ ],
+ )
+
+ async def execute(self, **kwargs: Any) -> ToolResult:
+ refusal = _wrong_mode_result("mastery_note_explained", kwargs)
+ if refusal is not None:
+ return refusal
+ path_id = _resolve_path_id(kwargs)
+ if not path_id:
+ return _no_path_result()
+ kp_id = str(kwargs.get("knowledge_point_id") or "")
+ summary = str(kwargs.get("summary") or "").strip()[:2000]
+ if not summary:
+ return ToolResult(content="A teaching summary is required.", success=False)
+ service = _new_service()
+ if _load_path(service, path_id) is None:
+ return _no_built_path_result("mastery_note_explained")
+
+ def note(tx):
+ import time
+
+ if find_knowledge_point(tx.progress, kp_id)[0] is None:
+ raise ValueError("Unknown objective; refresh mastery_status.")
+ tx.progress.explained_objectives[kp_id] = {
+ "summary": summary,
+ "timestamp": time.time(),
+ "session_id": _resolve_session_id(kwargs),
+ "turn_id": _resolve_turn_id(kwargs),
+ }
+ tx.emit("objective.explained", {"knowledge_point_id": kp_id, "summary": summary})
+
+ try:
+ await asyncio.to_thread(service.store.mutate, path_id, note)
+ except (ValueError, FileNotFoundError) as exc:
+ return ToolResult(content=str(exc), success=False)
+ return _json_result(
+ {"knowledge_point_id": kp_id, "stage": "explained", "mastery_credit": False},
+ meta_key="mastery_note_explained",
+ )
+
+
class MasteryQuizTool(BaseTool):
"""Register an objective-type question; the engine holds the answer."""
@@ -821,6 +901,13 @@ class MasteryQuizTool(BaseTool):
"mastery_assess instead."
),
parameters=[
+ ToolParameter(
+ name="visual",
+ type="object",
+ required=False,
+ sensitive=True,
+ description="Optional source-grounded task: {task: identification|relationship|table_graph|comparison, sources:[{kb_name, asset_id OR source_path/page, source_hash?, region?}], reference_quote, accepted_answers: verified equivalent terms (including other languages), key_status: verified|unverified, answer_cues: none|visible|unverified, hints_used}. Retrieve and inspect original pixels first. reference_quote and expected_answer must use the source-grounded canonical terminology; accepted_answers may add verified translations for learner replies. Uncertain or conflicting keys stay ungraded. Visible or unknown cues mean guided practice, not independent mastery. Do not invent masks or regions.",
+ ),
ToolParameter(
name="knowledge_point_id",
type="string",
@@ -961,6 +1048,34 @@ class MasteryQuizTool(BaseTool):
content=f"Unknown objective {kp_id!r}; call mastery_status for valid ids.",
success=False,
)
+ visual_context: dict[str, Any] = {}
+ accepted_answers: list[str] = []
+ if kwargs.get("visual") is not None:
+ from deeptutor.learning.visual_practice import prepare_visual
+
+ if not isinstance(kwargs["visual"], dict):
+ return ToolResult(
+ content="visual must be an object with exact source references.", success=False
+ )
+ try:
+ visual_context, accepted_answers = await asyncio.to_thread(
+ prepare_visual,
+ kwargs["visual"],
+ expected_answer=expected,
+ options={option["label"]: option["body"] for option in options},
+ attached_kbs=kwargs.get("_attached_kb_names"),
+ inspected_image_hashes=kwargs.get("_inspected_image_hashes", []),
+ )
+ from deeptutor.learning.visual_practice import account_for_recent_assistance
+
+ visual_context = account_for_recent_assistance(
+ visual_context, progress.quiz_attempts, kp_id
+ )
+ except (OSError, ValueError, TypeError) as exc:
+ return ToolResult(
+ content=f"Visual practice is unavailable: {exc}. Inspect/clarify the source; no assessment was created.",
+ success=False,
+ )
pending = PendingQuestion(
question_id=uuid.uuid4().hex,
knowledge_point_id=kp_id,
@@ -971,6 +1086,8 @@ class MasteryQuizTool(BaseTool):
options=options,
explanation=str(kwargs.get("explanation") or "").strip()[:2000],
difficulty=_normalize_difficulty(kwargs.get("difficulty")),
+ visual_context=visual_context,
+ accepted_answers=accepted_answers,
)
from deeptutor.learning.service import MasteryInteractionError
@@ -1197,6 +1314,7 @@ class MasteryGradeTool(BaseTool):
item
for item in reversed(evidence_items)
if getattr(item, "knowledge_point_id", "") == pending.knowledge_point_id
+ and getattr(item, "question_id", "") == pending.question_id
),
None,
)
@@ -1220,6 +1338,7 @@ class MasteryGradeTool(BaseTool):
if evidence is not None
else None,
quality=getattr(evidence, "quality", None) if evidence is not None else None,
+ result=str(interaction.result.get("result") or ""),
)
mastered = bool(kp and is_mastered(progress, kp))
gate = gate_kind(kp) if kp else ""
@@ -1248,6 +1367,8 @@ class MasteryGradeTool(BaseTool):
"exposed, and when you pose the next question with "
"mastery_quiz put that call last, since it ends the turn."
)
+ if interaction.result.get("result") == "ungraded":
+ next_move = "going with source clarification or an explanation. This was ungraded, created no negative evidence and did not change mastery. Do not turn uncertainty into a wrong answer."
payload = {
"is_correct": is_correct,
"replayed": replayed,
@@ -1268,9 +1389,19 @@ class MasteryGradeTool(BaseTool):
question_id=pending.question_id,
is_correct=is_correct,
learner_answer=interaction.user_answer or answer,
- correct_label=expected_answer,
+ correct_label=expected_answer
+ if interaction.result.get("result") != "ungraded"
+ else "",
choice_options=choice_options,
- explanation=pending.explanation,
+ explanation=(
+ str(interaction.result.get("diagnosis"))
+ if interaction.result.get("result") == "ungraded"
+ else pending.explanation
+ ),
+ result=str(interaction.result.get("result") or ""),
+ independent=interaction.result.get("independent")
+ if pending.visual_context
+ else None,
),
# A graded card is not an answer to the learner. The verdict and
# the explanation are now on it, so the model that stops here
@@ -2579,6 +2710,7 @@ def _revise_points(
module_id=module.id,
prerequisite_ids=relation_values(raw, "prerequisite_ids", kp.prerequisite_ids),
topic_source_ids=relation_values(raw, "topic_source_ids", kp.topic_source_ids),
+ required_visual_tasks=raw.get("required_visual_tasks", kp.required_visual_tasks),
)
)
rewrite_ids[kp.id] = points[-1].id
@@ -2609,6 +2741,9 @@ def _revise_points(
if isinstance(raw, dict)
else []
),
+ required_visual_tasks=(raw.get("required_visual_tasks") or [])
+ if isinstance(raw, dict)
+ else [],
)
)
@@ -2676,6 +2811,9 @@ def _parse_modules_with_refs(
name=kp_name,
type=KnowledgeType(kp_type),
module_id=module_id,
+ required_visual_tasks=(raw_kp.get("required_visual_tasks") or [])
+ if isinstance(raw_kp, dict)
+ else [],
)
)
relation_refs[kps[-1].id] = RelationRefs(
@@ -2710,6 +2848,7 @@ def _parse_modules(
MASTERY_TOOL_TYPES: tuple[type[BaseTool], ...] = (
MasteryStatusTool,
+ MasteryNoteExplainedTool,
MasteryQuizTool,
MasteryGradeTool,
MasterySkipQuestionTool,
@@ -2736,6 +2875,7 @@ __all__ = [
"MasteryLeaveTool",
"MasteryPathsTool",
"MasteryModeTool",
+ "MasteryNoteExplainedTool",
"MasteryProfileTool",
"MasteryQuizTool",
"MasteryRepairQuestionTool",
diff --git a/deeptutor/learning/assessment.py b/deeptutor/learning/assessment.py
index 96725691b..7dfb21097 100644
--- a/deeptutor/learning/assessment.py
+++ b/deeptutor/learning/assessment.py
@@ -108,6 +108,7 @@ class AssessmentRecord(BaseModel):
created_at: float = Field(default_factory=time.time)
updated_at: float = Field(default_factory=time.time)
bookmarked: bool = False
+ visual_context: dict[str, Any] = Field(default_factory=dict)
resolved: bool | None = None
attempt_id: str = ""
diff --git a/deeptutor/learning/models.py b/deeptutor/learning/models.py
index 555f32d7c..7a1b71961 100644
--- a/deeptutor/learning/models.py
+++ b/deeptutor/learning/models.py
@@ -89,6 +89,9 @@ class KnowledgePoint(BaseModel):
# Selected non-goal sources that justify this objective. They are curriculum
# provenance, not proof of factual correctness or retrieval permission.
topic_source_ids: list[str] = Field(default_factory=list)
+ required_visual_tasks: list[
+ Literal["identification", "relationship", "table_graph", "comparison"]
+ ] = Field(default_factory=list)
class LearningModule(BaseModel):
@@ -132,6 +135,8 @@ class QuizAttempt(BaseModel):
# attempts are excluded from mastery, errors, and spaced repetition.
voided: bool = False
void_reason: str = ""
+ independent: bool = True
+ visual_context: dict[str, Any] = Field(default_factory=dict)
class RetryAttempt(BaseModel):
@@ -181,6 +186,7 @@ class LearningEvidence(BaseModel):
response_time: float | None = None
session_id: str = ""
turn_id: str = ""
+ visual_context: dict[str, Any] = Field(default_factory=dict)
class RepetitionState(BaseModel):
@@ -267,6 +273,8 @@ class PendingQuestion(BaseModel):
explanation: str = ""
difficulty: str = ""
created_at: float = Field(default_factory=time.time)
+ visual_context: dict[str, Any] = Field(default_factory=dict)
+ accepted_answers: list[str] = Field(default_factory=list)
@field_validator("options", mode="before")
@classmethod
@@ -570,6 +578,7 @@ class LearningProgress(BaseModel):
# Temporarily skipped objectives. These never count as mastered; routing
# just prefers any other eligible waypoint until only deferred ones remain.
deferred_objectives: dict[str, DeferredObjective] = Field(default_factory=dict)
+ explained_objectives: dict[str, dict[str, Any]] = Field(default_factory=dict)
# A single outstanding question; grading reads its expected answer so the
# model never has to recall it across turns.
pending_question: PendingQuestion | None = None
diff --git a/deeptutor/learning/policy.py b/deeptutor/learning/policy.py
index de69c1dd1..5989000b4 100644
--- a/deeptutor/learning/policy.py
+++ b/deeptutor/learning/policy.py
@@ -85,11 +85,47 @@ def is_assessed_mastered(progress: LearningProgress, kp: KnowledgePoint) -> bool
* MEMORY / PROCEDURE: recency-weighted accuracy ≥ the type's threshold.
* CONCEPT / DESIGN: a recorded qualitative pass (``mastery_assess``).
"""
+ if not set(kp.required_visual_tasks).issubset(visual_achievements(progress, kp.id)):
+ return False
if kp.type in QUALITATIVE_TYPES:
return bool(progress.qualitative_mastery.get(kp.id, False))
return progress.mastery_levels.get(kp.id, 0.0) >= gate_threshold(kp.type)
+def visual_achievements(progress: LearningProgress, kp_id: str) -> list[str]:
+ """Independent source-verified visual tasks, separate from generic text scores."""
+ return sorted(
+ {
+ event.visual_context["task"]
+ for event in progress.learning_evidence
+ if event.knowledge_point_id == kp_id
+ and event.result == "correct"
+ and event.visual_context.get("task")
+ and event.visual_context.get("key_status") == "verified"
+ and not event.hints_used
+ }
+ )
+
+
+def learning_stage(progress: LearningProgress, kp_id: str) -> str:
+ attempts = [
+ item
+ for item in progress.quiz_attempts
+ if item.knowledge_point_id == kp_id and not item.voided
+ ]
+ evidence = [item for item in progress.learning_evidence if item.knowledge_point_id == kp_id]
+ if any(item.independent and item.is_correct for item in attempts) or any(
+ item.assessment_type == "qualitative" and item.result == "correct" and not item.hints_used
+ for item in evidence
+ ):
+ return "independently_demonstrated"
+ if any(not item.independent for item in attempts) or any(item.hints_used for item in evidence):
+ return "practiced_with_help"
+ if attempts or evidence:
+ return "practiced_not_demonstrated"
+ return "explained" if kp_id in progress.explained_objectives else "not_yet_covered"
+
+
def mastery_source(progress: LearningProgress, kp: KnowledgePoint) -> str:
"""Return ``system`` or ``learner`` provenance for a cleared waypoint."""
@@ -444,6 +480,7 @@ def objective_report(
{
"question_id": attempt.question_id,
"is_correct": attempt.is_correct,
+ "independent": attempt.independent,
"answer": str(attempt.user_answer or ""),
"error_type": attempt.error_type.value if attempt.error_type else "",
"at": attempt.timestamp,
@@ -479,6 +516,9 @@ def objective_report(
"correct_count": sum(1 for attempt in attempts if attempt["is_correct"]),
# The learner's own words, kept as the evidence behind a qualitative pass.
"explanation": progress.feynman_explanations.get(kp_id, ""),
+ "explained": progress.explained_objectives.get(kp_id),
+ "required_visual_tasks": kp.required_visual_tasks,
+ "demonstrated_visual_tasks": visual_achievements(progress, kp_id),
"review": _review_report(progress, kp_id, state, due_at, now=moment),
"errors": [
{
diff --git a/deeptutor/learning/question_card.py b/deeptutor/learning/question_card.py
index ecb0fa512..7afc0ad2c 100644
--- a/deeptutor/learning/question_card.py
+++ b/deeptutor/learning/question_card.py
@@ -66,7 +66,9 @@ def build_question_card(
until the answer is committed, and this payload is rendered to the learner.
"""
public = public_pending_question(pending)
- return {
+ from deeptutor.learning.visual_practice import public_visual
+
+ payload = {
"question_id": public.question_id,
"prompt": public.prompt,
"question_type": public.question_type,
@@ -81,6 +83,9 @@ def build_question_card(
# tap a letter cannot say "I don't know" or show their work.
"allow_free_text": True,
}
+ if pending.visual_context:
+ payload["visual"] = public_visual(pending.visual_context)
+ return payload
def build_grade_result(
@@ -91,6 +96,8 @@ def build_grade_result(
correct_label: str,
choice_options: dict[str, str],
explanation: str,
+ result: str = "",
+ independent: bool | None = None,
) -> dict[str, Any]:
"""What the answered card shows once the gate has ruled on it.
@@ -99,7 +106,7 @@ def build_grade_result(
it is graded — which is exactly when they are worth reading.
"""
label = str(correct_label or "").strip()
- return {
+ payload = {
"question_id": str(question_id or ""),
"is_correct": bool(is_correct),
"learner_answer": str(learner_answer or ""),
@@ -107,6 +114,11 @@ def build_grade_result(
"correct_body": str(choice_options.get(label) or "") if label else "",
"explanation": str(explanation or ""),
}
+ if result:
+ payload["result"] = result
+ if independent is not None:
+ payload["independent"] = independent
+ return payload
__all__ = [
diff --git a/deeptutor/learning/service.py b/deeptutor/learning/service.py
index 19d4e1c97..e5bd1d3f9 100644
--- a/deeptutor/learning/service.py
+++ b/deeptutor/learning/service.py
@@ -2,7 +2,7 @@ from __future__ import annotations
import logging
import time
-from typing import TYPE_CHECKING, Literal
+from typing import TYPE_CHECKING, Any, Literal
import uuid
from deeptutor.learning.grading import classify_error, grade_answer
@@ -82,6 +82,7 @@ def _reserved_objective_ids(progress: LearningProgress) -> set[str]:
ids.update(progress.feynman_retries)
ids.update(progress.feynman_explanations)
ids.update(progress.deferred_objectives)
+ ids.update(progress.explained_objectives)
ids.update(attempt.knowledge_point_id for attempt in progress.quiz_attempts)
ids.update(record.knowledge_point_id for record in progress.error_records)
ids.update(task.knowledge_point_id for task in progress.review_queue)
@@ -173,6 +174,8 @@ def _update_quiz_evidence_for_attempts(
attempt = unmatched.pop(match_index)
event.result = "correct" if attempt.is_correct else "incorrect"
event.quality = 1.0 if attempt.is_correct else 0.0
+ if event.hints_used and attempt.is_correct:
+ event.quality = min(event.quality, 0.6)
def assign_objective_identities(
@@ -351,6 +354,9 @@ class LearningService:
for key in list(progress.deferred_objectives.keys()):
if key not in new_kp_ids:
del progress.deferred_objectives[key]
+ for key in list(progress.explained_objectives):
+ if key not in new_kp_ids:
+ del progress.explained_objectives[key]
progress.error_records = [
r for r in progress.error_records if r.knowledge_point_id in new_kp_ids
]
@@ -459,7 +465,7 @@ class LearningService:
correctness = [
a.is_correct
for a in progress.quiz_attempts
- if a.knowledge_point_id == kp_id and not a.voided
+ if a.knowledge_point_id == kp_id and not a.voided and a.independent
]
return compute_mastery(correctness)
@@ -520,6 +526,7 @@ class LearningService:
scheduler: SpacedRepetitionScheduler | None = None,
session_id: str = "",
turn_id: str = "",
+ visual_context: dict[str, Any] | None = None,
) -> bool:
"""Mutate one aggregate with a grade without performing I/O."""
is_correct = bool(expected_answer) and grade_answer(
@@ -544,6 +551,8 @@ class LearningService:
user_answer=user_answer,
self_attribution=self_attribution,
error_type=None if is_correct else classify_error(user_answer),
+ independent=not visual_context or not visual_context.get("hints_used"),
+ visual_context=visual_context or {},
),
)
evidence = None
@@ -559,6 +568,11 @@ class LearningService:
turn_id=turn_id,
assessment_type="review" if already_scheduled else "quiz",
)
+ if visual_context:
+ evidence.visual_context = visual_context
+ evidence.hints_used = int(visual_context.get("hints_used") or 0)
+ if evidence.hints_used and is_correct:
+ evidence.quality = min(evidence.quality or 1.0, 0.6)
self.update_mastery(
progress, knowledge_point_id, self.calculate_mastery(progress, knowledge_point_id)
)
@@ -920,17 +934,46 @@ class LearningService:
]
pending.expected_answer = authoritative_answer
interaction.question = pending
- is_correct = self._apply_grade(
- tx.progress,
- question_id=pending.question_id,
- knowledge_point_id=pending.knowledge_point_id,
- module_id=pending.module_id,
- user_answer=graded_answer,
- expected_answer=authoritative_answer,
- question_type=pending.question_type,
- scheduler=scheduler,
- session_id=session_id,
- turn_id=turn_id,
+ visual_result = ""
+ visual_diagnosis = ""
+ if pending.visual_context:
+ from deeptutor.learning.visual_practice import (
+ account_for_recent_assistance,
+ evaluate_visual,
+ )
+
+ pending.visual_context = account_for_recent_assistance(
+ pending.visual_context, tx.progress.quiz_attempts, pending.knowledge_point_id
+ )
+ visual_result, visual_diagnosis = evaluate_visual(pending, raw_answer)
+ if (
+ tx.progress.explained_objectives.get(pending.knowledge_point_id, {}).get(
+ "timestamp", 0
+ )
+ > pending.created_at
+ ):
+ pending.visual_context["hints_used"] = max(
+ 1, pending.visual_context.get("hints_used", 0)
+ )
+ pending.visual_context["teaching_after_question"] = True
+ if visual_result == "correct":
+ graded_answer = authoritative_answer
+ is_correct = (
+ None
+ if visual_result == "ungraded"
+ else self._apply_grade(
+ tx.progress,
+ question_id=pending.question_id,
+ knowledge_point_id=pending.knowledge_point_id,
+ module_id=pending.module_id,
+ user_answer=graded_answer,
+ expected_answer=authoritative_answer,
+ question_type=pending.question_type,
+ scheduler=scheduler,
+ session_id=session_id,
+ turn_id=turn_id,
+ visual_context=pending.visual_context,
+ )
)
if (
tx.progress.pending_question is not None
@@ -945,9 +988,18 @@ class LearningService:
"is_correct": is_correct,
"knowledge_point_id": pending.knowledge_point_id,
}
+ if pending.visual_context:
+ interaction.result.update(
+ {
+ "result": visual_result,
+ "diagnosis": visual_diagnosis,
+ "visual_context": pending.visual_context,
+ "independent": not bool(pending.visual_context.get("hints_used")),
+ }
+ )
tx.put_interaction(interaction)
tx.emit(
- "attempt.recorded",
+ "practice.ungraded" if visual_result == "ungraded" else "attempt.recorded",
{
"interaction_id": interaction.interaction_id,
"knowledge_point_id": pending.knowledge_point_id,
@@ -956,7 +1008,7 @@ class LearningService:
session_id=interaction.session_id,
turn_id=interaction.turn_id,
)
- if tx.progress.learning_evidence:
+ if visual_result != "ungraded" and tx.progress.learning_evidence:
latest = tx.progress.learning_evidence[-1]
if latest.knowledge_point_id == pending.knowledge_point_id:
tx.emit(
@@ -1317,6 +1369,7 @@ class LearningService:
progress.review_queue = []
progress.learner_mastery_overrides = {}
progress.deferred_objectives = {}
+ progress.explained_objectives = {}
progress.pending_question = None
progress.feynman_retries = {}
progress.feynman_explanations = {}
@@ -1494,6 +1547,34 @@ class LearningService:
question_type = question.question_type if question is not None else "short"
previous_expected = question.expected_answer if question is not None else ""
previous_is_correct = attempts[0].is_correct if attempts else None
+ if action_name == "correct" and question is not None and question.visual_context:
+ from deeptutor.learning.visual_practice import prepare_visual
+
+ candidate_answer = str(expected_answer or question.expected_answer)
+ if not question.visual_context.get("pixels_inspected"):
+ raise MasteryInteractionError(
+ "This visual practice had no verified pixel inspection. Void it and pose a supported replacement; a challenge cannot create mastery evidence."
+ )
+ try:
+ corrected_visual, corrected_aliases = prepare_visual(
+ {
+ **question.visual_context,
+ "accepted_answers": [],
+ "key_status": "verified",
+ },
+ expected_answer=candidate_answer,
+ options=question.choice_map,
+ )
+ except (OSError, ValueError, TypeError) as exc:
+ raise MasteryInteractionError(
+ "Visual correction lacks current source support; void the assessment and pose a supported replacement."
+ ) from exc
+ if corrected_visual["key_status"] != "verified":
+ raise MasteryInteractionError(
+ "Visual correction lacks a source-supported key; void the assessment instead of granting unsupported mastery."
+ )
+ question.accepted_answers = corrected_aliases
+ question.visual_context = corrected_visual
user_answer = (interaction.user_answer if interaction is not None else "") or (
str(attempts[0].user_answer or "") if attempts else ""
)
diff --git a/deeptutor/learning/tests/test_visual_learning_workflow.py b/deeptutor/learning/tests/test_visual_learning_workflow.py
new file mode 100644
index 000000000..68c84d2f7
--- /dev/null
+++ b/deeptutor/learning/tests/test_visual_learning_workflow.py
@@ -0,0 +1,440 @@
+"""Inspectable source-to-question-to-durable-evidence workflows (#1611)."""
+
+from __future__ import annotations
+
+from hashlib import sha256
+import json
+from types import SimpleNamespace
+
+from PIL import Image, ImageDraw
+import pymupdf
+import pytest
+
+from deeptutor.capabilities.mastery import tools
+from deeptutor.learning.models import (
+ KnowledgePoint,
+ KnowledgeType,
+ LearningModule,
+ LearningProgress,
+)
+from deeptutor.learning.policy import is_assessed_mastered, visual_achievements
+from deeptutor.learning.scheduler import SpacedRepetitionScheduler
+from deeptutor.learning.service import LearningService
+from deeptutor.learning.storage import LearningStore
+from deeptutor.services.parsing.types import ParsedDocument
+from deeptutor.services.rag.visual_assets import VisualAssetStore, collect_visual_assets
+from deeptutor.services.session.sqlite_store import SQLiteSessionStore
+
+
+def lesson(tmp_path, monkeypatch):
+ kb = tmp_path / "kb"
+ (kb / "raw").mkdir(parents=True)
+ source = kb / "raw" / "anatomy.pdf"
+ image = tmp_path / "images" / "numbered.png"
+ image.parent.mkdir()
+ picture = Image.new("RGB", (400, 250), "white")
+ draw = ImageDraw.Draw(picture)
+ draw.ellipse((40, 40, 160, 200), outline="red", width=4)
+ draw.text((95, 110), "1", fill="black")
+ draw.ellipse((220, 40, 340, 200), outline="blue", width=4)
+ draw.text((270, 110), "2", fill="black")
+ picture.save(image)
+ pdf = pymupdf.open()
+ page = pdf.new_page()
+ page.insert_text(
+ (50, 50), "Figure 1: Number 1 refers to an artery. The artery lies left of the vein."
+ )
+ page.insert_image(pymupdf.Rect(50, 80, 450, 330), filename=str(image))
+ page.insert_text((50, 360), "Table 1: speed (m/s), row A: 12, row B: 20. Row B is larger.")
+ pdf.save(source, deflate=True)
+ pdf.close()
+ parsed = ParsedDocument(
+ markdown="",
+ asset_dir=image.parent,
+ parser_signature="fixture",
+ blocks=[
+ {
+ "type": "image",
+ "img_path": str(image),
+ "page_idx": 0,
+ "image_caption": ["Figure 1: numbered structures"],
+ "text": "Number 1 refers to an artery. The artery lies left of the vein.",
+ }
+ ],
+ )
+ candidate = collect_visual_assets(parsed, source, kb)[0]
+ VisualAssetStore(kb).publish([candidate])
+ monkeypatch.setattr(
+ "deeptutor.multi_user.knowledge_access.resolve_for_rag",
+ lambda name: SimpleNamespace(base_dir=kb.parent, name=kb.name) if name == "kb" else None,
+ )
+ store = LearningStore(tmp_path / "learning")
+ service = LearningService(store)
+ progress = LearningProgress(book_id="visual")
+ kp = KnowledgePoint(
+ id="identify",
+ name="Identify and interpret the original source",
+ type=KnowledgeType.MEMORY,
+ module_id="m1",
+ required_visual_tasks=["identification"],
+ )
+ progress.modules = [
+ LearningModule(id="m1", name="Source practice", order=0, knowledge_points=[kp])
+ ]
+ progress.knowledge_types[kp.id] = kp.type
+ service.save(progress)
+ monkeypatch.setattr(tools, "_new_service", lambda: service)
+ session_store = SQLiteSessionStore(tmp_path / "chat.db")
+ monkeypatch.setattr(
+ "deeptutor.services.session.get_sqlite_session_store", lambda: session_store
+ )
+ return SimpleNamespace(
+ kb=kb,
+ source=source,
+ asset=candidate.record["asset_id"],
+ image_hash=candidate.record["image_sha256"],
+ service=service,
+ store=store,
+ session_store=session_store,
+ kp=kp,
+ )
+
+
+def visual(lesson, *, task="identification", cues="none", quote="Number 1 refers to an artery."):
+ return {
+ "task": task,
+ "sources": [{"kb_name": "kb", "asset_id": lesson.asset}],
+ "reference_quote": quote,
+ "accepted_answers": ["artery", "an artery", "动脉"],
+ "key_status": "verified",
+ "answer_cues": cues,
+ }
+
+
+async def pose(lesson, **overrides):
+ kwargs = {
+ "_mastery_path_id": "visual",
+ "_attached_kb_names": ["kb"],
+ "_inspected_image_hashes": [lesson.image_hash],
+ "knowledge_point_id": "identify",
+ "question": "Identify numbered structure 1 in the original figure.",
+ "expected_answer": "artery",
+ "explanation": "The source explains that structure 1 is an artery; the original numbered figure supplies the target.",
+ "visual": visual(lesson),
+ **overrides,
+ }
+ if "_inspected_image_hashes" not in overrides:
+ from deeptutor.services.rag.source_visuals import retrieve_visual
+
+ kwargs["_inspected_image_hashes"] = [
+ sha256(
+ retrieve_visual(
+ lesson.kb,
+ "kb",
+ **{
+ key: ref[key]
+ for key in ("asset_id", "source_path", "page", "source_hash", "region")
+ if ref.get(key) is not None
+ },
+ ).images[0][1]
+ ).hexdigest()
+ for ref in kwargs["visual"]["sources"]
+ ]
+ result = await tools.MasteryQuizTool().execute(**kwargs)
+ assert result.success, result.content
+ card = result.metadata["mastery_question"]
+ return card
+
+
+async def test_teach_then_source_visual_practice_equivalent_answer_and_restart(lesson):
+ explained = await tools.MasteryNoteExplainedTool().execute(
+ _mastery_path_id="visual",
+ knowledge_point_id="identify",
+ summary="Explained artery/vein prerequisites concisely at beginner level using the numbered source.",
+ )
+ assert explained.success
+ before = lesson.store.load("visual")
+ assert before.explained_objectives["identify"]["summary"]
+ assert before.mastery_levels.get("identify", 0) == 0
+ card = await pose(lesson)
+ assert card["visual"]["sources"][0]["image_url"].endswith(lesson.asset + "?dt_workspace=")
+ assert "reference_quote" not in json.dumps(card) and "accepted_answers" not in json.dumps(card)
+ result = await tools.MasteryGradeTool().execute(
+ _mastery_path_id="visual", question_id=card["question_id"], answer="动脉"
+ )
+ assert result.success
+ grade = result.metadata["mastery_grade"]
+ assert grade["result"]["is_correct"]
+ fresh = LearningService(LearningStore(lesson.store._root))
+ progress = fresh.store.load("visual")
+ assert progress.quiz_attempts[0].visual_context["sources"][0]["asset_id"] == lesson.asset
+ assert progress.quiz_attempts[0].independent
+ assert visual_achievements(progress, "identify") == ["identification"]
+
+
+async def test_assistance_immediate_repeat_and_delayed_independence_are_distinct(lesson):
+ first = await pose(lesson, visual=visual(lesson, cues="visible"))
+ lesson.service.grade_interaction(
+ "visual",
+ answer="artery",
+ question_id=first["question_id"],
+ scheduler=SpacedRepetitionScheduler(),
+ )
+ second = await pose(lesson)
+ progress, _, _ = lesson.service.grade_interaction(
+ "visual",
+ answer="artery",
+ question_id=second["question_id"],
+ scheduler=SpacedRepetitionScheduler(),
+ )
+ assert all(not attempt.independent for attempt in progress.quiz_attempts)
+ assert not is_assessed_mastered(progress, lesson.kp)
+ assert progress.mastery_levels["identify"] == 0
+ assert all(e.hints_used and e.quality <= 0.6 for e in progress.learning_evidence)
+ for attempt in progress.quiz_attempts:
+ attempt.timestamp -= 86400
+ lesson.service.save(progress)
+ third = await pose(lesson)
+ progress, _, _ = lesson.service.grade_interaction(
+ "visual",
+ answer="an artery",
+ question_id=third["question_id"],
+ scheduler=SpacedRepetitionScheduler(),
+ )
+ assert progress.quiz_attempts[-1].independent
+ assert visual_achievements(progress, "identify") == ["identification"]
+
+
+async def test_supported_wrong_choice_is_graded_and_faulty_assessment_repair_removes_effect(lesson):
+ card = await pose(
+ lesson,
+ question_type="choice",
+ options=[{"label": "A", "body": "artery"}, {"label": "B", "body": "vein"}],
+ expected_answer="A",
+ )
+ wrong = next(option["label"] for option in card["options"] if option["body"] == "vein")
+ progress, interaction, _ = lesson.service.grade_interaction(
+ "visual",
+ answer=wrong,
+ question_id=card["question_id"],
+ scheduler=SpacedRepetitionScheduler(),
+ )
+ assert interaction.result["result"] == "incorrect"
+ assert progress.learning_evidence[0].result == "incorrect"
+ from deeptutor.learning.service import MasteryInteractionError
+
+ with pytest.raises(MasteryInteractionError, match="source-supported"):
+ lesson.service.repair_question(
+ "visual",
+ question_id=card["question_id"],
+ action="correct",
+ expected_answer=wrong,
+ reason="Learner says their answer should be accepted.",
+ )
+ progress, details = lesson.service.repair_question(
+ "visual",
+ question_id=card["question_id"],
+ action="void",
+ reason="Independent review found the target labeling ambiguous.",
+ scheduler=SpacedRepetitionScheduler(),
+ )
+ assert progress.quiz_attempts[0].voided
+ assert not progress.learning_evidence
+ assert "identify" not in progress.repetition_states
+ assert progress.mastery_levels.get("identify", 0) == 0
+
+
[email protected]("case", ["key", "wording", "changed", "conflicting_key"])
+async def test_ambiguous_unverified_or_changed_evidence_creates_no_negative_mastery(lesson, case):
+ v = visual(lesson)
+ if case == "key":
+ v["reference_quote"] = "Invented unsupported key"
+ card = await pose(
+ lesson, visual=v, expected_answer="vein" if case == "conflicting_key" else "artery"
+ )
+ if case == "changed":
+ lesson.source.write_bytes(lesson.source.read_bytes() + b"changed source")
+ answer = "some unverified terminology" if case == "wording" else "artery"
+ progress, interaction, _ = lesson.service.grade_interaction(
+ "visual",
+ answer=answer,
+ question_id=card["question_id"],
+ scheduler=SpacedRepetitionScheduler(),
+ )
+ assert interaction.result["result"] == "ungraded"
+ assert interaction.result["is_correct"] is None
+ assert (
+ not progress.quiz_attempts and not progress.learning_evidence and not progress.error_records
+ )
+ assert not progress.repetition_states
+
+
+async def test_retry_is_one_attempt_but_new_question_is_new_evidence_and_outline_cannot_transfer_it(
+ lesson,
+):
+ first = await pose(lesson)
+ for _ in range(2):
+ lesson.service.grade_interaction(
+ "visual", answer="artery", question_id=first["question_id"]
+ )
+ second = await pose(lesson)
+ progress, _, _ = lesson.service.grade_interaction(
+ "visual", answer="artery", question_id=second["question_id"]
+ )
+ assert len(progress.quiz_attempts) == len(progress.learning_evidence) == 2
+ old_id = progress.modules[0].knowledge_points[0].id
+ replacement = [
+ LearningModule(
+ id="m1",
+ name="Different subject",
+ order=0,
+ knowledge_points=[
+ KnowledgePoint(
+ id=old_id,
+ name="Unrelated geometry theorem",
+ type=KnowledgeType.MEMORY,
+ module_id="m1",
+ )
+ ],
+ )
+ ]
+ progress = lesson.service.replace_modules_for_path("visual", replacement)
+ assert progress.modules[0].knowledge_points[0].id != old_id
+ assert not is_assessed_mastered(progress, progress.modules[0].knowledge_points[0])
+
+
+async def test_relationship_table_and_comparison_use_both_original_sources(lesson):
+ for task, expected, quote in [
+ ("relationship", "left", "The artery lies left of the vein."),
+ ("table_graph", "Row B", "Row B is larger."),
+ ]:
+ refs = (
+ [{"kb_name": "kb", "asset_id": lesson.asset}]
+ if task == "relationship"
+ else [{"kb_name": "kb", "source_path": "anatomy.pdf", "page": 1}]
+ )
+ card = await pose(
+ lesson,
+ expected_answer=expected,
+ question="Interpret the relationship/value in the original source.",
+ visual={
+ "task": task,
+ "sources": refs,
+ "reference_quote": quote,
+ "key_status": "verified",
+ "accepted_answers": [expected],
+ "answer_cues": "none",
+ },
+ )
+ progress, interaction, _ = lesson.service.grade_interaction(
+ "visual", answer=expected, question_id=card["question_id"]
+ )
+ assert interaction.result["result"] == "correct"
+ assert progress.learning_evidence[-1].visual_context["task"] == task
+ v = visual(lesson, task="comparison")
+ v["sources"] = [v["sources"][0], {"kb_name": "kb", "source_path": "anatomy.pdf", "page": 1}]
+ card = await pose(lesson, visual=v)
+ assert len(card["visual"]["sources"]) == 2
+
+
+def test_generic_text_success_does_not_satisfy_required_visual_task(lesson):
+ progress = lesson.store.load("visual")
+ progress.mastery_levels["identify"] = 1.0
+ assert not is_assessed_mastered(progress, lesson.kp)
+
+
+async def test_pdf_region_can_assess_relationship_without_displaying_the_reference_key(lesson):
+ card = await pose(
+ lesson,
+ expected_answer="left",
+ question="Describe where numbered structure 1 is relative to structure 2.",
+ visual={
+ "task": "relationship",
+ "sources": [
+ {
+ "kb_name": "kb",
+ "source_path": "anatomy.pdf",
+ "page": 1,
+ "region": [0.07, 0.08, 0.8, 0.4],
+ }
+ ],
+ "reference_quote": "The artery lies left of the vein.",
+ "key_status": "verified",
+ "accepted_answers": ["left", "左侧"],
+ "answer_cues": "none",
+ },
+ )
+ assert card["visual"]["hints_used"] == 0
+ assert card["visual"]["key_status"] == "verified"
+ progress, interaction, _ = lesson.service.grade_interaction(
+ "visual", answer="左侧", question_id=card["question_id"]
+ )
+ assert interaction.result["result"] == "correct"
+ assert progress.quiz_attempts[-1].independent
+ assert "relationship" in visual_achievements(progress, "identify")
+
+
+async def test_help_after_question_is_assisted_and_deferral_never_claims_mastery(lesson):
+ card = await pose(lesson)
+ await tools.MasteryNoteExplainedTool().execute(
+ _mastery_path_id="visual",
+ knowledge_point_id="identify",
+ summary="Gave a source-based hint after the learner asked for help with structure 1.",
+ )
+ progress, _interaction, _ = lesson.service.grade_interaction(
+ "visual", answer="artery", question_id=card["question_id"]
+ )
+ assert not progress.quiz_attempts[0].independent
+ assert progress.learning_evidence[0].hints_used
+ deferred = await tools.MasteryDeferObjectiveTool().execute(
+ _mastery_path_id="visual", knowledge_point_id="identify", reason="Move on for now"
+ )
+ assert deferred.success
+ fresh = lesson.store.load("visual")
+ assert "identify" in fresh.deferred_objectives
+ assert not is_assessed_mastered(fresh, lesson.kp)
+
+
+async def test_declared_vision_or_a_url_without_matching_accepted_pixels_is_guided(lesson):
+ card = await pose(lesson, _inspected_image_hashes=["some_other_image"])
+ assert card["visual"]["answer_cues"] == "unverified"
+ progress, interaction, _replayed = lesson.service.grade_interaction(
+ "visual", answer="artery", question_id=card["question_id"]
+ )
+ assert interaction.result["result"] == "ungraded"
+ assert not progress.quiz_attempts
+ assert not visual_achievements(progress, "identify")
+
+
+async def test_canonical_writer_keeps_visual_context_and_ungraded_attempt_once(lesson):
+ session = await lesson.session_store.create_session()
+ v = visual(lesson)
+ v["key_status"] = "unverified"
+ card = await pose(lesson, visual=v)
+ for _ in range(2):
+ result = await tools.MasteryGradeTool().execute(
+ _mastery_path_id="visual",
+ _session_id=session["id"],
+ _turn_id="turn1",
+ question_id=card["question_id"],
+ answer="artery",
+ )
+ assert result.success
+ assert result.metadata["mastery_grade"]["result"]["result"] == "ungraded"
+ with lesson.session_store._connect() as connection:
+ row = connection.execute("SELECT assessment_json FROM assessment_attempts").fetchall()
+ assert len(row) == 1
+ payload = json.loads(row[0][0])
+ assert payload["result"] == "ungraded"
+ assert payload["visual_context"]["sources"][0]["asset_id"] == lesson.asset
diff --git a/deeptutor/learning/visual_practice.py b/deeptutor/learning/visual_practice.py
new file mode 100644
index 000000000..33aec6cbb
--- /dev/null
+++ b/deeptutor/learning/visual_practice.py
@@ -0,0 +1,250 @@
+"""Source-bound visual questions, conservative keys and explicit assistance (#1611)."""
+
+from __future__ import annotations
+
+from hashlib import sha256
+import re
+from typing import Any
+import unicodedata
+
+
+def normalized_answer(value: str) -> str:
+ text = unicodedata.normalize("NFKC", str(value)).casefold().strip()
+ return re.sub(r"\s+", " ", text).rstrip(".。!!")
+
+
+def _answer_in_quote(answer: str, quote: str) -> bool:
+ needle = normalized_answer(answer)
+ if not needle:
+ return False
+ if needle.isascii():
+ return re.search(r"(?<![a-z0-9])" + re.escape(needle) + r"(?![a-z0-9])", quote) is not None
+ return needle in quote
+
+
+def prepare_visual(
+ request: dict[str, Any],
+ *,
+ expected_answer: str,
+ options: dict[str, str],
+ attached_kbs: list[str] | None = None,
+ inspected_image_hashes: list[str] | None = None,
+) -> tuple[dict[str, Any], list[str]]:
+ from deeptutor.multi_user.knowledge_access import resolve_for_rag
+ from deeptutor.services.rag.kb_paths import resolve_kb_dir
+ from deeptutor.services.rag.source_visuals import retrieve_visual
+
+ task = str(request.get("task") or "identification")
+ if task not in {"identification", "relationship", "table_graph", "comparison"}:
+ raise ValueError(
+ "Visual tasks support identification, relationship, table_graph and comparison."
+ )
+ refs = request.get("sources")
+ if not isinstance(refs, list) or not 1 <= len(refs) <= 2:
+ raise ValueError("Select one or two exact source figures/pages; comparisons require two.")
+ if task == "comparison" and len(refs) != 2:
+ raise ValueError("A visual comparison requires both original sources.")
+ resolved = []
+ source_text = []
+ for ref in refs:
+ if not isinstance(ref, dict):
+ raise ValueError("Each visual source must be an object.")
+ kb_name = str(ref.get("kb_name") or "")
+ if attached_kbs is not None and kb_name not in attached_kbs:
+ raise ValueError(
+ "Visual practice must use a knowledge base attached to this topic/turn."
+ )
+ resource = resolve_for_rag(kb_name)
+ if resource is None:
+ raise ValueError("The selected source knowledge base is not accessible.")
+ selectors = {
+ key: ref[key]
+ for key in ("asset_id", "source_path", "page", "region", "source_hash", "figure")
+ if ref.get(key) is not None
+ }
+ evidence = retrieve_visual(
+ resolve_kb_dir(resource.base_dir, resource.name), kb_name, **selectors
+ )
+ if evidence.error or not evidence.images:
+ raise ValueError(f"Cannot establish original visual evidence: {evidence.content}")
+ record = evidence.images[0][0]
+ resolved.append(
+ {
+ "kb_name": kb_name,
+ "asset_id": record.get("asset_id") or "",
+ "source_path": record["source_path"],
+ "page": record.get("page_number"),
+ "source_hash": record["source_document_id"],
+ "region": record.get("region"),
+ "image_url": evidence.sources[0]["visual_asset_url"],
+ "url": evidence.sources[0]["url"],
+ "image_sha256": sha256(evidence.images[0][1]).hexdigest(),
+ "visible_answer_text": _answer_in_quote(
+ options.get(expected_answer, expected_answer),
+ normalized_answer(record.get("text", "")),
+ ),
+ }
+ )
+ source_text.append(
+ "\n".join(
+ str(record.get(key) or "")
+ for key in ("caption", "context", "text", "page_context", "table_html", "notes")
+ )
+ )
+ quote = str(request.get("reference_quote") or "").strip()[:3000]
+ # Answer equivalences are explicit and source-specific, never fuzzy
+ # substring/keyword overlap. Unknown prose is clarified, not penalized.
+ correct_body = options.get(expected_answer, expected_answer)
+ raw_aliases = request.get("accepted_answers") or []
+ if not isinstance(raw_aliases, list) or len(raw_aliases) > 12:
+ raise ValueError("accepted_answers must contain at most 12 verified equivalents.")
+ aliases = list(
+ dict.fromkeys(
+ [correct_body, *[str(x).strip()[:200] for x in raw_aliases if str(x).strip()]]
+ )
+ )
+ text = normalized_answer(re.sub(r"<[^>]+>", " ", "\n".join(source_text)))
+ quote_normalized = normalized_answer(re.sub(r"<[^>]+>", " ", quote))
+ verified = bool(
+ quote_normalized
+ and quote_normalized in text
+ and _answer_in_quote(correct_body, quote_normalized)
+ and request.get("key_status") == "verified"
+ )
+ cues = str(request.get("answer_cues") or "unverified")
+ if cues not in {"none", "visible", "unverified"}:
+ raise ValueError(
+ "answer_cues must be none, visible or unverified; masking requires independently verified regions."
+ )
+ # Page text can reveal a table value directly. Treat it as guided reading,
+ # even when the caller mistakenly declares no visible cue.
+ inspected = (
+ all(ref.get("image_sha256") in inspected_image_hashes for ref in resolved)
+ if inspected_image_hashes is not None
+ else bool(request.get("pixels_inspected"))
+ )
+ if not inspected:
+ verified = False
+ cues = "unverified"
+ elif any(
+ ref.get("page")
+ and not ref.get("asset_id")
+ and (not ref.get("region") or ref.get("visible_answer_text"))
+ for ref in resolved
+ ):
+ cues = "visible"
+ return {
+ "task": task,
+ "sources": resolved,
+ "answer_cues": cues,
+ "key_status": "verified" if verified else "unverified",
+ "pixels_inspected": inspected,
+ "reference_quote": quote,
+ "reference_answer": correct_body,
+ "hints_used": max(int(request.get("hints_used") or 0), int(cues != "none")),
+ }, aliases
+
+
+def public_visual(context: dict[str, Any]) -> dict[str, Any]:
+ return {
+ key: context[key]
+ for key in (
+ "task",
+ "sources",
+ "answer_cues",
+ "key_status",
+ "hints_used",
+ "pixels_inspected",
+ )
+ if key in context
+ }
+
+
+def account_for_recent_assistance(context: dict[str, Any], attempts, kp_id: str) -> dict[str, Any]:
+ """Use the existing retention session boundary, not a visual scheduler."""
+ import time
+
+ from deeptutor.learning.scheduler import _SAME_SESSION_DAYS
+
+ def locations(visual):
+ return {(ref.get("source_hash"), ref.get("page")) for ref in visual.get("sources", [])}
+
+ current = locations(context)
+ for attempt in reversed(attempts):
+ prior = attempt.visual_context
+ if (
+ not attempt.voided
+ and attempt.knowledge_point_id == kp_id
+ and (
+ prior.get("hints_used")
+ or (
+ normalized_answer(prior.get("reference_quote", ""))
+ == normalized_answer(context.get("reference_quote", ""))
+ and normalized_answer(prior.get("reference_answer", ""))
+ == normalized_answer(context.get("reference_answer", ""))
+ )
+ )
+ and prior.get("task") == context.get("task")
+ and locations(prior) == current
+ and time.time() - attempt.timestamp < _SAME_SESSION_DAYS * 86400
+ ):
+ return {
+ **context,
+ "hints_used": max(1, context.get("hints_used", 0)),
+ "recently_assisted": True,
+ }
+ return context
+
+
+def evaluate_visual(pending, answer: str) -> tuple[str, str]:
+ """Return (result, diagnosis); uncertainty produces no mastery evidence."""
+ from deeptutor.multi_user.knowledge_access import resolve_for_rag
+ from deeptutor.services.rag.kb_paths import resolve_kb_dir
+ from deeptutor.services.rag.source_visuals import retrieve_visual
+
+ visual = pending.visual_context
+ if not visual.get("pixels_inspected"):
+ return (
+ "ungraded",
+ "No matching source pixels were verified in the tutor's model input. Re-retrieve the original image with a vision-capable model; this attempt does not lower mastery.",
+ )
+ if visual.get("key_status") != "verified" or not visual.get("sources"):
+ return (
+ "ungraded",
+ "The reference key is not independently supported by source text; clarify it before grading.",
+ )
+ for ref in visual.get("sources", []):
+ resource = resolve_for_rag(ref["kb_name"])
+ if resource is None:
+ return "ungraded", "Source access is unavailable; this attempt does not lower mastery."
+ selectors = {
+ key: ref[key]
+ for key in ("asset_id", "source_path", "page", "region", "source_hash")
+ if ref.get(key) not in (None, "")
+ }
+ evidence = retrieve_visual(
+ resolve_kb_dir(resource.base_dir, resource.name), ref["kb_name"], **selectors
+ )
+ if evidence.error or not evidence.images:
+ return (
+ "ungraded",
+ "Original source evidence changed or is unavailable; clarify the source before assessment.",
+ )
+ normalized = normalized_answer(answer)
+ aliases = {normalized_answer(alias) for alias in pending.accepted_answers}
+ if normalized in aliases:
+ return "correct", "Accepted source-specific equivalent terminology."
+ if pending.question_type == "choice":
+ from deeptutor.learning.pending import is_readable_choice_answer, resolve_choice_submission
+
+ if not is_readable_choice_answer(answer, pending.choice_map):
+ return "ungraded", "Clarify the selected answer before assessing it."
+ label = resolve_choice_submission(answer, pending.choice_map)
+ if label:
+ return (
+ "correct" if label == pending.expected_answer else "incorrect"
+ ), "Compare the chosen answer with the original source evidence."
+ return (
+ "ungraded",
+ "This wording is not a verified equivalent. Ask for clarification or repair the reference; do not infer a wrong answer from surface similarity.",
+ )
diff --git a/deeptutor/runtime/agentic/messages.py b/deeptutor/runtime/agentic/messages.py
index 649a38f6b..792bb909b 100644
--- a/deeptutor/runtime/agentic/messages.py
+++ b/deeptutor/runtime/agentic/messages.py
@@ -5,6 +5,42 @@ from __future__ import annotations
from typing import Any
+def image_content_hashes(messages: list[dict]) -> list[str]:
+ """Hashes of bounded image bytes on the accepted provider wire (#1611)."""
+ from base64 import b64decode
+ from hashlib import sha256
+
+ hashes = set()
+ for message in messages:
+ content = message.get("content")
+ for part in content if isinstance(content, list) else []:
+ if not isinstance(part, dict):
+ continue
+ value = part.get("image_url")
+ url = value.get("url") if isinstance(value, dict) else value
+ source = part.get("source")
+ if (
+ part.get("type") == "image"
+ and isinstance(source, dict)
+ and source.get("type") == "base64"
+ ):
+ url = f"data:{source.get('media_type', '')};base64,{source.get('data', '')}"
+ if (
+ not isinstance(url, str)
+ or not url.startswith("data:image/")
+ or ";base64," not in url
+ ):
+ continue
+ encoded = url.split(";base64,", 1)[1]
+ if len(encoded) > 7 * 1024 * 1024:
+ continue
+ try:
+ hashes.add(sha256(b64decode(encoded, validate=True)).hexdigest())
+ except ValueError:
+ continue
+ return sorted(hashes)
+
+
def assistant_message_with_tool_calls(
content: str,
tool_calls: list[dict[str, Any]],
diff --git a/deeptutor/runtime/bootstrap/builtin_capabilities.py b/deeptutor/runtime/bootstrap/builtin_capabilities.py
index 11a4ec989..cf9552e38 100644
--- a/deeptutor/runtime/bootstrap/builtin_capabilities.py
+++ b/deeptutor/runtime/bootstrap/builtin_capabilities.py
@@ -166,6 +166,7 @@ BUILTIN_CAPABILITY_SPECS: dict[str, BuiltinCapabilitySpec] = {
tools_used=[
"mastery_status",
"mastery_quiz",
+ "mastery_note_explained",
"mastery_grade",
"mastery_skip_question",
"mastery_repair_question",
diff --git a/deeptutor/services/rag/source_visuals.py b/deeptutor/services/rag/source_visuals.py
index a276e435b..f742d6311 100644
--- a/deeptutor/services/rag/source_visuals.py
+++ b/deeptutor/services/rag/source_visuals.py
@@ -128,6 +128,7 @@ def page_image(
"Page image exceeds the model byte budget. Select a smaller region."
)
text = sheet.get_text("text", clip=clip)[:6000]
+ page_context = sheet.get_text("text")[:6000]
except ValueError:
raise
except Exception as exc:
@@ -143,6 +144,7 @@ def page_image(
"region": region,
"mime_type": mime,
"text": text,
+ "page_context": page_context,
"render_scale": scale,
"kind": "source_page",
"source_locator": f"page:{page}",
diff --git a/deeptutor/tools/builtin_specs.py b/deeptutor/tools/builtin_specs.py
index 6f32ec015..3b9223e9c 100644
--- a/deeptutor/tools/builtin_specs.py
+++ b/deeptutor/tools/builtin_specs.py
@@ -105,6 +105,7 @@ BUILTIN_TOOL_SPECS: tuple[BuiltinToolSpec, ...] = (
(
("mastery_status", "MasteryStatusTool"),
("mastery_quiz", "MasteryQuizTool"),
+ ("mastery_note_explained", "MasteryNoteExplainedTool"),
("mastery_grade", "MasteryGradeTool"),
("mastery_skip_question", "MasterySkipQuestionTool"),
("mastery_repair_question", "MasteryRepairQuestionTool"),
diff --git a/deeptutor/tools/mastery_tool.py b/deeptutor/tools/mastery_tool.py
index 17d51fc51..412c6102b 100644
--- a/deeptutor/tools/mastery_tool.py
+++ b/deeptutor/tools/mastery_tool.py
@@ -15,6 +15,7 @@ from deeptutor.capabilities.mastery.tools import (
MasteryGradeTool,
MasteryLeaveTool,
MasteryModeTool,
+ MasteryNoteExplainedTool,
MasteryPathsTool,
MasteryProfileTool,
MasteryQuizTool,
@@ -34,6 +35,7 @@ __all__ = [
"MasteryGradeTool",
"MasteryLeaveTool",
"MasteryModeTool",
+ "MasteryNoteExplainedTool",
"MasteryPathsTool",
"MasteryProfileTool",
"MasteryQuizTool",
diff --git a/docs-for-user/SOURCE_VISUALS.md b/docs-for-user/SOURCE_VISUALS.md
index b37e8f52b..63e2d9bff 100644
--- a/docs-for-user/SOURCE_VISUALS.md
+++ b/docs-for-user/SOURCE_VISUALS.md
@@ -79,6 +79,61 @@ unavailable pixels and changed-source abstentions are checked independently of
language-model fluency.
+## Source-grounded practice and learning evidence
+
+In Guided Learning, request explanation, practice or review explicitly. An
+explanation-only turn need not pose a question; `mastery_note_explained` records
+what was taught without granting mastery. Deferring an objective preserves it
+as unfinished. For goals that require visual understanding, outline objectives
+can declare `required_visual_tasks`: `identification`, `relationship`,
+`table_graph` or `comparison`. Generic text scores cannot satisfy those visual
+requirements. The objective view shows the remaining independent evidence.
+
+Ask for practice on an exact original figure or page from an attached KB. The
+tutor retrieves its pixels, then poses a `mastery_quiz` with a `visual` context.
+The question card displays the original evidence and document/page links.
+Numbered-label identification, source-backed relationship questions, table or
+graph interpretation and bounded two-source comparisons are supported. No
+automatic segmentation, invented label masks or generated replacement diagram
+is used. A source image that fails to load disables submission while still
+allowing the learner to skip or ask for help.
+
+The reference key uses canonical source terminology and a supporting caption,
+explanation or page quote. Explicitly verified equivalent terms, including
+other languages, can answer the same question. Choice questions distinguish
+supported wrong answers; unknown free-text wording asks for clarification
+instead of fuzzy keyword grading. Uncertain/conflicting keys, inaccessible or
+changed sources, and missing model-input pixels remain ungraded and create no
+negative mastery or retention evidence. A figure URL or a declared vision
+capability is insufficient: the active loop checks matching image bytes in the
+accepted provider request, including actual text-only fallback.
+
+Visible/unknown answer cues are guided practice, with assistance retained and
+weaker review evidence. They do not demonstrate independent mastery. A full
+page that exposes the reference answer stays guided; a verified original-page
+region can support independent interpretation when the answer cue is outside
+the view, with the full source retained for reference verification. Recent
+assistance, repeated answer-key exposure and help given after a question also
+remain distinguishable through the existing retention session boundary.
+
+Use **Review or challenge this assessment** to request source review. Existing
+`mastery_repair_question` invalidates a faulty assessment and recomputes its
+mastery/error/review effects. Correcting a visual key also requires source
+support; a challenge alone never grants mastery. Manual learner overrides retain
+their separate provenance. Source/figure identity, objective, task, cues/help,
+answer and attempt identity survive restart. A transport retry records one
+attempt; a new question records new evidence. Outline replacement preserves
+trusted objective identity rather than transferring an achievement by title or
+position. Unassessed, explained, assisted and independently demonstrated work
+remain distinct.
+
+Regression workflows use independently specified synthetic numbered figures,
+relationships and table values, including equivalent-language answers, wrong
+choices, ungraded ambiguity, repair, deferral, repeat submissions, outline
+replacement, source changes and restart. This supported set provides explicit
+recovery for uncertainty rather than promising unrestricted grading of every
+image or arbitrary prose answer.
+
## Token estimates for image requests
When the provider returns usage, DeepTutor uses those reported counters. When
diff --git a/tests/agents/chat/test_agent_loop.py b/tests/agents/chat/test_agent_loop.py
index d55096d88..bdd1840e4 100644
--- a/tests/agents/chat/test_agent_loop.py
+++ b/tests/agents/chat/test_agent_loop.py
@@ -305,6 +305,9 @@ async def test_active_mastery_retrieves_verified_pixels_without_persisting_them(
]
if evidence == "available":
assert len(images) == 1
+ assert context.extension("source_visual_evidence")["image_hashes"] == [
+ candidate.record["image_sha256"]
+ ]
assert (
base64.b64decode(images[0]["image_url"]["url"].split(",", 1)[1]) == image.read_bytes()
)
@@ -318,6 +321,7 @@ async def test_active_mastery_retrieves_verified_pixels_without_persisting_them(
assert candidate.record["asset_id"] in str(request[image_position])
else:
assert images == []
+ assert context.extension("source_visual_evidence")["image_hashes"] == []
tool_text = next(msg["content"] for msg in request if msg.get("tool_call_id") == "figure")
assert (
"cannot inspect source image pixels"
@@ -3824,4 +3828,5 @@ async def test_image_fallback_strips_all_canonical_copies_after_wire_deduplicati
assert not has_image_parts(attempted[1])
assert not has_image_parts(attempted[2])
assert not has_image_parts(messages)
+ assert context.extension("source_visual_evidence")["image_hashes"] == []
assert "Repeated image" not in str(messages)
diff --git a/web/components/chat/home/MasteryQuestionCard.tsx b/web/components/chat/home/MasteryQuestionCard.tsx
index e1c9678cb..a100da28a 100644
--- a/web/components/chat/home/MasteryQuestionCard.tsx
+++ b/web/components/chat/home/MasteryQuestionCard.tsx
@@ -100,6 +100,7 @@ export const MasteryQuestionCard = memo(function MasteryQuestionCard({
submittedAnswer,
onSubmit,
onSkip,
+ onChallenge,
}: {
question: MasteryQuestion;
/** The gate's ruling, once it has been made. */
@@ -117,10 +118,12 @@ export const MasteryQuestionCard = memo(function MasteryQuestionCard({
/** Drop this question and let the tutor move on. Absent where the surface
* cannot start a turn of its own, which hides the control entirely. */
onSkip?: (questionId: string) => void | boolean | Promise<void | boolean>;
+ onChallenge?: (questionId: string) => void | boolean | Promise<void | boolean>;
}) {
const { t } = useTranslation();
const [picked, setPicked] = useState<string>("");
const [skipping, setSkipping] = useState(false);
+ const [sourceUnavailable, setSourceUnavailable] = useState(false);
const [freeText, setFreeText] = useState("");
const [freeSelected, setFreeSelected] = useState(false);
const { sending, failed: sendFailed, submit } = useCardSubmission(onSubmit);
@@ -136,6 +139,7 @@ export const MasteryQuestionCard = memo(function MasteryQuestionCard({
// Skipping settles the card exactly as answering does: the engine closed the
// question, so there is nothing left on it to send.
const settled = answered || skipped === true;
+ const ungraded = grade?.result === "ungraded";
const locked = settled || sending || skipping;
const handleSkip = useCallback(() => {
@@ -148,12 +152,12 @@ export const MasteryQuestionCard = memo(function MasteryQuestionCard({
}, [locked, onSkip, question.questionId]);
const handleSubmit = useCallback(() => {
- if (locked || !answer) return;
+ if (locked || !answer || sourceUnavailable) return;
void submit({
text: answer,
answers: [{ questionId: question.questionId, text: answer }],
});
- }, [answer, locked, submit, question.questionId]);
+ }, [answer, locked, sourceUnavailable, submit, question.questionId]);
// Which answer to treat as the learner's, preferring what the graded record
// committed over what this browser happens to remember.
@@ -168,7 +172,7 @@ export const MasteryQuestionCard = memo(function MasteryQuestionCard({
].filter(Boolean);
// The rule down the left margin is the whole status indicator.
- const rule = grade
+ const rule = ungraded ? "border-dashed border-[var(--border)]" : grade
? grade.isCorrect
? "border-[var(--primary)]"
: "border-[var(--destructive)]"
@@ -180,6 +184,7 @@ export const MasteryQuestionCard = memo(function MasteryQuestionCard({
const optionState = (option: { label: string }): OptionState => {
if (grade) {
+ if (ungraded) return "idle";
if (option.label && option.label === grade.correctLabel) return "correct";
if (option.label && option.label === learnerAnswer && !grade.isCorrect)
return "wrong";
@@ -200,6 +205,17 @@ export const MasteryQuestionCard = memo(function MasteryQuestionCard({
<div className="mt-0.5 font-serif text-[15.5px] font-semibold leading-relaxed tracking-[-0.01em] text-[var(--foreground)]">
<InlineMarkdown content={question.prompt} />
</div>
+ {question.visual && <div className="mt-3 space-y-2 text-xs text-[var(--muted-foreground)]">
+ <p>{grade?.independent === false || question.visual.hintsUsed || question.visual.answerCues !== "none" ? t("Guided visual practice — this does not demonstrate independent mastery.") : t("Visual practice with the original source.")}</p>
+ {sourceUnavailable && <p role="alert">{t("The source image is unavailable. Open the source or ask your tutor to inspect its current version.")}</p>}
+ {question.visual.keyStatus !== "verified" && <p>{t("The reference key is uncertain. This practice will remain ungraded.")}</p>}
+ {question.visual.pixelsInspected === false && <p>{t("Current model input has no verified source pixels. Retrieve the image again with a vision-capable model before grading.")}</p>}
+ {question.visual.sources.map((source, index) => <figure key={`${source.imageUrl}-${index}`}>
+ {/* Authenticated source endpoints need the browser's own cookies. */}
+ <a href={source.imageUrl} target="_blank" rel="noreferrer"><img src={source.imageUrl} alt={t("Original source evidence")} onError={() => setSourceUnavailable(true)} className="max-h-80 max-w-full object-contain" /></a>
+ <figcaption><a className="underline" href={source.url} target="_blank" rel="noreferrer">{source.sourcePath}{source.page ? ` · ${t("Page")} ${source.page}` : ""}</a></figcaption>
+ </figure>)}
+ </div>}
{hasChoices ? (
<div className="mt-2.5 flex flex-col gap-0.5">
@@ -254,14 +270,14 @@ export const MasteryQuestionCard = memo(function MasteryQuestionCard({
<span
className={
"font-semibold " +
- (grade.isCorrect
+ (ungraded ? "text-[var(--muted-foreground)]" : grade.isCorrect
? "text-[var(--primary)]"
: "text-[var(--destructive)]")
}
>
- {grade.isCorrect ? t("Correct") : t("Not quite")}
+ {ungraded ? t("Ungraded — mastery unchanged") : grade.isCorrect ? t("Correct") : t("Not quite")}
</span>
- {!grade.isCorrect && grade.correctLabel ? (
+ {!ungraded && !grade.isCorrect && grade.correctLabel ? (
<span className="text-[var(--foreground)]">
{" · "}
{t("Answer: {{label}}", { label: grade.correctLabel })}
@@ -273,6 +289,7 @@ export const MasteryQuestionCard = memo(function MasteryQuestionCard({
<InlineMarkdown content={grade.explanation} />
</div>
) : null}
+ {onChallenge && <button type="button" className="mt-2 text-xs underline" onClick={() => onChallenge(question.questionId)}>{t("Review or challenge this assessment")}</button>}
</div>
) : null}
@@ -312,7 +329,7 @@ export const MasteryQuestionCard = memo(function MasteryQuestionCard({
<button
type="button"
onClick={handleSubmit}
- disabled={locked || !answer}
+ disabled={locked || !answer || sourceUnavailable}
className="rounded-md bg-[var(--primary)] px-3 py-1.5 text-[12px] font-medium text-[var(--primary-foreground)] transition-opacity hover:opacity-90 disabled:cursor-not-allowed disabled:opacity-40"
>
{t("Submit")}
diff --git a/web/components/space/learning/MasteryStudy.tsx b/web/components/space/learning/MasteryStudy.tsx
index 555f13395..ffdb96c57 100644
--- a/web/components/space/learning/MasteryStudy.tsx
+++ b/web/components/space/learning/MasteryStudy.tsx
@@ -809,6 +809,11 @@ export function MasteryStudy({
onSubmitUserReply={submitUserReply}
onAnswerMasteryQuestion={answerMasteryQuestion}
onSkipMasteryQuestion={skipMasteryQuestion}
+ onChallengeMasteryQuestion={(questionId) => {
+ if (state.isStreaming || sessionLoading || sessionError) return false;
+ sendMessage(t('Please review assessment {{questionId}} against its original source evidence. If it is unsupported, invalidate it and recompute learning state. Do not grant mastery merely because I challenged it.', { questionId }));
+ return true;
+ }}
onPreviewAttachment={handlePreviewMessageAttachment}
onConfirmOutline={confirmResearchOutline}
onLoadMessageTrace={(messageId) =>
diff --git a/web/components/space/learning/ObjectiveDetail.tsx b/web/components/space/learning/ObjectiveDetail.tsx
index c4f14360d..4d5b7cdb2 100644
--- a/web/components/space/learning/ObjectiveDetail.tsx
+++ b/web/components/space/learning/ObjectiveDetail.tsx
@@ -115,6 +115,10 @@ export function ObjectiveDetail({
</Row>
)}
+ {report.explained && <Row label={t("Explained")}>{report.explained.summary}</Row>}
+ {!!report.required_visual_tasks?.length && <Row label={t("Independent visual evidence")}>{report.demonstrated_visual_tasks?.length ?? 0} / {report.required_visual_tasks.length}</Row>}
+ {!!report.required_visual_tasks?.some((task) => !report.demonstrated_visual_tasks?.includes(task)) && <p className="text-xs text-[var(--muted-foreground)]">{t("Independent visual evidence is still required for this objective.")}</p>}
+ {report.attempts.some((attempt) => attempt.independent === false) && <p className="text-xs text-[var(--muted-foreground)]">{t("Assisted practice is recorded separately from independent mastery.")}</p>}
{evidence.length > 0 && (
<div>
<div className="text-xs text-[var(--muted-foreground)]">
@@ -130,6 +134,7 @@ export function ObjectiveDetail({
>
<span>
{t(evidence.result)} · {t(evidence.assessment_type)}
+ {evidence.visual_context?.task && ` · ${t("Visual practice")}${evidence.hints_used ? ` · ${t("Assisted")}` : ""}`}
{evidence.quality !== null &&
` · ${Math.round(evidence.quality * 100)}%`}
</span>
diff --git a/web/features/chat/components/ChatWorkspace.tsx b/web/features/chat/components/ChatWorkspace.tsx
index b63bf33d8..9cae05f61 100644
--- a/web/features/chat/components/ChatWorkspace.tsx
+++ b/web/features/chat/components/ChatWorkspace.tsx
@@ -2585,6 +2585,7 @@ export default function ChatWorkspace() {
onSubmitUserReply={submitUserReply}
onAnswerMasteryQuestion={answerMasteryQuestion}
onSkipMasteryQuestion={skipMasteryQuestion}
+ onChallengeMasteryQuestion={(questionId) => handleSend(t('Please review assessment {{questionId}} against its original source evidence. If it is unsupported, invalidate it and recompute learning state. Do not grant mastery merely because I challenged it.', { questionId }))}
onLoadMessageTrace={(messageId) =>
state.sessionId
? loadMessageTrace(state.sessionId, messageId)
diff --git a/web/features/chat/messages/ChatMessageList.tsx b/web/features/chat/messages/ChatMessageList.tsx
index 4e4454304..88150b741 100644
--- a/web/features/chat/messages/ChatMessageList.tsx
+++ b/web/features/chat/messages/ChatMessageList.tsx
@@ -643,6 +643,7 @@ export const AssistantMessage = memo(function AssistantMessage({
onSubmitUserReply,
onAnswerMasteryQuestion,
onSkipMasteryQuestion,
+ onChallengeMasteryQuestion,
researchRequestSnapshot,
onTraceToggle,
masteryGrades,
@@ -710,6 +711,7 @@ export const AssistantMessage = memo(function AssistantMessage({
onSkipMasteryQuestion?: (
questionId: string,
) => void | boolean | Promise<void | boolean>;
+ onChallengeMasteryQuestion?: (questionId: string) => void | boolean | Promise<void | boolean>;
}) {
const { t } = useTranslation();
const events = useMemo(() => msg.events ?? [], [msg.events]);
@@ -819,6 +821,7 @@ export const AssistantMessage = memo(function AssistantMessage({
: false
}
onSkip={onSkipMasteryQuestion}
+ onChallenge={onChallengeMasteryQuestion}
/>
);
},
@@ -827,6 +830,7 @@ export const AssistantMessage = memo(function AssistantMessage({
masterySkips,
onAnswerMasteryQuestion,
onSkipMasteryQuestion,
+ onChallengeMasteryQuestion,
],
);
// Set by ``request_credential`` when a configuration step needs a secret the
@@ -1923,6 +1927,7 @@ export const ChatMessageList = memo(function ChatMessageList({
onSubmitUserReply,
onAnswerMasteryQuestion,
onSkipMasteryQuestion,
+ onChallengeMasteryQuestion,
showModeBadge = true,
onLoadMessageTrace,
onReleaseMessageTrace,
@@ -1979,6 +1984,7 @@ export const ChatMessageList = memo(function ChatMessageList({
onSkipMasteryQuestion?: (
questionId: string,
) => void | boolean | Promise<void | boolean>;
+ onChallengeMasteryQuestion?: (questionId: string) => void | boolean | Promise<void | boolean>;
/** Names of KBs confirmed to exist. Omitted when the KB list is unavailable. */
availableKbNames?: Set<string>;
/** Qualified KB ref -> display name, from the same catalog the composer
@@ -2314,6 +2320,7 @@ export const ChatMessageList = memo(function ChatMessageList({
onSubmitUserReply={onSubmitUserReply}
onAnswerMasteryQuestion={onAnswerMasteryQuestion}
onSkipMasteryQuestion={onSkipMasteryQuestion}
+ onChallengeMasteryQuestion={onChallengeMasteryQuestion}
researchRequestSnapshot={
pairedUserMessage?.requestSnapshot ?? null
}
diff --git a/web/lib/learning-api.ts b/web/lib/learning-api.ts
index 354b7d353..c0b70e8bf 100644
--- a/web/lib/learning-api.ts
+++ b/web/lib/learning-api.ts
@@ -23,6 +23,7 @@ export interface LearningKnowledgePoint {
id: string;
name: string;
type: string;
+ required_visual_tasks?: string[];
}
export interface LearningModule {
@@ -233,6 +234,7 @@ export async function fetchProgressEvents(
// Mirrors deeptutor/learning/policy.py objective_report.
export interface ObjectiveAttempt {
+ independent?: boolean;
question_id: string;
prompt: string;
answer: string;
@@ -268,6 +270,7 @@ export interface LearningEvidence {
response_time: number | null;
session_id: string;
turn_id: string;
+ visual_context?: { task?: string; sources?: { image_url: string; url: string; source_path: string }[]; hints_used?: number };
}
export interface ObjectiveErrorRecord {
@@ -288,6 +291,9 @@ export interface ObjectiveReport {
gate: "quantitative" | "qualitative";
mastered: boolean;
assessed_mastered: boolean;
+ explained?: { summary: string; timestamp: number };
+ required_visual_tasks?: string[];
+ demonstrated_visual_tasks?: string[];
mastery_source: "system" | "learner" | "";
override_note: string;
mastery: number;
diff --git a/web/lib/mastery-question.ts b/web/lib/mastery-question.ts
index 1b5c1432d..48a2a53c4 100644
--- a/web/lib/mastery-question.ts
+++ b/web/lib/mastery-question.ts
@@ -44,6 +44,7 @@ export interface MasteryQuestion {
attempt: number;
options: MasteryQuestionOption[];
allowFreeText: boolean;
+ visual?: { task: string; answerCues: string; keyStatus: string; hintsUsed: number; pixelsInspected?: boolean; sources: { imageUrl: string; url: string; sourcePath: string; page: number | null }[] };
}
export interface MasteryGradeResult {
@@ -53,6 +54,8 @@ export interface MasteryGradeResult {
correctLabel: string;
correctBody: string;
explanation: string;
+ result?: string;
+ independent?: boolean;
}
function asRecord(value: unknown): Record<string, unknown> | null {
@@ -82,6 +85,10 @@ function parseQuestion(value: unknown): MasteryQuestion | null {
.filter((option) => option.label)
: [];
const attempt = Number(raw.attempt);
+ const visual = asRecord(raw.visual);
+ const visualSources = Array.isArray(visual?.sources) ? visual.sources.map(asRecord).filter((ref): ref is Record<string, unknown> => ref !== null)
+ .filter((ref) => text(ref.image_url).startsWith("/api/knowledge-bases/") && text(ref.url).startsWith("/api/knowledge-bases/"))
+ .map((ref) => ({ imageUrl: text(ref.image_url), url: text(ref.url), sourcePath: text(ref.source_path), page: typeof ref.page === "number" ? ref.page : null })) : [];
return {
questionId,
prompt: text(raw.prompt),
@@ -91,6 +98,7 @@ function parseQuestion(value: unknown): MasteryQuestion | null {
attempt: Number.isFinite(attempt) && attempt > 0 ? Math.floor(attempt) : 1,
options,
allowFreeText: raw.allow_free_text !== false,
+ ...(visual ? { visual: { task: text(visual.task), answerCues: text(visual.answer_cues), keyStatus: text(visual.key_status), hintsUsed: Number(visual.hints_used) || 0, pixelsInspected: visual.pixels_inspected === true, sources: visualSources } } : {}),
};
}
@@ -106,6 +114,8 @@ function parseGrade(value: unknown): MasteryGradeResult | null {
correctLabel: text(raw.correct_label),
correctBody: text(raw.correct_body),
explanation: text(raw.explanation),
+ ...(text(raw.result) ? { result: text(raw.result) } : {}),
+ ...(typeof raw.independent === "boolean" ? { independent: raw.independent } : {}),
};
}
diff --git a/web/locales/de/app.json b/web/locales/de/app.json
index 51387ef5a..410c675b7 100644
--- a/web/locales/de/app.json
+++ b/web/locales/de/app.json
@@ -5472,5 +5472,20 @@
"visualCoverage.asset_read_failed": "Abbildung konnte nicht gelesen werden; prüfe die Originalseite",
"visualCoverage.table_structure_requires_source_verification": "Tabellenstruktur muss anhand der Originalseite geprüft werden",
"visualCoverage.referenced_image_not_retained": "Referenzierte Abbildung wurde nicht gespeichert; prüfe die Quellseite",
- "visualCoverage.extraction_coverage_unknown": "Extraktionsabdeckung ist noch ungeprüft"
+ "visualCoverage.extraction_coverage_unknown": "Extraktionsabdeckung ist noch ungeprüft",
+ "Guided visual practice — this does not demonstrate independent mastery.": "Angeleitete visuelle Übung – kein Nachweis eigenständiger Beherrschung.",
+ "Visual practice with the original source.": "Visuelle Übung mit der Originalquelle.",
+ "The reference key is uncertain. This practice will remain ungraded.": "Der Lösungsschlüssel ist unsicher. Diese Übung bleibt unbewertet.",
+ "Original source evidence": "Originaler Quellenbeleg",
+ "Ungraded — mastery unchanged": "Unbewertet – Lernstand unverändert",
+ "Review or challenge this assessment": "Bewertung prüfen oder anfechten",
+ "The source image is unavailable. Open the source or ask your tutor to inspect its current version.": "Die Quellabbildung ist nicht verfügbar. Öffne die Quelle oder bitte deinen Tutor, die aktuelle Version zu prüfen.",
+ "Independent visual evidence": "Eigenständiger visueller Nachweis",
+ "Explained": "Erklärt",
+ "Visual practice": "Visuelle Übung",
+ "Assisted": "Mit Hilfe",
+ "Independent visual evidence is still required for this objective.": "Für dieses Lernziel ist noch ein eigenständiger visueller Nachweis erforderlich.",
+ "Assisted practice is recorded separately from independent mastery.": "Angeleitete Übung wird getrennt von eigenständiger Beherrschung erfasst.",
+ "Please review assessment {{questionId}} against its original source evidence. If it is unsupported, invalidate it and recompute learning state. Do not grant mastery merely because I challenged it.": "Bitte prüfe Bewertung {{questionId}} anhand der Originalquelle. Wenn sie unbelegt ist, annulliere sie und berechne den Lernstand neu. Vergib keinen Beherrschungsnachweis allein wegen meines Einspruchs.",
+ "Current model input has no verified source pixels. Retrieve the image again with a vision-capable model before grading.": "In der aktuellen Modelleingabe sind keine verifizierten Quellbilder enthalten. Lade das Bild vor der Bewertung mit einem visuellen Modell erneut."
}
diff --git a/web/locales/en/app.json b/web/locales/en/app.json
index fd7cc4cb6..869989547 100644
--- a/web/locales/en/app.json
+++ b/web/locales/en/app.json
@@ -5472,5 +5472,20 @@
"visualCoverage.asset_read_failed": "Image read failed; inspect the original page",
"visualCoverage.table_structure_requires_source_verification": "Verify table structure against the original page",
"visualCoverage.referenced_image_not_retained": "Referenced image not retained; inspect its source page",
- "visualCoverage.extraction_coverage_unknown": "Extraction coverage has not been verified"
+ "visualCoverage.extraction_coverage_unknown": "Extraction coverage has not been verified",
+ "Guided visual practice — this does not demonstrate independent mastery.": "Guided visual practice — this does not demonstrate independent mastery.",
+ "Visual practice with the original source.": "Visual practice with the original source.",
+ "The reference key is uncertain. This practice will remain ungraded.": "The reference key is uncertain. This practice will remain ungraded.",
+ "Original source evidence": "Original source evidence",
+ "Ungraded — mastery unchanged": "Ungraded — mastery unchanged",
+ "Review or challenge this assessment": "Review or challenge this assessment",
+ "The source image is unavailable. Open the source or ask your tutor to inspect its current version.": "The source image is unavailable. Open the source or ask your tutor to inspect its current version.",
+ "Independent visual evidence": "Independent visual evidence",
+ "Explained": "Explained",
+ "Visual practice": "Visual practice",
+ "Assisted": "Assisted",
+ "Independent visual evidence is still required for this objective.": "Independent visual evidence is still required for this objective.",
+ "Assisted practice is recorded separately from independent mastery.": "Assisted practice is recorded separately from independent mastery.",
+ "Please review assessment {{questionId}} against its original source evidence. If it is unsupported, invalidate it and recompute learning state. Do not grant mastery merely because I challenged it.": "Please review assessment {{questionId}} against its original source evidence. If it is unsupported, invalidate it and recompute learning state. Do not grant mastery merely because I challenged it.",
+ "Current model input has no verified source pixels. Retrieve the image again with a vision-capable model before grading.": "Current model input has no verified source pixels. Retrieve the image again with a vision-capable model before grading."
}
diff --git a/web/locales/pl/app.json b/web/locales/pl/app.json
index d8001a5ef..f89767b9f 100644
--- a/web/locales/pl/app.json
+++ b/web/locales/pl/app.json
@@ -5488,5 +5488,20 @@
"visualCoverage.asset_read_failed": "Nie udało się odczytać obrazu; sprawdź oryginalną stronę",
"visualCoverage.table_structure_requires_source_verification": "Struktura tabeli wymaga sprawdzenia na oryginalnej stronie",
"visualCoverage.referenced_image_not_retained": "Nie zachowano wskazanego obrazu; sprawdź stronę źródłową",
- "visualCoverage.extraction_coverage_unknown": "Pokrycie wyodrębniania nie zostało zweryfikowane"
+ "visualCoverage.extraction_coverage_unknown": "Pokrycie wyodrębniania nie zostało zweryfikowane",
+ "Guided visual practice — this does not demonstrate independent mastery.": "Ćwiczenie wizualne ze wsparciem — nie dowodzi samodzielnego opanowania.",
+ "Visual practice with the original source.": "Ćwiczenie wizualne z oryginalnym źródłem.",
+ "The reference key is uncertain. This practice will remain ungraded.": "Klucz odpowiedzi jest niepewny. Ćwiczenie pozostanie bez oceny.",
+ "Original source evidence": "Oryginalny materiał źródłowy",
+ "Ungraded — mastery unchanged": "Bez oceny — poziom opanowania bez zmian",
+ "Review or challenge this assessment": "Sprawdź lub zakwestionuj tę ocenę",
+ "The source image is unavailable. Open the source or ask your tutor to inspect its current version.": "Obraz źródłowy jest niedostępny. Otwórz źródło lub poproś tutora o sprawdzenie bieżącej wersji.",
+ "Independent visual evidence": "Samodzielny dowód rozumienia obrazu",
+ "Explained": "Wyjaśnione",
+ "Visual practice": "Ćwiczenie wizualne",
+ "Assisted": "Ze wsparciem",
+ "Independent visual evidence is still required for this objective.": "Ten cel nadal wymaga samodzielnego dowodu rozumienia obrazu.",
+ "Assisted practice is recorded separately from independent mastery.": "Ćwiczenia ze wsparciem są zapisywane oddzielnie od samodzielnego opanowania.",
+ "Please review assessment {{questionId}} against its original source evidence. If it is unsupported, invalidate it and recompute learning state. Do not grant mastery merely because I challenged it.": "Sprawdź ocenę {{questionId}} na podstawie oryginalnego źródła. Jeśli jest nieuzasadniona, unieważnij ją i przelicz stan nauki. Samo zakwestionowanie nie dowodzi opanowania.",
+ "Current model input has no verified source pixels. Retrieve the image again with a vision-capable model before grading.": "Bieżące dane wejściowe modelu nie zawierają zweryfikowanego obrazu źródłowego. Przed oceną pobierz go ponownie modelem obsługującym obrazy."
}
diff --git a/web/locales/zh/app.json b/web/locales/zh/app.json
index ba624a7df..a0232ad34 100644
--- a/web/locales/zh/app.json
+++ b/web/locales/zh/app.json
@@ -5472,5 +5472,20 @@
"visualCoverage.asset_read_failed": "读取图像失败;请检查原始页面",
"visualCoverage.table_structure_requires_source_verification": "表格结构需要对照原始页面核验",
"visualCoverage.referenced_image_not_retained": "引用的图像未保留;请检查该来源页面",
- "visualCoverage.extraction_coverage_unknown": "提取覆盖情况尚未核验"
+ "visualCoverage.extraction_coverage_unknown": "提取覆盖情况尚未核验",
+ "Guided visual practice — this does not demonstrate independent mastery.": "辅助视觉练习,不代表已独立掌握。",
+ "Visual practice with the original source.": "使用原始来源的视觉练习。",
+ "The reference key is uncertain. This practice will remain ungraded.": "参考答案尚不确定,本次练习将保留为未评分。",
+ "Original source evidence": "原始来源证据",
+ "Ungraded — mastery unchanged": "未评分,掌握状态不变",
+ "Review or challenge this assessment": "复核或申诉本次判定",
+ "The source image is unavailable. Open the source or ask your tutor to inspect its current version.": "来源图像不可用。请打开原始来源,或让导师检查当前版本。",
+ "Independent visual evidence": "独立视觉证据",
+ "Explained": "已讲解",
+ "Visual practice": "视觉练习",
+ "Assisted": "有辅助",
+ "Independent visual evidence is still required for this objective.": "此目标仍需要独立视觉证据。",
+ "Assisted practice is recorded separately from independent mastery.": "辅助练习与独立掌握分开记录。",
+ "Please review assessment {{questionId}} against its original source evidence. If it is unsupported, invalidate it and recompute learning state. Do not grant mastery merely because I challenged it.": "请对照原始来源复核判定 {{questionId}}。如果没有可靠依据,请作废并重算学习状态;不能只因我提出申诉就授予掌握状态。",
+ "Current model input has no verified source pixels. Retrieve the image again with a vision-capable model before grading.": "当前模型输入没有已核验的来源像素。请使用支持视觉的模型重新读取原图,再进行评分。"
}
diff --git a/web/tests/mastery-question-card.spec.tsx b/web/tests/mastery-question-card.spec.tsx
index 48a89c2dd..29390f68b 100644
--- a/web/tests/mastery-question-card.spec.tsx
+++ b/web/tests/mastery-question-card.spec.tsx
@@ -1,4 +1,4 @@
-import { render, screen } from "@testing-library/react";
+import { fireEvent, render, screen } from "@testing-library/react";
import userEvent from "@testing-library/user-event";
import { describe, expect, it, vi } from "vitest";
@@ -21,6 +21,23 @@ const question = (overrides: Partial<MasteryQuestion> = {}): MasteryQuestion =>
});
describe("MasteryQuestionCard", () => {
+ it("shows original evidence and assisted meaning without exposing a reference key", () => {
+ render(<MasteryQuestionCard question={question({ visual: { task: "identification", answerCues: "visible", keyStatus: "unverified", hintsUsed: 1, sources: [{ imageUrl: "/api/knowledge-bases/kb/visual-assets/known", url: "/api/knowledge-bases/kb/files/book.pdf#page=4", sourcePath: "book.pdf", page: 4 }] } })} grade={null} answered={false} submittedAnswer="" onSubmit={() => true} />);
+ expect(screen.getByRole("img", { name: "Original source evidence" })).toHaveAttribute("src", "/api/knowledge-bases/kb/visual-assets/known");
+ expect(screen.getByText("Guided visual practice — this does not demonstrate independent mastery.")).toBeInTheDocument();
+ expect(screen.getByText("The reference key is uncertain. This practice will remain ungraded.")).toBeInTheDocument();
+ fireEvent.error(screen.getByRole("img"));
+ expect(screen.getByRole("alert")).toHaveTextContent("The source image is unavailable");
+ });
+ it("keeps uncertain grades neutral and sends a source-review challenge", async () => {
+ const user = userEvent.setup();
+ const onChallenge = vi.fn(() => true);
+ render(<MasteryQuestionCard question={question()} grade={{ questionId: "q-free", isCorrect: false, result: "ungraded", learnerAnswer: "ambiguous", correctLabel: "", correctBody: "", explanation: "Source evidence needs clarification." }} answered submittedAnswer="" onSubmit={() => true} onChallenge={onChallenge} />);
+ expect(screen.getByText("Ungraded — mastery unchanged")).toBeInTheDocument();
+ expect(screen.queryByText("Not quite")).toBeNull();
+ await user.click(screen.getByRole("button", { name: "Review or challenge this assessment" }));
+ expect(onChallenge).toHaveBeenCalledWith("q-free");
+ });
it("enables submit from a free-text-only question", async () => {
const user = userEvent.setup();
const onSubmit = vi.fn(() => true);