Skip to content

DeepTutor: Lifelong Personalized Tutoring. https://deeptutor.info/.

fix(mastery): retain incremental grouped replies and partial feedback

yepyhun9cfa5cb8fd46
diff --git a/deeptutor/capabilities/mastery/loop.py b/deeptutor/capabilities/mastery/loop.py
index 755de52e8..ade10fb2c 100644
--- a/deeptutor/capabilities/mastery/loop.py
+++ b/deeptutor/capabilities/mastery/loop.py
@@ -561,8 +561,19 @@ class MasteryLoopCapability:
for item in results
if isinstance(item, dict)
]
+ complete = result.get("complete", grade.get("complete", True)) is not False
+ status = (
+ "The question group was already graded by the engine. "
+ if complete else
+ "Only the submitted group replies were graded by the engine. "
+ "These children remain unanswered: "
+ + ", ".join(result.get("remaining_question_ids", [])) + ". "
+ "Preserve unanswered children and continue at the requested pace. "
+ "Explain reported gaps, but never infer confusion from an unanswered "
+ "item or invent its reply. "
+ )
return (
- "[Mastery] The question group was already graded by the engine. "
+ "[Mastery] " + status +
"Do not grade it again or repeat its answer keys. Each result belongs "
"only to its own objective; the parent group gives no extra credit. "
"Address the specific gap, then use the fresh status for the next step.\n"
diff --git a/deeptutor/capabilities/mastery/prompts/en/mastery_loop.yaml b/deeptutor/capabilities/mastery/prompts/en/mastery_loop.yaml
index a7521f3fb..dbefca5df 100644
--- a/deeptutor/capabilities/mastery/prompts/en/mastery_loop.yaml
+++ b/deeptutor/capabilities/mastery/prompts/en/mastery_loop.yaml
@@ -86,6 +86,7 @@ session:
# Teaching strategy. Kept separate from the loop protocol above: that one says
# how a round runs, this one says what to do about the objective in front of you.
playbook: |-
+ **Incremental group replies:** A learner may answer only some children. Pass those real answers to `mastery_grade`, preserve unanswered siblings, and read `complete` / `remaining_question_ids` before claiming the whole group is finished. Do not ask for already graded answers again.
**Registered grouped practice:** When the learner requests questions together, use `mastery_quiz.questions` for a bounded group of two to four separately answerable memory/procedure questions. Each item must name a real objective and carry its own supported question and private answer contract; do not concatenate expected answers or pose official numbered assessments in prose. A group is one active interaction with separate results, not extra mastery credit. If an unrelated older question is still pending, explain that briefly. Replace it only for an explicit current learner request, using `pending_action: replace`, the fresh `replace_question_id`, and `replacement_request` as a literal quote from that request. Otherwise retain it. Never infer abandonment from silence or a pacing preference. Do not call the single-question tool twice and promise two cards. Unsupported objective types still use their native qualitative assessment path.
**Concise but explanatory:** Do not replace teaching with a list of names, priority labels or compressed definitions. At the learner's requested level, explain the necessary meaning and one useful connection or concrete example before assessing it. After feedback that an explanation was too terse, supply the missing link rather than another equally compressed formulation. Split useful teaching into connected steps when needed; there is no fixed word quota or compulsory question after each step.
**Source-grounded visual study:** Honor explanation, practice and review as explicit learner choices; explain untaught prerequisites at the requested level before assessment. Use `mastery_note_explained` after an explanation; this records coverage without mastery credit. An outline is never evidence of teaching. A concise explanation-only turn may end without a quiz; explicit deferral uses `mastery_defer_objective` and does not claim mastery.
diff --git a/deeptutor/capabilities/mastery/prompts/zh/mastery_loop.yaml b/deeptutor/capabilities/mastery/prompts/zh/mastery_loop.yaml
index d0e7397d6..b7995cd03 100644
--- a/deeptutor/capabilities/mastery/prompts/zh/mastery_loop.yaml
+++ b/deeptutor/capabilities/mastery/prompts/zh/mastery_loop.yaml
@@ -83,6 +83,7 @@ session:
# 教学策略。与 loop 协议分开:上面说的是「一轮怎么走」,这里说的是「针对这个知识点做什么」。
playbook: |-
+ **分次回答组合题:** 学习者可以只回答部分子题。把这些实际回答传给 `mastery_grade`,保留尚未回答的子题,并在宣称整组已完成前检查 `complete` / `remaining_question_ids`。不要重复索取已经评分的回答。
**已登记的组合练习**:学习者要求一起回答问题时,用 `mastery_quiz.questions` 建立二至四题的有界组合;每题独立作答,只用于 memory/procedure 知识点。每题必须指向真实知识点并保存自己的、有来源支持的题目与私密答案契约;不能把答案拼接后一起模糊判分,也不能把正式编号考核写在正文里。组合只有一个活跃交互,但结果分别归属各题,不额外授予掌握分。若仍有不相关的旧待答题,简短说明;只有学习者当前明确要求替换时,才用 `pending_action: replace`、最新 `replace_question_id` 和当前请求的逐字引用 `replacement_request` 替换,否则保留。沉默或节奏偏好不是放弃旧题的证据。不要连续调用单题工具后承诺两张卡片;不支持的知识点类型仍走原有定性考核流程。
**简洁但讲清楚**:不要用名称列表、重要性标签或压缩定义代替教学。按学习者需要的深度,先讲必要的含义以及一个有用联系或具体例子,再考核。学习者说解释太简略时,补上缺失的联系,不要换成同样简略的说法。需要时把教学拆成连贯小步;没有固定字数配额,也不要求每一步之后都出题。
**来源扎实的视觉学习:** 尊重学习者明确选择的讲解、练习或复习,按要求的层次先解释必要前置知识。讲解后调用 `mastery_note_explained`,只记录已讲解覆盖,不授予掌握分;生成提纲不等于教学。仅需简洁讲解时可以不出题;明确暂缓用 `mastery_defer_objective`,不能假称已掌握。
diff --git a/deeptutor/capabilities/mastery/tools.py b/deeptutor/capabilities/mastery/tools.py
index 7fec6fc57..654ea4c3f 100644
--- a/deeptutor/capabilities/mastery/tools.py
+++ b/deeptutor/capabilities/mastery/tools.py
@@ -798,7 +798,22 @@ class MasteryStatusTool(BaseTool):
"question_id": interaction.interaction_id,
"status": interaction.status.value,
}
- if interaction.status == InteractionStatus.ANSWERED:
+ if interaction.question.questions:
+ graded_ids = {
+ result["question_id"] for result in interaction.result.get("results", [])
+ }
+ pending_interaction["answers"] = dict(interaction.answers)
+ pending_interaction["remaining_question_ids"] = [
+ q.question_id for q in interaction.question.questions
+ if q.question_id not in graded_ids
+ ]
+ pending_interaction["instruction"] = (
+ "Keep committed child replies verbatim. Grade only actual replies; "
+ "unanswered children remain open and earn no score. Continue at the requested "
+ "pace; explain reported gaps, but never infer confusion from an unanswered "
+ "item or invent a missing reply."
+ )
+ elif interaction.status == InteractionStatus.ANSWERED:
# The answer is learner-authored state, not the hidden answer
# key. Returning it lets a restart grade rather than ask twice.
pending_interaction["learner_answer"] = interaction.user_answer
@@ -1330,7 +1345,7 @@ class MasteryGradeTool(BaseTool):
name="answers",
type="object",
required=False,
- description="For a group only: exact child question_id to learner answer strings. Use committed replies verbatim; never one concatenated answer.",
+ description="For a group only: one or more exact child question_id to actual learner answer strings. Partial replies are saved and graded independently; unanswered children stay open. Use committed replies verbatim; never invent missing replies or concatenate answers.",
),
ToolParameter(
name="question_id",
@@ -1568,9 +1583,9 @@ class MasteryGradeTool(BaseTool):
return ToolResult(content=str(exc), success=False)
group_result = dict(interaction.result)
group_result["results"] = [dict(result) for result in group_result["results"]]
- for leaf, result in zip(
- interaction.question.questions, group_result["results"], strict=True
- ):
+ leaves = {leaf.question_id: leaf for leaf in interaction.question.questions}
+ for result in group_result["results"]:
+ leaf = leaves[result["question_id"]]
kp, _, _ = find_knowledge_point(progress, leaf.knowledge_point_id)
evidence = next(
(
@@ -1612,11 +1627,13 @@ class MasteryGradeTool(BaseTool):
"question_id": pending.question_id,
"question_type": "group",
"is_correct": group_result["is_correct"],
+ "complete": group_result.get("complete", True),
+ "remaining_question_ids": group_result.get("remaining_question_ids", []),
"replayed": replayed,
"path_revision": progress.version,
"result": group_result,
"next": next_objective(progress).to_dict(),
- "instruction": "Give concise, explanatory feedback per question. Mixed results remain separate; the parent grants no extra mastery. Keep qualitative assessment gates and adapt subsequent teaching to the learner's needs.",
+ "instruction": "Give explanatory feedback only for answered questions. Preserve independent results; the parent grants no extra mastery. If complete=false, unanswered children remain open. Continue at the requested pace and explain reported gaps; do not infer confusion from an unanswered item, invent replies or regrade committed answers. Keep qualitative gates and adapt teaching to the learner's needs.",
},
meta_key="mastery_grade",
)
diff --git a/deeptutor/core/turn_request.py b/deeptutor/core/turn_request.py
index a9cc807f5..d5f12c40a 100644
--- a/deeptutor/core/turn_request.py
+++ b/deeptutor/core/turn_request.py
@@ -91,7 +91,7 @@ class MasteryCardAnswer(BaseModel):
question_id: str = Field(min_length=1)
text: str = Field(min_length=1)
- answers: dict[str, StrictStr] | None = Field(default=None, min_length=2, max_length=4)
+ answers: dict[str, StrictStr] | None = Field(default=None, min_length=1, max_length=4)
@field_validator("answers")
@classmethod
diff --git a/deeptutor/learning/service.py b/deeptutor/learning/service.py
index 5ffd57eda..e15c5b1a6 100644
--- a/deeptutor/learning/service.py
+++ b/deeptutor/learning/service.py
@@ -831,16 +831,26 @@ class LearningService:
if interaction_id and interaction.interaction_id != interaction_id:
raise StaleInteractionError(interaction_id, interaction.interaction_id)
if interaction.question.questions:
- submitted = self._validate_group_answers(interaction.question, answers)
- if interaction.status in {InteractionStatus.ANSWERED, InteractionStatus.GRADED}:
- if interaction.answers != submitted:
- raise MasteryInteractionError(
- "This group already has a different committed answer"
- )
- return interaction
if interaction.status == InteractionStatus.ABANDONED:
raise NoPendingInteractionError("The question was abandoned")
+ submitted = self._merge_group_answers(interaction, answers)
+ if submitted == interaction.answers:
+ return interaction
interaction.answers = submitted
+ complete = len(submitted) == len(interaction.question.questions)
+ interaction.status = (
+ InteractionStatus.ANSWERED if complete else InteractionStatus.AWAITING_INPUT
+ )
+ interaction.session_id = session_id or interaction.session_id
+ interaction.turn_id = turn_id or interaction.turn_id
+ tx.put_interaction(interaction)
+ tx.emit(
+ "interaction.answered" if complete else "interaction.answers_recorded",
+ {"interaction_id": interaction.interaction_id, "complete": complete},
+ session_id=interaction.session_id,
+ turn_id=interaction.turn_id,
+ )
+ return interaction
if interaction.status in {
InteractionStatus.REGISTERED,
InteractionStatus.AWAITING_INPUT,
@@ -926,11 +936,7 @@ class LearningService:
raise StaleInteractionError(question_id, interaction.interaction_id)
if interaction.status == InteractionStatus.GRADED:
if interaction.question.questions and answers is not None:
- submitted = self._validate_group_answers(interaction.question, answers)
- if submitted != interaction.answers:
- raise MasteryInteractionError(
- "This group already has a different committed answer"
- )
+ self._merge_group_answers(interaction, answers)
return interaction, True
if interaction.status == InteractionStatus.ABANDONED:
raise NoPendingInteractionError("The question was abandoned")
@@ -943,7 +949,7 @@ class LearningService:
scheduler=scheduler,
session_id=session_id,
turn_id=turn_id,
- ), False
+ )
raw_answer, is_correct, visual_result, visual_diagnosis = self._evaluate_question(
tx,
@@ -1039,36 +1045,55 @@ class LearningService:
ids = {question.question_id for question in pending.questions}
if (
not isinstance(answers, dict)
- or set(answers) != ids
+ or not answers
+ or not set(answers).issubset(ids)
or any(
not isinstance(value, str) or not value.strip() or len(value) > 10000
for value in answers.values()
)
):
raise MasteryInteractionError(
- "Answer every group question once using its exact question_id"
+ "Supply at least one actual answer using its exact group question_id"
)
from deeptutor.learning.pending import resolve_choice_submission
for question in pending.questions:
- if question.question_type == "choice" and not resolve_choice_submission(
- answers[question.question_id], question.choice_map
+ if (
+ question.question_id in answers
+ and question.question_type == "choice"
+ and not resolve_choice_submission(answers[question.question_id], question.choice_map)
):
raise MasteryInteractionError(
"Choose one unambiguous option for each choice question"
)
return {key: value.strip() for key, value in answers.items()}
+ def _merge_group_answers(self, interaction: MasteryInteraction, answers: Any) -> dict[str, str]:
+ submitted = self._validate_group_answers(
+ interaction.question, interaction.answers if answers is None else answers
+ )
+ if any(
+ key in interaction.answers and interaction.answers[key] != value
+ for key, value in submitted.items()
+ ):
+ raise MasteryInteractionError("This group already has a different committed answer")
+ return {**interaction.answers, **submitted}
+
@staticmethod
- def _group_result(question_id: str, results: list[dict]) -> dict:
+ def _group_result(pending: PendingQuestion, results: list[dict]) -> dict:
values = [result["is_correct"] for result in results]
+ graded_ids = {result["question_id"] for result in results}
+ remaining = [q.question_id for q in pending.questions if q.question_id not in graded_ids]
+ complete = not remaining
return {
- "question_id": question_id,
+ "question_id": pending.question_id,
"question_type": "group",
"results": results,
- "is_correct": None if None in values else all(values),
+ "complete": complete,
+ "remaining_question_ids": remaining,
+ "is_correct": None if not complete or None in values else all(values),
"result": "ungraded"
- if all(value is None for value in values)
+ if not complete or all(value is None for value in values)
else "mixed"
if len(set(values)) > 1
else "correct"
@@ -1085,23 +1110,35 @@ class LearningService:
scheduler=None,
session_id="",
turn_id="",
- ) -> MasteryInteraction:
- """One transaction, native grading per leaf; a parent earns no credit."""
+ ) -> tuple[MasteryInteraction, bool]:
+ """Commit new replies once; unanswered siblings remain awaiting input."""
pending = interaction.question
- submitted = self._validate_group_answers(
- pending, interaction.answers if answers is None else answers
- )
- if interaction.status == InteractionStatus.ANSWERED and submitted != interaction.answers:
- raise MasteryInteractionError(
- "Grade the group's committed answers, not a substituted answer"
- )
+ submitted = self._merge_group_answers(interaction, answers)
known_ids = {kp.id for module in tx.progress.modules for kp in module.knowledge_points}
if any(q.knowledge_point_id not in known_ids for q in pending.questions):
raise MasteryInteractionError("An objective changed; refresh the question group")
from deeptutor.learning.question_card import build_grade_result
- results = []
+ leaves = []
for question in pending.questions:
+ if question.question_id not in submitted:
+ continue
+ stored = tx.get_interaction(question.question_id)
+ if stored is not None and (
+ stored.result.get("group_question_id") != pending.question_id
+ or stored.question.knowledge_point_id != question.knowledge_point_id
+ or stored.user_answer != submitted[question.question_id]
+ ):
+ raise MasteryInteractionError("This child belongs to another interaction or reply")
+ leaves.append((question, stored))
+
+ results = []
+ new_results = False
+ for question, stored in leaves:
+ if stored is not None and stored.status == InteractionStatus.GRADED:
+ results.append(self._group_leaf_result(stored))
+ continue
+ new_results = True
leaf = MasteryInteraction(
interaction_id=question.question_id,
path_id=interaction.path_id,
@@ -1163,20 +1200,26 @@ class LearningService:
session_id=session_id,
turn_id=turn_id,
)
- interaction.status = InteractionStatus.GRADED
+ if not new_results:
+ return interaction, True
+ interaction.result = self._group_result(pending, results)
+ complete = interaction.result["complete"]
+ interaction.status = (
+ InteractionStatus.GRADED if complete else InteractionStatus.AWAITING_INPUT
+ )
interaction.answers = submitted
interaction.session_id = session_id or interaction.session_id
interaction.turn_id = turn_id or interaction.turn_id
- interaction.result = self._group_result(pending.question_id, results)
- tx.progress.pending_question = None
+ if complete:
+ tx.progress.pending_question = None
tx.put_interaction(interaction)
tx.emit(
- "interaction.graded",
+ "interaction.graded" if complete else "interaction.partially_graded",
dict(interaction.result),
session_id=interaction.session_id,
turn_id=interaction.turn_id,
)
- return interaction
+ return interaction, False
def _evaluate_question(
self,
@@ -1973,42 +2016,54 @@ class LearningService:
progress, details = self._store.mutate(book_id, repair)
return progress, details
+ @staticmethod
+ def _group_leaf_result(stored: MasteryInteraction) -> dict:
+ """Project a committed leaf, including later repairs, without regrading."""
+ from deeptutor.learning.question_card import build_grade_result
+
+ correct = None if stored.result.get("voided") else stored.result.get("is_correct")
+ result = build_grade_result(
+ question_id=stored.interaction_id,
+ is_correct=bool(correct),
+ learner_answer=stored.user_answer,
+ correct_label=stored.question.expected_answer if correct is not None else "",
+ choice_options=stored.question.choice_map,
+ explanation=stored.question.explanation
+ if correct is not None
+ else str(stored.result.get("reason") or stored.result.get("diagnosis") or ""),
+ result="ungraded" if correct is None else "",
+ independent=stored.result.get("independent")
+ if stored.question.visual_context
+ else None,
+ )
+ result.update(
+ {"is_correct": correct, "knowledge_point_id": stored.question.knowledge_point_id}
+ )
+ if stored.result.get("diagnosis"):
+ result["diagnosis"] = stored.result["diagnosis"]
+ return result
+
def _refresh_group_result(self, tx, leaf: MasteryInteraction) -> None:
- """Keep the parent presentation consistent after a native leaf repair."""
+ """Keep a complete or partial parent consistent after a native leaf repair."""
parent = tx.get_interaction(leaf.result["group_question_id"])
if parent is None:
return
- from deeptutor.learning.question_card import build_grade_result
-
results = []
questions = []
for question in parent.question.questions:
stored = tx.get_interaction(question.question_id)
- if stored is None:
- return
+ if stored is None or stored.result.get("group_question_id") != parent.interaction_id:
+ questions.append(question)
+ continue
questions.append(stored.question)
- key = stored.question.expected_answer
- correct = None if stored.result.get("voided") else stored.result.get("is_correct")
- result = build_grade_result(
- question_id=question.question_id,
- is_correct=bool(correct),
- learner_answer=stored.user_answer,
- correct_label=key if correct is not None else "",
- choice_options=stored.question.choice_map,
- explanation=stored.question.explanation
- if correct is not None
- else str(stored.result.get("reason") or ""),
- result="ungraded" if correct is None else "",
- independent=stored.result.get("independent")
- if stored.question.visual_context
- else None,
- )
- result.update(
- {"is_correct": correct, "knowledge_point_id": question.knowledge_point_id}
- )
- results.append(result)
- parent.result = self._group_result(parent.interaction_id, results)
+ results.append(self._group_leaf_result(stored))
parent.question.questions = questions
+ parent.result = self._group_result(parent.question, results)
+ if (
+ tx.progress.pending_question is not None
+ and tx.progress.pending_question.question_id == parent.interaction_id
+ ):
+ tx.progress.pending_question = parent.question
tx.put_interaction(parent)
def defer_objective(
diff --git a/tests/capabilities/test_mastery_question_groups.py b/tests/capabilities/test_mastery_question_groups.py
index 8ae3b8cff..eb3359e4e 100644
--- a/tests/capabilities/test_mastery_question_groups.py
+++ b/tests/capabilities/test_mastery_question_groups.py
@@ -94,9 +94,9 @@ def test_independent_mixed_results_and_duplicate_credit(service):
@pytest.mark.parametrize(
"answers",
- [{"q1": "alpha"}, {"q1": "alpha", "q2": ""}, {"q1": "alpha", "q2": "beta", "other": "x"}],
+ [{}, {"q1": "alpha", "q2": ""}, {"q1": "alpha", "q2": "beta", "other": "x"}],
)
-def test_incomplete_or_unknown_answers_do_not_mutate(service, answers):
+def test_empty_or_unknown_answers_do_not_mutate(service, answers):
service.register_question("p", group())
before = service._store.load("p").model_dump()
with pytest.raises(MasteryInteractionError):
@@ -104,6 +104,131 @@ def test_incomplete_or_unknown_answers_do_not_mutate(service, answers):
assert service._store.load("p").model_dump() == before
+def test_partial_answers_are_retained_and_graded_once_after_reload(service):
+ service.register_question("p", group())
+ recorded = service.record_question_answer(
+ "p", "second only", interaction_id="g", answers={"q2": "beta"}
+ )
+ assert recorded.status == InteractionStatus.AWAITING_INPUT
+ assert recorded.answers == {"q2": "beta"}
+ reloaded = LearningService(LearningStore(service._store._root))
+ progress, partial, replayed = reloaded.grade_interaction("p", answer="", question_id="g")
+ assert not replayed
+ assert partial.status == InteractionStatus.AWAITING_INPUT
+ assert partial.result["complete"] is False
+ assert partial.result["remaining_question_ids"] == ["q1"]
+ assert partial.result["is_correct"] is None
+ assert [result["question_id"] for result in partial.result["results"]] == ["q2"]
+ assert [attempt.question_id for attempt in progress.quiz_attempts] == ["q2"]
+ assert progress.pending_question.question_id == "g"
+ replay, _, replayed = reloaded.grade_interaction(
+ "p", answer="", question_id="g", answers={"q2": "beta"}
+ )
+ assert replayed and len(replay.quiz_attempts) == 1
+ final, completed, replayed = reloaded.grade_interaction(
+ "p", answer="", question_id="g", answers={"q1": "alpha"}
+ )
+ assert not replayed
+ assert completed.status == InteractionStatus.GRADED
+ assert completed.answers == {"q1": "alpha", "q2": "beta"}
+ assert completed.result["complete"] is True
+ assert completed.result["remaining_question_ids"] == []
+ assert [r["question_id"] for r in completed.result["results"]] == ["q1", "q2"]
+ assert len(final.quiz_attempts) == len(final.learning_evidence) == 2
+ assert final.pending_question is None
+
+
+def test_changed_partial_reply_rolls_back_without_affecting_siblings(service):
+ service.register_question("p", group())
+ service.grade_interaction("p", answer="", question_id="g", answers={"q2": "wrong"})
+ before = service._store.load("p").model_dump()
+ with pytest.raises(MasteryInteractionError, match="committed answer"):
+ service.grade_interaction(
+ "p", answer="", question_id="g", answers={"q1": "alpha", "q2": "beta"}
+ )
+ assert service._store.load("p").model_dump() == before
+
+
+def test_partial_leaf_repair_updates_parent_without_inventing_sibling_reply(service):
+ service.register_question("p", group())
+ service.grade_interaction("p", answer="", question_id="g", answers={"q2": "wrong"})
+ service.repair_question(
+ "p", "q2", action="correct", expected_answer="wrong", reason="Verified key correction"
+ )
+ parent = service._store.get_interaction("p", "g")
+ assert parent.result["results"][0]["is_correct"] is True
+ assert parent.result["results"][0].get("result") != "incorrect"
+ assert parent.result["complete"] is False
+ assert parent.question.questions[0].question_id == "q1"
+ assert parent.question.questions[1].expected_answer == "wrong"
+ assert parent.answers == {"q2": "wrong"}
+ assert service._store.load("p").pending_question.questions[1].expected_answer == "wrong"
+ service.repair_question("p", "q2", action="void", reason="Uncertain key")
+ assert service._store.get_interaction("p", "g").result["results"][0]["is_correct"] is None
+ assert service._store.load("p").pending_question.question_id == "g"
+
+
+def test_third_answer_first_does_not_score_unanswered_questions(service):
+ progress = service._store.load("p")
+ progress.modules[0].knowledge_points.append(
+ KnowledgePoint(id="kp3", name="third", type=KnowledgeType.MEMORY, module_id="m")
+ )
+ service.save(progress)
+ pending = group()
+ pending.questions.append(leaf("q3", "kp3", "gamma"))
+ service.register_question("p", pending)
+ progress, interaction, _ = service.grade_interaction(
+ "p", answer="", question_id="g", answers={"q3": "gamma"}
+ )
+ assert [attempt.question_id for attempt in progress.quiz_attempts] == ["q3"]
+ assert interaction.result["remaining_question_ids"] == ["q1", "q2"]
+ assert interaction.status == InteractionStatus.AWAITING_INPUT
+
+
+def test_partial_group_cannot_reuse_a_leaf_from_another_group(service):
+ service.register_question("p", group())
+ service.grade_interaction("p", answer="", question_id="g", answers={"q2": "beta"})
+ replacement = group()
+ replacement.question_id = "new-group"
+ service.register_question(
+ "p", replacement, pending_action="replace", replace_question_id="g"
+ )
+ before = service._store.load("p").model_dump()
+ with pytest.raises(MasteryInteractionError, match="another interaction"):
+ service.grade_interaction(
+ "p", answer="", question_id="new-group", answers={"q2": "beta"}
+ )
+ assert service._store.load("p").model_dump() == before
+
+
+
+
+async def test_partial_grade_tool_and_status_preserve_child_identity(service, monkeypatch):
+ from deeptutor.capabilities.mastery import tools
+
+ monkeypatch.setattr(tools, "_new_service", lambda: service)
+ synced = []
+
+ async def sync(**kwargs):
+ synced.append(kwargs["pending"].question_id)
+
+ monkeypatch.setattr(tools, "_sync_mastery_attempt_to_question_bank", sync)
+ service.register_question("p", group())
+ graded = await tools.MasteryGradeTool().execute(
+ _mastery_path_id="p", question_id="g", answers={"q2": "beta"}
+ )
+ assert graded.success
+ assert graded.metadata["mastery_grade"]["complete"] is False
+ assert graded.metadata["mastery_grade"]["remaining_question_ids"] == ["q1"]
+ assert synced == ["q2"]
+ status = await tools.MasteryStatusTool().execute(_mastery_path_id="p")
+ pending = status.metadata["mastery_status"]["pending_interaction"]
+ assert pending["answers"] == {"q2": "beta"}
+ assert pending["remaining_question_ids"] == ["q1"]
+ assert "PRIVATE" not in json.dumps(pending)
+
+
def test_explicit_replacement_retains_abandoned_history(service):
service.register_question("p", leaf("old"))
with pytest.raises(MasteryInteractionError):
diff --git a/tests/services/session/test_mastery_group_runtime.py b/tests/services/session/test_mastery_group_runtime.py
index 61e3786c4..1fa783907 100644
--- a/tests/services/session/test_mastery_group_runtime.py
+++ b/tests/services/session/test_mastery_group_runtime.py
@@ -20,13 +20,35 @@ def test_card_contract_keeps_independent_group_answers():
@pytest.mark.parametrize(
- "answers", [{}, {"first": "A"}, {"first": "", "second": "B"}, {"first": 1, "second": "B"}]
+ "answers", [{}, {"first": "", "second": "B"}, {"first": 1, "second": "B"}]
)
-def test_group_transport_rejects_incomplete_or_untyped_values(answers):
+def test_group_transport_rejects_empty_or_untyped_values(answers):
with pytest.raises(ValidationError):
MasteryCardAnswer(question_id="group", text="display", answers=answers)
+def test_group_transport_accepts_one_actual_reply():
+ card = MasteryCardAnswer(question_id="group", text="second only", answers={"second": "B"})
+ assert card.answers == {"second": "B"}
+
+
+def test_partial_group_seed_does_not_claim_unanswered_questions_were_graded():
+ context = UnifiedContext(metadata={
+ "mastery_mode": True,
+ "mastery_card_grade": {
+ "result": {
+ "question_type": "group", "complete": False,
+ "remaining_question_ids": ["first"],
+ "results": [{"question_id": "second", "is_correct": True}],
+ },
+ },
+ })
+ seed = MasteryLoopCapability().pre_loop_seed(context)
+ assert "first" in seed and "unanswered" in seed
+ assert "second: correct" in seed
+ assert "The question group was already graded" not in seed
+
+
@pytest.mark.asyncio
async def test_runtime_forwards_group_map_not_display_summary(monkeypatch):
calls = []
diff --git a/web/components/chat/home/MasteryQuestionCard.tsx b/web/components/chat/home/MasteryQuestionCard.tsx
index c96e2039f..2882c0ca2 100644
--- a/web/components/chat/home/MasteryQuestionCard.tsx
+++ b/web/components/chat/home/MasteryQuestionCard.tsx
@@ -367,12 +367,14 @@ function GroupedMasteryQuestionCard({
const [answers, setAnswers] = useState<Record<string, { text: string; available: boolean }>>({});
const [skipping, setSkipping] = useState(false);
const [skipFailed, setSkipFailed] = useState(false);
- const { sending, failed, submit } = useCardSubmission(onSubmit, rejectionKey);
const questions = question.questions ?? [];
- const settled = answered || Boolean(grade) || skipped === true;
+ const acknowledgementKey = grade ? JSON.stringify((grade.results ?? []).map((part) => part.questionId)) : undefined;
+ const { sending, failed, submit } = useCardSubmission(onSubmit, rejectionKey, acknowledgementKey);
+ const settled = answered || (Boolean(grade) && grade?.complete !== false) || skipped === true;
const locked = settled || sending || skipping;
- const ready = questions.length >= 2 && questions.every((part) =>
- Boolean(answers[part.questionId]?.text) && answers[part.questionId]?.available);
+ const remaining = questions.filter((part) => !grade?.results?.some((result) => result.questionId === part.questionId));
+ const drafted = remaining.filter((part) => Boolean(answers[part.questionId]?.text));
+ const ready = questions.length >= 2 && drafted.length > 0 && drafted.every((part) => answers[part.questionId]?.available);
const updateAnswer = useCallback((questionId: string, text: string, available: boolean) => {
setAnswers((current) => current[questionId]?.text === text && current[questionId]?.available === available
@@ -381,8 +383,8 @@ function GroupedMasteryQuestionCard({
const handleSubmit = () => {
if (locked || !ready) return;
- const parts = questions.map((part) => ({ questionId: part.questionId, text: answers[part.questionId].text }));
- void submit({ text: parts.map((part, index) => `${index + 1}. ${part.text}`).join("\n"), answers: parts });
+ const parts = drafted.map((part) => ({ questionId: part.questionId, text: answers[part.questionId].text }));
+ void submit({ text: parts.map((part) => `${questions.findIndex((question) => question.questionId === part.questionId) + 1}. ${part.text}`).join("\n"), answers: parts });
};
const handleSkip = async () => {
@@ -405,7 +407,7 @@ function GroupedMasteryQuestionCard({
{questions.map((part) => {
const partGrade = grade?.results?.find((result) => result.questionId === part.questionId) ?? null;
return <SingleMasteryQuestionCard key={part.questionId} question={part} grade={partGrade}
- answered={settled} skipped={skipped} submittedAnswer={partGrade?.learnerAnswer ?? ""}
+ answered={settled || Boolean(partGrade)} skipped={skipped} submittedAnswer={partGrade?.learnerAnswer ?? ""}
onSubmit={onSubmit} onChallenge={onChallenge} groupInput={{ locked, onChange: updateAnswer }} />;
})}
{!settled ? <div className="mt-3 flex items-center justify-between gap-3">
diff --git a/web/features/chat/messages/ChatMessageList.tsx b/web/features/chat/messages/ChatMessageList.tsx
index 7effeb4dc..4477fef13 100644
--- a/web/features/chat/messages/ChatMessageList.tsx
+++ b/web/features/chat/messages/ChatMessageList.tsx
@@ -814,10 +814,10 @@ export const AssistantMessage = memo(function AssistantMessage({
<MasteryQuestionCard
question={question}
grade={grade}
- answered={Boolean(grade)}
+ answered={Boolean(grade) && grade?.complete !== false}
skipped={masterySkips?.has(question.questionId) ?? false}
submittedAnswer={grade?.learnerAnswer ?? ""}
- rejectionKey={!grade ? masteryAnswerRejections?.get(question.questionId)?.retryKey : undefined}
+ rejectionKey={!grade || grade.complete === false ? masteryAnswerRejections?.get(question.questionId)?.retryKey : undefined}
onSubmit={({ text, answers }) =>
onAnswerMasteryQuestion
? onAnswerMasteryQuestion({
diff --git a/web/hooks/use-card-submission.ts b/web/hooks/use-card-submission.ts
index 8a6f870b3..5e2c4e9ec 100644
--- a/web/hooks/use-card-submission.ts
+++ b/web/hooks/use-card-submission.ts
@@ -22,11 +22,12 @@ export type SubmitUserReply = (
* Both question cards implement that same three-state dance, so it lives here
* rather than twice.
*/
-export function useCardSubmission(onSubmit: SubmitUserReply, rejectionKey?: string) {
+export function useCardSubmission(onSubmit: SubmitUserReply, rejectionKey?: string, acknowledgementKey?: string) {
const [sending, setSending] = useState(false);
const [failed, setFailed] = useState(false);
const [failureMessage, setFailureMessage] = useState<string | null>(null);
const [submittedAfterRejection, setSubmittedAfterRejection] = useState<string>();
+ const [submittedAfterAcknowledgement, setSubmittedAfterAcknowledgement] = useState<string>();
const submit = useCallback(
async (payload: UserReplyPayload) => {
@@ -34,6 +35,7 @@ export function useCardSubmission(onSubmit: SubmitUserReply, rejectionKey?: stri
setFailed(false);
setFailureMessage(null);
setSubmittedAfterRejection(rejectionKey);
+ setSubmittedAfterAcknowledgement(acknowledgementKey);
let accepted: void | boolean;
try {
accepted = await onSubmit(payload);
@@ -49,12 +51,13 @@ export function useCardSubmission(onSubmit: SubmitUserReply, rejectionKey?: stri
setFailed(true);
}
},
- [onSubmit, rejectionKey],
+ [onSubmit, rejectionKey, acknowledgementKey],
);
// A delivered turn may still reject its answer contract on the server.
// Compare durable event identities, not copy: identical retries must each
// reopen the existing editors, without clearing the learner's answers.
const rejected = Boolean(rejectionKey && rejectionKey !== submittedAfterRejection);
- return { sending: sending && !rejected, failed: failed || rejected, failureMessage, submit };
+ const acknowledged = acknowledgementKey !== submittedAfterAcknowledgement;
+ return { sending: sending && !rejected && !acknowledged, failed: (failed && !acknowledged) || rejected, failureMessage, submit };
}
diff --git a/web/lib/mastery-question.ts b/web/lib/mastery-question.ts
index 0553f47f2..8e77c2891 100644
--- a/web/lib/mastery-question.ts
+++ b/web/lib/mastery-question.ts
@@ -53,7 +53,7 @@ export interface MasteryQuestion {
export interface MasteryGradeResult {
questionId: string;
- isCorrect: boolean;
+ isCorrect: boolean | null;
learnerAnswer: string;
correctLabel: string;
correctBody: string;
@@ -61,6 +61,9 @@ export interface MasteryGradeResult {
result?: string;
independent?: boolean;
results?: MasteryGradeResult[];
+ /** Older complete grades omit these incremental-group fields. */
+ complete?: boolean;
+ remainingQuestionIds?: string[];
}
export interface MasteryAnswerRejection {
@@ -138,7 +141,7 @@ function parseGrade(value: unknown): MasteryGradeResult | null {
: undefined;
return {
questionId,
- isCorrect: raw.is_correct === true,
+ isCorrect: raw.is_correct === null ? null : raw.is_correct === true,
learnerAnswer: text(raw.learner_answer),
correctLabel: text(raw.correct_label),
correctBody: text(raw.correct_body),
@@ -146,6 +149,10 @@ function parseGrade(value: unknown): MasteryGradeResult | null {
...(text(raw.result) ? { result: text(raw.result) } : {}),
...(typeof raw.independent === "boolean" ? { independent: raw.independent } : {}),
...(results ? { results } : {}),
+ ...(typeof raw.complete === "boolean" ? { complete: raw.complete } : {}),
+ ...(Array.isArray(raw.remaining_question_ids) ? {
+ remainingQuestionIds: raw.remaining_question_ids.filter((id): id is string => typeof id === "string" && Boolean(id.trim())),
+ } : {}),
};
}
@@ -264,6 +271,11 @@ export function collectMasteryAnswerRejections(
for (const message of messages) {
for (const event of message.events ?? []) {
if (event.type !== "tool_result") continue;
+ // Successful partial acknowledgements supersede earlier rejection events,
+ // without hiding a later rejection of the remaining answers.
+ const gradePayload = asRecord(toolResultPayload(event.metadata, "mastery_grade"));
+ const grade = parseGrade(gradePayload?.result);
+ if (grade) rejected.delete(grade.questionId);
const payload = asRecord(toolResultPayload(event.metadata, "mastery_answer_rejected"));
const questionId = text(payload?.question_id).trim();
if (!questionId) continue;
diff --git a/web/tests/card-submission-acknowledgement.spec.tsx b/web/tests/card-submission-acknowledgement.spec.tsx
new file mode 100644
index 000000000..267bdd0c2
--- /dev/null
+++ b/web/tests/card-submission-acknowledgement.spec.tsx
@@ -0,0 +1,39 @@
+import { act, renderHook } from "@testing-library/react";
+import { describe, expect, it, vi } from "vitest";
+
+import { useCardSubmission } from "@/hooks/use-card-submission";
+
+describe("incremental card acknowledgement", () => {
+ it("reopens after each new durable leaf set, not a replay of the same set", async () => {
+ const onSubmit = vi.fn(() => true);
+ const { result, rerender } = renderHook(({ acknowledgement }) =>
+ useCardSubmission(onSubmit, undefined, acknowledgement),
+ { initialProps: { acknowledgement: "[]" } });
+ await act(async () => result.current.submit({ answers: [{ questionId: "a", text: "A" }] }));
+ expect(result.current.sending).toBe(true);
+ rerender({ acknowledgement: '["a"]' });
+ expect(result.current.sending).toBe(false);
+ expect(result.current.failed).toBe(false);
+ await act(async () => result.current.submit({ answers: [{ questionId: "b", text: "B" }] }));
+ expect(result.current.sending).toBe(true);
+ rerender({ acknowledgement: '["a"]' });
+ expect(result.current.sending).toBe(true);
+ rerender({ acknowledgement: '["a","b"]' });
+ expect(result.current.sending).toBe(false);
+ });
+
+ it("still reopens on rejection or failed delivery without an acknowledgement", async () => {
+ const onSubmit = vi.fn().mockResolvedValueOnce(false).mockResolvedValueOnce(true);
+ const { result, rerender } = renderHook(({ rejection }) =>
+ useCardSubmission(onSubmit, rejection, '["a"]'),
+ { initialProps: { rejection: undefined as string | undefined } });
+ await act(async () => result.current.submit({ text: "B" }));
+ expect(result.current.sending).toBe(false);
+ expect(result.current.failed).toBe(true);
+ await act(async () => result.current.submit({ text: "B" }));
+ expect(result.current.sending).toBe(true);
+ rerender({ rejection: "reject-next" });
+ expect(result.current.sending).toBe(false);
+ expect(result.current.failed).toBe(true);
+ });
+});
diff --git a/web/tests/mastery-question-group.spec.tsx b/web/tests/mastery-question-group.spec.tsx
index 7e2221571..d965e1d4d 100644
--- a/web/tests/mastery-question-group.spec.tsx
+++ b/web/tests/mastery-question-group.spec.tsx
@@ -6,8 +6,10 @@ import { MasteryQuestionCard } from "@/components/chat/home/MasteryQuestionCard"
import { ChatMessageList } from "@/features/chat/messages";
import type { StreamEvent } from "@/features/chat/model/protocol";
import { initI18n } from "@/i18n/init";
+import type { UserReplyPayload } from "@/hooks/use-card-submission";
import {
collectMasteryGrades,
+ collectMasteryAnswerRejections,
extractMasteryQuestion,
} from "@/lib/mastery-question";
@@ -53,6 +55,16 @@ describe("grouped mastery assessment", () => {
expect(JSON.stringify(withPrivateKeys)).not.toContain("private");
});
+ it("clears old rejection on partial acknowledgement, but keeps a later rejection", () => {
+ const rejected = event({ mastery_answer_rejected: { question_id: "group-1" } });
+ const acknowledged = event({ mastery_grade: { result: { question_id: "group-1",
+ question_type: "group", complete: false, is_correct: null, results: [
+ { question_id: "part-a", is_correct: true, learner_answer: "A" },
+ ] } } });
+ expect(collectMasteryAnswerRejections([{ events: [rejected, acknowledged] }]).size).toBe(0);
+ expect(collectMasteryAnswerRejections([{ events: [acknowledged, rejected] }]).has("group-1")).toBe(true);
+ });
+
it("rejects duplicate, nested, or incomplete group identities", () => {
for (const questions of [[leaf("part-a")], [leaf("part-a"), leaf("part-a")],
[leaf("part-a"), group]]) {
@@ -60,7 +72,7 @@ describe("grouped mastery assessment", () => {
}
});
- it("requires both answers and sends one submission with stable child IDs", async () => {
+ it("allows one answer, or both together, with stable child IDs", async () => {
const user = userEvent.setup();
const onSubmit = vi.fn(() => true);
render(<MasteryQuestionCard question={question()} grade={null} answered={false}
@@ -68,7 +80,7 @@ describe("grouped mastery assessment", () => {
const submit = screen.getByRole("button", { name: "Submit" });
expect(submit).toBeDisabled();
await user.click(screen.getByRole("button", { name: /Matching description/ }));
- expect(submit).toBeDisabled();
+ expect(submit).toBeEnabled();
await user.type(screen.getByRole("textbox"), "They have different roles");
expect(submit).toBeEnabled();
await user.click(submit);
@@ -81,6 +93,71 @@ describe("grouped mastery assessment", () => {
});
});
+ it("keeps the remaining leaf editable on reload after a partial grade", async () => {
+ const onSubmit = vi.fn(() => true);
+ const grades = collectMasteryGrades([{ events: [event({ mastery_grade: {
+ result: { question_id: "group-1", question_type: "group", complete: false,
+ remaining_question_ids: ["part-b"], is_correct: null, result: "ungraded",
+ results: [{ question_id: "part-a", is_correct: true, learner_answer: "A", explanation: "Confirmed first part." }] },
+ } })] }]);
+ expect(grades.get("group-1")).toMatchObject({ complete: false, isCorrect: null, remainingQuestionIds: ["part-b"] });
+ const user = userEvent.setup();
+ render(<MasteryQuestionCard question={question()} grade={grades.get("group-1")!}
+ answered={false} submittedAnswer="" onSubmit={onSubmit} />);
+ expect(screen.getByText("Correct")).toBeInTheDocument();
+ expect(screen.getByRole("button", { name: /Matching description/ })).toBeDisabled();
+ expect(screen.getByRole("textbox")).toBeEnabled();
+ await user.type(screen.getByRole("textbox"), "Remaining answer");
+ await user.click(screen.getByRole("button", { name: "Submit" }));
+ expect(onSubmit).toHaveBeenCalledExactlyOnceWith({ text: "2. Remaining answer",
+ answers: [{ questionId: "part-b", text: "Remaining answer" }] });
+ });
+
+ it("releases the sending latch on partial acknowledgement and preserves unsent edits", async () => {
+ const user = userEvent.setup();
+ const onSubmit = vi.fn<(payload: UserReplyPayload) => boolean>(() => true);
+ const props = { question: question(), answered: false, submittedAnswer: "", onSubmit };
+ const { rerender } = render(<MasteryQuestionCard {...props} grade={null} />);
+ await user.click(screen.getByRole("button", { name: /Matching description/ }));
+ await user.click(screen.getByRole("button", { name: "Submit" }));
+ expect(screen.getByRole("textbox")).toBeDisabled();
+ rerender(<MasteryQuestionCard {...props} grade={null} rejectionKey="rejected-1" />);
+ await user.type(screen.getByRole("textbox"), "Unsent draft");
+ const partial = collectMasteryGrades([{ events: [event({ mastery_grade: { result: {
+ question_id: "group-1", question_type: "group", complete: false, remaining_question_ids: ["part-b"],
+ is_correct: null, results: [{ question_id: "part-a", is_correct: true, learner_answer: "A" }],
+ } } })] }]).get("group-1")!;
+ rerender(<MasteryQuestionCard {...props} grade={partial} />);
+ expect(screen.getByRole("textbox")).toHaveValue("Unsent draft");
+ expect(screen.getByRole("textbox")).toBeEnabled();
+ await user.click(screen.getByRole("button", { name: "Submit" }));
+ expect(onSubmit.mock.calls[1][0].answers).toEqual([{ questionId: "part-b", text: "Unsent draft" }]);
+ expect(screen.getByRole("button", { name: "Submit" })).toBeDisabled();
+ });
+
+ it("actual message list leaves an incomplete group open and retains rejected remaining answers", async () => {
+ const user = userEvent.setup();
+ const onAnswer = vi.fn(() => true);
+ const posed = { id: 1, role: "assistant" as const, content: "", parentMessageId: null,
+ capability: "mastery_path", events: [event({ mastery_question: group })] };
+ const partial = event({ mastery_grade: { result: { question_id: "group-1", question_type: "group",
+ complete: false, remaining_question_ids: ["part-b"], is_correct: null,
+ results: [{ question_id: "part-a", is_correct: true, learner_answer: "A" }] } } });
+ const conversation = (events: StreamEvent[]) => <ChatMessageList messages={[posed,
+ { id: 2, role: "assistant", content: "", parentMessageId: 1, capability: "mastery_path", events }]}
+ isStreaming={false} onCopyAssistantMessage={async () => undefined}
+ onRegenerateMessage={() => undefined} onAnswerMasteryQuestion={onAnswer} />;
+ const { rerender } = render(conversation([partial]));
+ expect(screen.getByRole("textbox")).toBeEnabled();
+ await user.type(screen.getByRole("textbox"), "Keep this after rejection");
+ await user.click(screen.getByRole("button", { name: "Submit" }));
+ const rejection = { ...event({ mastery_answer_rejected: { question_id: "group-1" } }),
+ turn_id: "retry-turn", seq: 2 };
+ rerender(conversation([partial, rejection]));
+ expect(screen.getByRole("textbox")).toHaveValue("Keep this after rejection");
+ expect(screen.getByRole("button", { name: "Submit" })).toBeEnabled();
+ });
+
it("retains both answers when delivery fails and allows a retry", async () => {
const user = userEvent.setup();
const onSubmit = vi.fn().mockResolvedValueOnce(false).mockResolvedValueOnce(true);