You're reading this. The skill predicted it. A workflow that turns every post into a calibrated experiment—score, blind-predict, retro, evolve. The future doesn't reward effort, it rewards those who see the pattern first. 1M followers in a month — not luck, system.
Python
fix: replace cheat-shoot line-level diff with char-level Levenshtein on normalized text (#14)
Jooonnn73536a4f355c
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 7152828..efb2d32 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -8,6 +8,38 @@ All notable changes to cheat-on-content will be documented here.## [Unreleased]+### Fixed — cheat-shoot DIFF_METRIC 在口语化场景的 v2 误触发(**BREAKING for v2-trigger-logic**)++**问题**:cheat-shoot Phase 3b 用 line-level unified diff 算 `diff_pct = (added + removed) * 100 / orig_lines`。但**创作者真实场景**——draft 是 markdown 长句(一行 ~50 字),拍摄稿是 whisper 转录的口语化短断句(每行 ~5-10 字)——同样的内容会 inflate diff_pct 到 100-200%,触发本不该的 v2 重判。++**实测复现**(use clone PR pre-fix):+- draft markdown 63 行 / ~380 字+- 拍摄转录 100 行 / 同 ~380 字+- 内容几乎完全保留(审稿人原句一字不差 + 5 年前反应概念 + 升维段所有金句)+- 唯一新增:1 句 brand 锚定 "全面拥抱 AI"+- **line-level diff_pct = 198%** ⚠️ v2 错误触发+- 语义内容真实 diff ≈ 15-25%++**修复**:拆 metric。+- **DIFF_METRIC=char_levenshtein_normalized**(新默认)—— [tools/diff_pct.py](tools/diff_pct.py) 先 normalize(去 markdown header / 分隔线 / 列表标记 / 装饰标点 / 折叠所有空白),再算 char-level Levenshtein / max(len_a, len_b)+- backend 优先级:`rapidfuzz`(C-backed,~ms 级;需 `pip install rapidfuzz`) → `difflib.SequenceMatcher`(stdlib,永远可用,~10ms 级)+- **V2_TRIGGER_THRESHOLD = 0.30** 保持不动(阈值经验合理)+- legacy line-level 保留为终极 fallback(只在 python3 + tools/diff_pct.py 都不可达时降级)++**测试**:3 fixture × 2 backend = 6 case,全过:++| Case | 内容 | 期望范围 | difflib | rapidfuzz |+|---|---|---|---|---|+| 1 | markdown 长句 vs 转录短断句(内容同) | < 30 | 7 | 12 |+| 2 | 完全不同主题 | ≥ 60 | 88 | 97 |+| 3 | 加 20% outro/CTA | 10-30 | 14 | 25 |++跑 `bash tools/diff_pct_test.sh` 复现。++**已知局限**:+- 历史 v2 prediction 文件保留 line-level 数字作 audit trail——不重新走过去的 prediction+- normalize 是启发式(中文标点)——对其他语言 / 非典型 markdown 可能需调整 regex+### Fixed — `rubric_notes.md` 实绩泄漏漏洞(**BREAKING for blind channel integrity**)**问题**:PR #11 引入的 cheat-score-blind sub-agent 承诺只读 `scripts/<id>.md` + `rubric_notes.md` 两个文件。但 cheat-bump Phase 5 把升级 Memo(含真实视频名 + 实绩 + 派生证据)写进了 `rubric_notes.md`——sub-agent 通过白名单读到了本不该看的实绩,盲打分变成"看过实绩的事后合理化"。实测复现:5 条已发视频里 2 条 sub-agent 自动标 `any_contamination_signal: true`(refusal=`non_blind_warning`,所有维度 confidence 降 medium)。diff --git a/skills/cheat-shoot/SKILL.md b/skills/cheat-shoot/SKILL.mdindex f0373c5..1a51efb 100644--- a/skills/cheat-shoot/SKILL.md+++ b/skills/cheat-shoot/SKILL.md@@ -41,8 +41,9 @@ cheat-shoot 自己**不**写预测内容——所有预测落盘逻辑在 cheat-## Constants- **REQUIRE_PREDICTION = true** — 拍前必须先有 v1 prediction 文件-- **V2_TRIGGER_THRESHOLD = 0.30** — 稿子 diff 超过 30%(行级 unified diff 行数 / 原稿子行数)→ 默认建议 v2 重判;低于 30% 询问用户是否仍要 v2-- **DIFF_METRIC = lines** — 用 `diff -u | grep '^[+-]' | wc -l` 算改动行数 / 原文件行数+- **V2_TRIGGER_THRESHOLD = 0.30** — normalize 后 char-level diff 超过 30% → 默认建议 v2 重判;低于 30% 询问用户是否仍要 v2+- **DIFF_METRIC = char_levenshtein_normalized**(**默认**)—— 通过 [`tools/diff_pct.py`](../../tools/diff_pct.py) 调用:先 normalize(去 markdown header / 分隔线 / 列表标记 / 装饰标点 / 折叠所有空白),再算 char-level Levenshtein / max(len_a, len_b)。preferred backend `rapidfuzz`,fallback `difflib.SequenceMatcher`(stdlib,永远可用)。**旧版 line-level 在口语化转录场景误报严重**(draft 长 markdown 句 vs whisper 转录的短断句,内容几乎不变但 line-level 算出 ~200% diff)—— PR #14 修复+- **DIFF_METRIC=lines** —— legacy fallback:当 python3 完全不可用或 tools/diff_pct.py 找不到时降级到 `diff -u | grep '^[+-]' | wc -l` 算法## Inputs@@ -99,11 +100,27 @@ c) 大改了,基本是另一条 → 走 _redo 流程:3. 若用户没保留(即兴)→ 标 `script_lost`,写占位文件 + 警告"v2 重判跳过——下次建议留稿(哪怕 voice memo 转录)",进 Phase 44. 提供了的话:算 diff```bash- added=$(diff -u scripts/<id>.md videos/<id>/script.md | grep -c '^+')- removed=$(diff -u scripts/<id>.md videos/<id>/script.md | grep -c '^-')- total_orig=$(wc -l < scripts/<id>.md)- diff_pct=$(( (added + removed) * 100 / total_orig ))+ # 解析 cheat-on-content 源码根(cheat-shoot 是 symlink 装的)+ SKILL_REAL="$(readlink -f ~/.claude/skills/cheat-shoot 2>/dev/null || readlink ~/.claude/skills/cheat-shoot 2>/dev/null)"+ if [[ -n "$SKILL_REAL" ]]; then+ REPO_ROOT="$(cd "$SKILL_REAL/../.." && pwd)"+ DIFF_TOOL="$REPO_ROOT/tools/diff_pct.py"+ fi++ if [[ -n "${DIFF_TOOL:-}" && -f "$DIFF_TOOL" ]] && command -v python3 >/dev/null 2>&1; then+ # 默认 char-level Levenshtein on normalized text(rapidfuzz preferred, difflib fallback)+ diff_pct=$(python3 "$DIFF_TOOL" "scripts/<id>.md" "videos/<id>/script.md")+ else+ # legacy line-level fallback——只在 python3 或 diff_pct.py 都不可用时用+ added=$(diff -u scripts/<id>.md videos/<id>/script.md | grep -c '^+')+ removed=$(diff -u scripts/<id>.md videos/<id>/script.md | grep -c '^-')+ total_orig=$(wc -l < scripts/<id>.md)+ diff_pct=$(( (added + removed) * 100 / total_orig ))+ echo "⚠️ fallback 到 line-level diff——口语化转录会 inflate diff_pct,可能误触发 v2"+ fi```++ **为什么 normalize + char-level**:line-level diff 在创作者真实场景(draft 是 markdown 长句、拍摄稿是 whisper 转录的口语化短行)算出 ~200% 差异但内容几乎不变。char-level Levenshtein 在 normalize 后稳定反映**内容**差异,而非格式差异。详见 [`tools/diff_pct.py`](../../tools/diff_pct.py) + `tools/diff_pct_test.sh`(3 fixture 在两个 backend 上全过)。5. **判定 v2 触发**:- `diff_pct >= 30` → 默认建议 v2 重判,**主动调用** `/cheat-predict — mode: v2 — prediction-file: predictions/<id>.md` 传 `videos/<id>/script.md` 作 input。cheat-predict 走 v2 模式 append `## 预测 v2`- `diff_pct < 30` → 询问用户:"只改了 N% 的内容,要重判吗?默认不(v1 预测仍有效)"。用户说要 → 同上调用;用户说不 → 跳过 v2,继续 Phase 4diff --git a/tools/diff_pct.py b/tools/diff_pct.pynew file mode 100755index 0000000..0a3298b--- /dev/null+++ b/tools/diff_pct.py@@ -0,0 +1,105 @@+#!/usr/bin/env python3+"""+diff_pct.py — char-level diff_pct on normalized text.++Replaces the legacy line-level diff in cheat-shoot Phase 3b, which+inflated diff_pct for spoken-style transcripts (long markdown lines vs+short transcribed lines) and falsely triggered v2 re-predictions.++Algorithm:+ 1. Normalize both files (strip markdown elements, collapse whitespace)+ 2. Char-level Levenshtein distance / max(len_a, len_b)+ 3. Output integer 0-100++Backend selection:+ - Prefer rapidfuzz (C-backed, ~ms for 10KB strings) — `pip install rapidfuzz`+ - Fallback to stdlib difflib.SequenceMatcher (slower but always available)++Usage:+ python3 tools/diff_pct.py <orig_path> <new_path>+Output:+ Single integer 0-100 on stdout.++Exit codes:+ 0 = success (number on stdout)+ 2 = bad args+ 3 = file read error+"""+from __future__ import annotations++import re+import sys+from pathlib import Path+++# Markdown / formatting noise that whisper-style transcripts don't have.+# Stripping these from BOTH sides equalizes the surface so we measure+# content-similarity, not formatting-similarity.+_MARKDOWN_HEADER = re.compile(r"^#+\s+.*$", re.MULTILINE)+_MARKDOWN_HR = re.compile(r"^[-=*]{3,}\s*$", re.MULTILINE)+_MARKDOWN_BLOCKQUOTE = re.compile(r"^>+\s*", re.MULTILINE)+_MARKDOWN_LIST = re.compile(r"^[-*+]\s+|^\d+\.\s+", re.MULTILINE)+_FORMATTING_PUNCT = re.compile(r"[「」『』\"`*_~]")+_ALL_WHITESPACE = re.compile(r"\s+")+++def normalize(text: str) -> str:+ """Strip markdown / formatting noise, collapse to single 'sentence'."""+ text = _MARKDOWN_HEADER.sub("", text)+ text = _MARKDOWN_HR.sub("", text)+ text = _MARKDOWN_BLOCKQUOTE.sub("", text)+ text = _MARKDOWN_LIST.sub("", text)+ text = _FORMATTING_PUNCT.sub("", text)+ text = _ALL_WHITESPACE.sub("", text)+ return text+++def diff_pct(a: str, b: str) -> tuple[int, str]:+ """Return (0-100 int, backend_name)."""+ a_n = normalize(a)+ b_n = normalize(b)+ max_len = max(len(a_n), len(b_n))+ if max_len == 0:+ return 0, "trivial"++ # Try rapidfuzz first (faster + matches the spec)+ try:+ from rapidfuzz.distance import Levenshtein++ d = Levenshtein.distance(a_n, b_n)+ return (d * 100) // max_len, "rapidfuzz"+ except ImportError:+ pass++ # Fallback to stdlib difflib SequenceMatcher.+ # ratio() returns 0-1 similarity; we want 0-100 distance.+ # SequenceMatcher's algorithm differs from pure Levenshtein but+ # for our use case (semantic content similarity) it's well-correlated.+ from difflib import SequenceMatcher++ sm = SequenceMatcher(a=a_n, b=b_n, autojunk=False)+ sim = sm.ratio()+ return int(round((1 - sim) * 100)), "difflib"+++def main() -> int:+ if len(sys.argv) != 3:+ print("usage: diff_pct.py <orig_path> <new_path>", file=sys.stderr)+ return 2++ try:+ a = Path(sys.argv[1]).read_text(encoding="utf-8")+ b = Path(sys.argv[2]).read_text(encoding="utf-8")+ except OSError as e:+ print(f"read error: {e}", file=sys.stderr)+ return 3++ pct, backend = diff_pct(a, b)+ print(pct)+ # backend goes to stderr so callers can inspect without parsing stdout+ print(f"backend={backend} a_norm_len={len(normalize(a))} b_norm_len={len(normalize(b))}", file=sys.stderr)+ return 0+++if __name__ == "__main__":+ sys.exit(main())diff --git a/tools/diff_pct_test.sh b/tools/diff_pct_test.shnew file mode 100755index 0000000..3c2ac61--- /dev/null+++ b/tools/diff_pct_test.sh@@ -0,0 +1,150 @@+#!/usr/bin/env bash+#+# diff_pct_test.sh — regression test for tools/diff_pct.py+#+# Validates 3 fixture cases that the legacy line-level diff failed:+# case 1: long markdown lines vs spoken-transcript short lines, same content+# → expected diff_pct < 30 (was ~198% under legacy)+# case 2: completely different topic+# → expected diff_pct ≥ 60+# case 3: orig + ~20% new content appended+# → expected diff_pct 10-30+#+# Usage:+# bash tools/diff_pct_test.sh+# Exit:+# 0 = all pass+# 1 = ≥1 failure+#+# Runs against whichever backend is installed (rapidfuzz preferred, difflib+# fallback). Both should pass these ranges — they're chosen to be wide+# enough to absorb backend-algorithm differences.++set -euo pipefail++SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"+DIFF_PCT="$SCRIPT_DIR/diff_pct.py"++if [[ ! -f "$DIFF_PCT" ]]; then+ echo "❌ diff_pct.py not found at $DIFF_PCT" >&2+ exit 1+fi++TMP=$(mktemp -d)+trap "rm -rf $TMP" EXIT++PASS=0+FAIL=0++run_case() {+ local label="$1"+ local orig="$2"+ local new="$3"+ local min="$4"+ local max="$5"++ local stderr_out+ stderr_out=$(mktemp)+ local actual+ actual=$(python3 "$DIFF_PCT" "$orig" "$new" 2>"$stderr_out")+ local backend+ backend=$(grep -oE 'backend=[a-z]+' "$stderr_out" | head -1 || echo "backend=?")+ rm -f "$stderr_out"++ if (( actual >= min && actual <= max )); then+ echo " ✓ $label: diff_pct=$actual ∈ [$min, $max] ($backend)"+ PASS=$((PASS+1))+ else+ echo " ✗ $label: diff_pct=$actual NOT in [$min, $max] ($backend)"+ FAIL=$((FAIL+1))+ fi+}++echo ""+echo "=== Case 1: long markdown line vs spoken-transcript short lines, same content ==="++cat > "$TMP/case1_orig.md" <<'EOF'+# 视频草稿 — 评审会复盘++「最近发现一个现象」——所有审稿人都在说一样的话:你的研究太老套了。++但你仔细看,他们引用的全是 5 年前的反应。AI 不是新东西。新的是这次大家集体觉醒了。++**升维点**:当所有人都在追新概念的时候,先看穿规律的人已经在用工具赚钱了。+EOF++cat > "$TMP/case1_shot.md" <<'EOF'+最近发现一个现象+所有审稿人都在说一样的话+你的研究太老套了+但你仔细看+他们引用的全是+5 年前的反应+AI 不是新东西+新的是这次+大家集体觉醒了+当所有人都在追新概念的时候+先看穿规律的人+已经在用工具赚钱了+EOF++run_case "spoken-style line breaks, content preserved" \+ "$TMP/case1_orig.md" "$TMP/case1_shot.md" 0 30++echo ""+echo "=== Case 2: completely different topic ==="++cat > "$TMP/case2_orig.md" <<'EOF'+# AI 焦虑++最近 AI 大模型发布得太快了,每周一个新工具。+你刚学会一个,下周它就被淘汰了。+这种焦虑本质上是工具焦虑,不是能力焦虑。+真正不变的是你解决问题的范式——选好题、定好评估、跑通闭环。+EOF++cat > "$TMP/case2_shot.md" <<'EOF'+今天聊聊我家的猫+它叫橘子 是一只英短+最喜欢窝在窗台上晒太阳+我每天下班回家+看到它就忘了所有烦恼+它今天还偷吃了我的牛奶+被我发现了 装作没事+猫真是世界上最神奇的生物+EOF++run_case "completely different topic" \+ "$TMP/case2_orig.md" "$TMP/case2_shot.md" 60 100++echo ""+echo "=== Case 3: orig + ~20% appended (outro / CTA scenario) ==="++# Realistic 创作者 scenario: 拍前稿 ~150 字, 拍时加一句 30 字 outro+# 增量 / 原稿 ≈ 20%, Levenshtein/max(orig,new) ≈ 16-20%+cat > "$TMP/case3_orig.md" <<'EOF'+# 视频 — 关于宿命论++我以前不信宿命论。直到我跑了这个工具——它让我拍了一条视频,预测了流量。+我想证明它是错的,告诉了观众希望集体观测让数据偏移。结果数据是准的。+我没逃出宿命论。我只是从一阶跳到二阶——AI 在观测观测者。+EOF++cat > "$TMP/case3_shot.md" <<'EOF'+我以前不信宿命论。直到我跑了这个工具——它让我拍了一条视频,预测了流量。+我想证明它是错的,告诉了观众希望集体观测让数据偏移。结果数据是准的。+我没逃出宿命论。我只是从一阶跳到二阶——AI 在观测观测者。+此时此刻看到这条的你——是出于好奇,还是在完成算法的最后一次落位?+EOF++run_case "~20% appended outro" \+ "$TMP/case3_orig.md" "$TMP/case3_shot.md" 10 30++echo ""+echo "=== Results: $PASS passed, $FAIL failed ==="++if [[ $FAIL -eq 0 ]]; then+ exit 0+else+ exit 1+fi