Business operating system for Claude Code — 57 skills, 21 agents, smart daemon. Unified inbox (WhatsApp/Email/Slack/Telegram), autonomous PR merge, full-AWS monitoring, revenue (Stripe+RevenueCat), e-commerce (Shopify), marketing (Klaviyo/Meta/GA4), voice (Bland/ElevenLabs), APM (Datadog/NewRelic/OTEL), YOLO mode.
Shell agent-teamsai-agentai-automationanthropicautonomous-agentawsbusiness-operationsclaudeclaude-codeclaude-code-plugindevopsecommerceinbox-zeroklaviyomarketingmcpmorning-briefingpr-automationshopifyterminal
feat(ops-inbox): first-class media enrichment + bridge media-retry (Fix M) (#507)
Aurora Capitalc71d0c2dd140
diff --git a/claude-ops/bin/wa-inbox-fresh.sh b/claude-ops/bin/wa-inbox-fresh.shindex 4fc6f8f6..3d3196e1 100755--- a/claude-ops/bin/wa-inbox-fresh.sh+++ b/claude-ops/bin/wa-inbox-fresh.sh@@ -116,6 +116,13 @@ elselog "wa-fresh: voice transcription not started"fi+# 4b. enrich video/image/document media (async — vision+whisper; never silently dropped)+if systemctl --user start --no-block whatsapp-enrich.service 2>/dev/null; then+ log "wa-fresh: media enrichment queued"+else+ log "wa-fresh: media enrichment not started"+fi+# 5. freshness reportafter=$(sqlite3 "$DB" "SELECT COALESCE(datetime(MAX(timestamp)),'?') FROM messages;") || {log "wa-fresh: ERROR cannot read messages store at $DB — do not trust it"diff --git a/claude-ops/scripts/install-whatsapp-bridge-linux.sh b/claude-ops/scripts/install-whatsapp-bridge-linux.shindex ceff4f84..bdf8bc61 100755--- a/claude-ops/scripts/install-whatsapp-bridge-linux.sh+++ b/claude-ops/scripts/install-whatsapp-bridge-linux.sh@@ -22,6 +22,8 @@# --no-backfill-timer Drop only the bridge unit, skip the 2h timer# --no-transcribe-timer Skip the 10-min voice-note transcription timer# (voice notes won't be auto-transcribed into content)+# --no-enrich-timer Skip the 10-min media enrichment timer+# (video/image/document won't be auto-described)set -euo pipefail@@ -35,6 +37,7 @@ SKIP_BUILD=0RESET_SESSION=0WITH_BACKFILL_TIMER=1WITH_TRANSCRIBE_TIMER=1+WITH_ENRICH_TIMER=1# Source-tree dir (where this script lives in claude-ops checkout)SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"@@ -53,6 +56,7 @@ while (("$#")); do--reset-session) RESET_SESSION=1; shift ;;--no-backfill-timer) WITH_BACKFILL_TIMER=0; shift ;;--no-transcribe-timer) WITH_TRANSCRIBE_TIMER=0; shift ;;+ --no-enrich-timer) WITH_ENRICH_TIMER=0; shift ;;-h|--help) usage 0 ;;*) echo "unknown flag: $1" >&2; usage 1 ;;esac@@ -81,6 +85,7 @@ echo " install dir: $INSTALL_DIR"echo " wa-phone: $WA_PHONE"echo " backfill timer: $([ "$WITH_BACKFILL_TIMER" -eq 1 ] && echo on || echo off)"echo " transcribe timer: $([ "$WITH_TRANSCRIBE_TIMER" -eq 1 ] && echo on || echo off)"+echo " enrich timer: $([ "$WITH_ENRICH_TIMER" -eq 1 ] && echo on || echo off)"# ─── Clone / refresh upstream ────────────────────────────────────────────────mkdir -p "$(dirname "$INSTALL_DIR")"@@ -107,6 +112,11 @@ mkdir -p "$INSTALL_DIR/transcriber"cp "$WA_ASSETS/transcribe_voice_notes.py" "$INSTALL_DIR/transcriber/transcribe_voice_notes.py"chmod +x "$INSTALL_DIR/transcriber/transcribe_voice_notes.py"+# ─── Ship media enricher (whatsapp-enrich.timer runs it) ─────────────────────+echo "▶ Installing enrich_media.py (vision+Whisper video/image/document → content)"+cp "$WA_ASSETS/enrich_media.py" "$INSTALL_DIR/transcriber/enrich_media.py"+chmod +x "$INSTALL_DIR/transcriber/enrich_media.py"+# ─── Ship wa-inbox-fresh.sh (pre-scan freshness gate, run FIRST by ops-inbox) ─echo "▶ Installing wa-inbox-fresh.sh → ~/bin (pre-scan freshness gate)"mkdir -p "$HOME/bin"@@ -122,8 +132,11 @@ chmod +x "$HOME/bin/wa-bridge-keepalive.sh"if [ "$SKIP_BUILD" -ne 1 ]; thenecho "▶ Bumping Go deps + building bridge"pushd "$INSTALL_DIR/whatsapp-bridge" >/dev/null- go get -u go.mau.fi/whatsmeow@latest- go get -u ./...+ # Pin whatsmeow to the known-good rev — whatsmeow has rapid API churn (context+ # params added/removed between releases) that breaks the claude-ops source patches.+ # Do NOT change to @latest without re-verifying all patches compile. (2026-06)+ go get go.mau.fi/[email protected]+ go get ./...go mod tidy# -tags sqlite_fts5 is REQUIRED: ops-inbox search + whatsapp-bridge-migrate.sh# add an fts5 virtual table & triggers on messages.db. The bridge's own@@ -177,6 +190,18 @@ if [ "$WITH_TRANSCRIBE_TIMER" -eq 1 ]; thenfifi+if [ "$WITH_ENRICH_TIMER" -eq 1 ]; then+ sed "s|__INSTALL_DIR__|$INSTALL_DIR|g" \+ "$WA_ASSETS/systemd/whatsapp-enrich.service" \+ > "$SYSTEMD_DIR/whatsapp-enrich.service"+ cp "$WA_ASSETS/systemd/whatsapp-enrich.timer" "$SYSTEMD_DIR/"+ # The enrich service reads OPENAI_API_KEY from this EnvironmentFile.+ if [ ! -f "$HOME/.config/systemd/env/mcp-secrets.env" ]; then+ echo " NOTE: ~/.config/systemd/env/mcp-secrets.env not found — the enrich"+ echo " service needs OPENAI_API_KEY there (KEY=value lines) to run."+ fi+fi+# ─── Disable deprecated wacli daemon if present ──────────────────────────────if systemctl --user list-unit-files claude-ops-wacli-keepalive.service 2>/dev/null | grep -q wacli-keepalive; thenecho "▶ Disabling deprecated claude-ops-wacli-keepalive.service (replaced by systemd-managed bridge)"@@ -202,6 +227,9 @@ fiif [ "$WITH_TRANSCRIBE_TIMER" -eq 1 ]; thensystemctl --user enable --now whatsapp-transcribe.timerfi+if [ "$WITH_ENRICH_TIMER" -eq 1 ]; then+ systemctl --user enable --now whatsapp-enrich.timer+fi# ─── Surface the pairing code if present ─────────────────────────────────────echo "▶ Waiting briefly for the bridge to come up"@@ -230,7 +258,8 @@ fiecho ""echo "▶ Status reference:"echo " systemctl --user status whatsapp-bridge.service"-echo " systemctl --user list-timers whatsapp-backfill.timer whatsapp-transcribe.timer"+echo " systemctl --user list-timers whatsapp-backfill.timer whatsapp-transcribe.timer whatsapp-enrich.timer"echo " curl -fsS -X POST http://127.0.0.1:8080/api/backfill # on-demand backfill"echo " systemctl --user start whatsapp-transcribe.service # on-demand transcribe"+echo " systemctl --user start whatsapp-enrich.service # on-demand media enrich"echo " ~/bin/wa-inbox-fresh.sh # pre-scan freshness gate"diff --git a/claude-ops/scripts/whatsapp/apply-patches.py b/claude-ops/scripts/whatsapp/apply-patches.pyindex 9b3555fb..95bc1960 100755--- a/claude-ops/scripts/whatsapp/apply-patches.py+++ b/claude-ops/scripts/whatsapp/apply-patches.py@@ -143,18 +143,15 @@ PAIR_REPLACEMENT = """\tif client.Store.ID == nil {\t\t\t// otherwise PairPhone silently hangs on the IQ response (no PairSuccess,\t\t\t// no error). Sleeping ~3s after Connect is the godoc-endorsed gate.\t\t\ttime.Sleep(3 * time.Second)-\t\t\t// claude-ops Fix B: current whatsmeow PairPhone has NO context parameter-\t\t\t// (signature: PairPhone(phone, showPushNotification, clientType,-\t\t\t// clientDisplayName)), so we can no longer pass a deadline into the call.-\t\t\t// Preserve Fix B's intent — an internal PairPhone hang must be recoverable —-\t\t\t// by racing the call against a 3-minute watchdog instead of a ctx deadline.+\t\t\t// claude-ops Fix B: wrap PairPhone with a 3-min context deadline.+\t\t\t// PairPhone now takes context as first parameter.\t\t\ttype pairResult struct {\t\t\t\tcode string\t\t\t\terr error\t\t\t}\t\t\tpairCh := make(chan pairResult, 1)\t\t\tgo func() {-\t\t\t\tc, perr := client.PairPhone(phone, true, whatsmeow.PairClientChrome, \"Chrome (Linux)\")+\t\t\t\tpairCtx, pairCancel := context.WithTimeout(context.Background(), 3*time.Minute)\n\t\t\t\tdefer pairCancel()\n\t\t\t\tc, perr := client.PairPhone(pairCtx, phone, true, whatsmeow.PairClientChrome, \"Chrome (Linux)\")\t\t\t\tpairCh <- pairResult{code: c, err: perr}\t\t\t}()\t\t\tselect {@@ -554,10 +551,7 @@ RESYNC_ENDPOINT_REPLACEMENT = """\t// claude-ops Fix D: POST /api/resync_app_sta\t\t\t\treq.Name = \"regular_low\"\t\t\t}\t\t}-\t\t// claude-ops API-drift port (2026-06): current whatsmeow FetchAppState has NO-\t\t// context.Context parameter — signature is FetchAppState(name, fullSync,-\t\t// onlyIfNotSynced). Dropped the leading context.Background() arg.-\t\tif err := client.FetchAppState(appstate.WAPatchName(req.Name), req.FullSync, false); err != nil {+\t\t\t\tif err := client.FetchAppState(context.Background(), appstate.WAPatchName(req.Name), req.FullSync, false); err != nil {\t\t\tw.WriteHeader(http.StatusInternalServerError)\t\t\tfmt.Fprintf(w, `{\"success\":false,\"message\":%q}`, err.Error())\t\t\treturn@@ -1090,9 +1084,7 @@ func healLTHash(ctx context.Context, client *whatsmeow.Client, patchName appstat\t\twdb.Close()\t\t// Re-fetch from version 0 (fullSync=true, onlyIfNotSynced=false).-\t\t// claude-ops API-drift port (2026-06): FetchAppState dropped its ctx param.-\t\t// (ctx is still used above for the wdb.ExecContext snapshot-row deletes.)-\t\terr = client.FetchAppState(patchName, true, false)+\t\t\t\terr = client.FetchAppState(ctx, patchName, true, false)\t\tif err == nil {\t\t\treturn nil\t\t}@@ -1147,7 +1139,7 @@ func sendRecoveryAndBackfill(client *whatsmeow.Client, patchName appstate.WAPatc\t)\t// Best-effort full server-side resync. Errors are non-fatal here — the caller\t// (archive auto-retry / recover endpoint) reports its own outcome.-\tif err := client.FetchAppState(patchName, true, false); err != nil {+\tif err := client.FetchAppState(context.Background(), patchName, true, false); err != nil {\t\tfmt.Printf("Fix L: best-effort resync of %s failed: %v\\n", string(patchName), err)\t\treturn err\t}@@ -1219,11 +1211,7 @@ ARCHIVE_ENDPOINT_REPLACEMENT = """\t// claude-ops Fix F: POST /api/archive — a\t\tctx := r.Context()\t\tdo := func() error {-\t\t\t// claude-ops API-drift port (2026-06): current whatsmeow SendAppState has NO-\t\t\t// context.Context parameter — signature is SendAppState(patch-\t\t\t// appstate.PatchInfo). Dropped the leading ctx arg. (ctx is still used below-\t\t\t// for healLTHash.)-\t\t\treturn client.SendAppState(patch)+\t\t\t\t\t\treturn client.SendAppState(ctx, patch)\t\t}\t\terr = do()\t\t// claude-ops Fix L: treat both a local LTHash mismatch and a server-side 409@@ -1459,6 +1447,216 @@ if __name__ == "__main__":mcp.run(transport='stdio')'''+# ─── main.go Fix N: ctx-drift — add context.Background() to upstream call sites ─+# whatsmeow added context.Context parameters to sqlstore.New, GetFirstDevice,+# GetGroupInfo, and GetContact. These are pristine upstream call sites not covered+# by any other fix replacement block; patch them with targeted single-line swaps.++CTX_SQLSTORE_NEEDLE = '\tcontainer, err := sqlstore.New("sqlite3", "file:store/whatsapp.db?_foreign_keys=on", dbLog)\n'+CTX_SQLSTORE_REPLACEMENT = '\tcontainer, err := sqlstore.New(context.Background(), "sqlite3", "file:store/whatsapp.db?_foreign_keys=on", dbLog)\n'+CTX_SQLSTORE_SENTINEL = "sqlstore.New(context.Background(),"++CTX_GETFIRSTDEVICE_NEEDLE = "\tdeviceStore, err := container.GetFirstDevice()\n"+CTX_GETFIRSTDEVICE_REPLACEMENT = (+ "\tdeviceStore, err := container.GetFirstDevice(context.Background())\n"+)+CTX_GETFIRSTDEVICE_SENTINEL = "container.GetFirstDevice(context.Background())"++CTX_GETGROUPINFO_NEEDLE = "\t\t\tgroupInfo, err := client.GetGroupInfo(jid)\n"+CTX_GETGROUPINFO_REPLACEMENT = (+ "\t\t\tgroupInfo, err := client.GetGroupInfo(context.Background(), jid)\n"+)+CTX_GETGROUPINFO_SENTINEL = "client.GetGroupInfo(context.Background(),"++CTX_GETCONTACT_NEEDLE = "\t\tcontact, err := client.Store.Contacts.GetContact(jid)\n"+CTX_GETCONTACT_REPLACEMENT = (+ "\t\tcontact, err := client.Store.Contacts.GetContact(context.Background(), jid)\n"+)+CTX_GETCONTACT_SENTINEL = "client.Store.Contacts.GetContact(context.Background(),"+++# ─── main.go Fix M: media-retry fallback on 403/404/410 ──────────────────────+# WhatsApp returns HTTP 403/404/410 when a media's directPath has gone stale+# server-side. whatsmeow treats those as terminal, so larger media silently+# drops. The documented recovery is SendMediaRetryReceipt: ask the sender's phone+# to re-upload, yielding a FRESH directPath. Four hunks, each anchored on UPSTREAM+# code so they apply on a pristine lharries/whatsapp-mcp clone:+# M1: imports — add "sync" (retry registry mutex)+# M2: imports — add waMmsRetry proto (decrypt the retry notification result)+# M3: insert the media-retry registry + downloadWithRetry() before downloadMedia()+# M4: route the download through downloadWithRetry()+# M5: deliver *events.MediaRetry to the blocked downloader++# M1 — "sync" import (anchor on the pristine "strings" import line).+MEDIA_RETRY_SYNC_IMPORT_NEEDLE = '\t"strings"\n\t"syscall"\n'+MEDIA_RETRY_SYNC_IMPORT_REPLACEMENT = '\t"strings"\n\t"sync"\n\t"syscall"\n'+MEDIA_RETRY_SYNC_IMPORT_SENTINEL = '\t"sync"\n\t"syscall"\n'++# M2 — waMmsRetry proto import (anchor on the pristine types/events import line).+MEDIA_RETRY_PROTO_IMPORT_NEEDLE = '\t"go.mau.fi/whatsmeow/types/events"\n'+MEDIA_RETRY_PROTO_IMPORT_REPLACEMENT = (+ '\t"go.mau.fi/whatsmeow/types/events"\n'+ '\twaMmsRetry "go.mau.fi/whatsmeow/proto/waMmsRetry"\n'+)+MEDIA_RETRY_PROTO_IMPORT_SENTINEL = 'waMmsRetry "go.mau.fi/whatsmeow/proto/waMmsRetry"'++# M3 — the registry + downloadWithRetry() helper, inserted right before+# downloadMedia(). Anchors on the pristine downloadMedia signature.+MEDIA_RETRY_BLOCK_NEEDLE = "func downloadMedia(client *whatsmeow.Client, messageStore *MessageStore, messageID, chatJID string) (bool, string, string, string, error) {"+MEDIA_RETRY_BLOCK_REPLACEMENT = """// --- claude-ops Fix M: media-retry fallback ---------------------------------+// WhatsApp returns HTTP 403/404/410 when a media's directPath has gone stale+// server-side (common for larger media, or media that failed its at-receipt+// download). whatsmeow treats those as terminal. The documented recovery is+// SendMediaRetryReceipt: ask the sender's phone to re-upload, which yields a+// FRESH directPath we can download from. This registry lets the synchronous+// download path block on the async *events.MediaRetry response.+var (+\tmediaRetryMu sync.Mutex+\tmediaRetryChans = map[string][]chan *events.MediaRetry{}+)++func registerMediaRetry(messageID string) chan *events.MediaRetry {+\tch := make(chan *events.MediaRetry, 1)+\tmediaRetryMu.Lock()+\tmediaRetryChans[messageID] = append(mediaRetryChans[messageID], ch)+\tmediaRetryMu.Unlock()+\treturn ch+}++func deliverMediaRetry(evt *events.MediaRetry) {+\tmediaRetryMu.Lock()+\tchans := mediaRetryChans[evt.MessageID]+\tmediaRetryMu.Unlock()+\tfor _, ch := range chans {+\t\tselect {+\t\tcase ch <- evt:+\t\tdefault:+\t\t}+\t}+}++func unregisterMediaRetry(messageID string, ch chan *events.MediaRetry) {+\tmediaRetryMu.Lock()+\tdefer mediaRetryMu.Unlock()+\twaiters := mediaRetryChans[messageID]+\tfor i, w := range waiters {+\t\tif w == ch {+\t\t\tmediaRetryChans[messageID] = append(waiters[:i], waiters[i+1:]...)+\t\t\tbreak+\t\t}+\t}+\tif len(mediaRetryChans[messageID]) == 0 {+\t\tdelete(mediaRetryChans, messageID)+\t}+}++// downloadWithRetry calls client.Download; on a 403/404/410 it asks the sender's+// phone to re-upload (SendMediaRetryReceipt), waits for the fresh directPath, and+// retries once. Any other error (or a failed retry) is returned to the caller.+func downloadWithRetry(client *whatsmeow.Client, messageStore *MessageStore, messageID, chatJID string, downloader *MediaDownloader) ([]byte, error) {+\tdata, err := client.Download(context.Background(), downloader)+\tif err == nil {+\t\treturn data, nil+\t}+\tes := err.Error()+\tif !(strings.Contains(es, "403") || strings.Contains(es, "404") || strings.Contains(es, "410")) {+\t\treturn nil, err+\t}++\t// Reconstruct enough MessageInfo for the retry receipt.+\tvar senderStr string+\tvar ts int64+\tvar fromMe bool+\t_ = messageStore.db.QueryRow(+\t\t"SELECT COALESCE(sender,''), CAST(strftime('%s', timestamp) AS INTEGER), is_from_me FROM messages WHERE id = ? AND chat_jid = ?",+\t\tmessageID, chatJID,+\t).Scan(&senderStr, &ts, &fromMe)++\tchat, perr := types.ParseJID(chatJID)+\tif perr != nil {+\t\treturn nil, fmt.Errorf("media-retry: bad chat jid %q: %v (orig: %w)", chatJID, perr, err)+\t}+\tvar sender types.JID+\tswitch {+\tcase fromMe && client.Store.ID != nil:+\t\tsender = *client.Store.ID+\tcase senderStr != "":+\t\tif strings.Contains(senderStr, "@") {+\t\t\tsender, _ = types.ParseJID(senderStr)+\t\t} else if chat.Server == types.GroupServer {+\t\t\tsender = types.NewJID(senderStr, types.DefaultUserServer)+\t\t} else {+\t\t\tsender = types.NewJID(senderStr, chat.Server)+\t\t}+\tdefault:+\t\tsender = chat+\t}+\tinfo := &types.MessageInfo{+\t\tID: messageID,+\t\tTimestamp: time.Unix(ts, 0),+\t\tMessageSource: types.MessageSource{+\t\t\tChat: chat,+\t\t\tSender: sender,+\t\t\tIsFromMe: fromMe,+\t\t\tIsGroup: chat.Server == types.GroupServer,+\t\t},+\t}++\tch := registerMediaRetry(messageID)+\tdefer unregisterMediaRetry(messageID, ch)++\tfmt.Printf("Media download hit %s for %s — requesting re-upload via media-retry...\\n", es, messageID)+\tif rerr := client.SendMediaRetryReceipt(context.Background(), info, downloader.MediaKey); rerr != nil {+\t\treturn nil, fmt.Errorf("media-retry receipt failed: %v (orig: %w)", rerr, err)+\t}++\tselect {+\tcase evt := <-ch:+\t\tnotif, derr := whatsmeow.DecryptMediaRetryNotification(evt, downloader.MediaKey)+\t\tif derr != nil {+\t\t\treturn nil, fmt.Errorf("media-retry decrypt failed: %v (orig: %w)", derr, err)+\t\t}+\t\tif notif.GetResult() != waMmsRetry.MediaRetryNotification_SUCCESS {+\t\t\treturn nil, fmt.Errorf("media-retry not successful: %s (orig: %w)", notif.GetResult().String(), err)+\t\t}+\t\tdownloader.DirectPath = notif.GetDirectPath()+\t\tdownloader.URL = ""+\t\tfmt.Printf("Media-retry returned fresh directPath for %s — retrying download...\\n", messageID)+\t\treturn client.Download(context.Background(), downloader)+\tcase <-time.After(30 * time.Second):+\t\treturn nil, fmt.Errorf("media-retry timed out after 30s (orig: %w)", err)+\t}+}++func downloadMedia(client *whatsmeow.Client, messageStore *MessageStore, messageID, chatJID string) (bool, string, string, string, error) {"""+MEDIA_RETRY_BLOCK_SENTINEL = "claude-ops Fix M: media-retry fallback"++# M4 — route downloadMedia's actual download through downloadWithRetry.+# Pristine upstream calls the OLD one-arg client.Download(downloader); whatsmeow+# @latest (which the installer `go get -u`s) takes (context, downloader), and+# downloadWithRetry uses the two-arg form. Anchor on the pristine one-arg line.+MEDIA_RETRY_CALL_NEEDLE = "\tmediaData, err := client.Download(downloader)"+MEDIA_RETRY_CALL_REPLACEMENT = "\t// media-retry fallback on 403/404/410 (claude-ops Fix M)\n\tmediaData, err := downloadWithRetry(client, messageStore, messageID, chatJID, downloader)"+MEDIA_RETRY_CALL_SENTINEL = "mediaData, err := downloadWithRetry(client, messageStore"++# M5 — deliver *events.MediaRetry to the blocked downloader (anchor on the+# pristine HistorySync case in the AddEventHandler switch).+MEDIA_RETRY_EVENT_NEEDLE = """\t\tcase *events.HistorySync:+\t\t\t// Process history sync events+\t\t\thandleHistorySync(client, messageStore, v, logger)+"""+MEDIA_RETRY_EVENT_REPLACEMENT = """\t\tcase *events.HistorySync:+\t\t\t// Process history sync events+\t\t\thandleHistorySync(client, messageStore, v, logger)++\t\tcase *events.MediaRetry:+\t\t\t// Deliver re-upload notifications to any download blocked on a 403/404/410+\t\t\t// (claude-ops Fix M — media-retry fallback).+\t\t\tdeliverMediaRetry(v)+"""+MEDIA_RETRY_EVENT_SENTINEL = "claude-ops Fix M — media-retry fallback"++def replace_idempotent(p: pathlib.Path, needle: str, replacement: str, sentinel: str, label: str) -> bool:@@ -1637,6 +1835,69 @@ def main() -> int:ARCHIVE_ENDPOINT_SENTINEL,"Fix F: POST /api/archive endpoint",)+ changed_go |= replace_idempotent(+ main_go,+ CTX_SQLSTORE_NEEDLE,+ CTX_SQLSTORE_REPLACEMENT,+ CTX_SQLSTORE_SENTINEL,+ "Fix N: sqlstore.New ctx arg",+ )+ changed_go |= replace_idempotent(+ main_go,+ CTX_GETFIRSTDEVICE_NEEDLE,+ CTX_GETFIRSTDEVICE_REPLACEMENT,+ CTX_GETFIRSTDEVICE_SENTINEL,+ "Fix N: GetFirstDevice ctx arg",+ )+ changed_go |= replace_idempotent(+ main_go,+ CTX_GETGROUPINFO_NEEDLE,+ CTX_GETGROUPINFO_REPLACEMENT,+ CTX_GETGROUPINFO_SENTINEL,+ "Fix N: GetGroupInfo ctx arg",+ )+ changed_go |= replace_idempotent(+ main_go,+ CTX_GETCONTACT_NEEDLE,+ CTX_GETCONTACT_REPLACEMENT,+ CTX_GETCONTACT_SENTINEL,+ "Fix N: GetContact ctx arg",+ )+ changed_go |= replace_idempotent(+ main_go,+ MEDIA_RETRY_SYNC_IMPORT_NEEDLE,+ MEDIA_RETRY_SYNC_IMPORT_REPLACEMENT,+ MEDIA_RETRY_SYNC_IMPORT_SENTINEL,+ 'Fix M: add "sync" import (media-retry registry)',+ )+ changed_go |= replace_idempotent(+ main_go,+ MEDIA_RETRY_PROTO_IMPORT_NEEDLE,+ MEDIA_RETRY_PROTO_IMPORT_REPLACEMENT,+ MEDIA_RETRY_PROTO_IMPORT_SENTINEL,+ "Fix M: add waMmsRetry proto import",+ )+ changed_go |= replace_idempotent(+ main_go,+ MEDIA_RETRY_BLOCK_NEEDLE,+ MEDIA_RETRY_BLOCK_REPLACEMENT,+ MEDIA_RETRY_BLOCK_SENTINEL,+ "Fix M: media-retry registry + downloadWithRetry()",+ )+ changed_go |= replace_idempotent(+ main_go,+ MEDIA_RETRY_CALL_NEEDLE,+ MEDIA_RETRY_CALL_REPLACEMENT,+ MEDIA_RETRY_CALL_SENTINEL,+ "Fix M: route downloadMedia through downloadWithRetry()",+ )+ changed_go |= replace_idempotent(+ main_go,+ MEDIA_RETRY_EVENT_NEEDLE,+ MEDIA_RETRY_EVENT_REPLACEMENT,+ MEDIA_RETRY_EVENT_SENTINEL,+ "Fix M: deliver *events.MediaRetry to blocked downloader",+ )print(" whatsapp.py:")changed_py = replace_idempotent(diff --git a/claude-ops/scripts/whatsapp/enrich_media.py b/claude-ops/scripts/whatsapp/enrich_media.pynew file mode 100644index 00000000..049dad12--- /dev/null+++ b/claude-ops/scripts/whatsapp/enrich_media.py@@ -0,0 +1,470 @@+#!/usr/bin/env python3+"""WhatsApp media enricher — makes ALL media first-class in ops-inbox.++Generalizes the voice-note transcriber to every media type so that no audio,+video, image, or document message is ever silently dropped during inbox triage.+For each recent media message whose `content` is still empty, it:++ 1. downloads the media via the bridge /api/download endpoint+ (the bridge now self-heals 403/404/410 via SendMediaRetryReceipt — Fix I),+ 2. analyzes it by type:+ audio -> OpenAI whisper-1 -> "[voice] <transcript>"+ video -> ffmpeg keyframes + vision -> "[video] <visual> | speech: <whisper of audio track>"+ image -> OpenAI vision (gpt-4o-mini) -> "[image] <description + any on-screen text>"+ document -> pdftotext / strings (best-eff)-> "[document] <filename>: <text excerpt>"+ 3. writes the result back into messages.content — ONLY where content is still+ empty, so it never clobbers real text and is fully idempotent.++The fuller analysis (untruncated) is also written to a sidecar JSON cache at+store/<chat>/<msgid>.enrich.json so an agent can read the complete description+later without re-billing the vision/whisper APIs.++Run by wa-inbox-fresh.sh (pre-scan) and the whatsapp-enrich.timer so new media+is enriched automatically before any inbox scan reads it.++Guards (per global cost-leak + no-stacking rules):+ * mkdir-based single-instance lock+ * --max cap per run (rate-floor on metered APIs)+ * idempotent: only ever touches empty-content media rows+ * per-type size ceilings; vision frames capped per video++Env: OPENAI_API_KEY (from ~/.config/systemd/env/mcp-secrets.env).+ WHATSAPP_BRIDGE_DB (default ~/.local/share/whatsapp-mcp/whatsapp-bridge/store/messages.db)+Args: --days N (lookback, default 2) --max N (cap per run, default 40)+ --types audio,video,image,document (default all)+"""++import argparse+import base64+import json+import os+import shutil+import signal+import sqlite3+import subprocess+import sys+import tempfile+import time+import urllib.request++DB = os.path.expanduser(+ os.environ.get(+ "WHATSAPP_BRIDGE_DB",+ "~/.local/share/whatsapp-mcp/whatsapp-bridge/store/messages.db",+ )+)+BRIDGE = "http://127.0.0.1:8080/api/download"+LOCKDIR = "/tmp/.wa-enrich.lock"+PIDFILE = os.path.join(LOCKDIR, "pid")++VISION_MODEL = os.environ.get("WA_ENRICH_VISION_MODEL", "gpt-4o-mini")+MAX_VIDEO_BYTES = int(+ os.environ.get("WA_ENRICH_MAX_VIDEO_BYTES", str(200 * 1024 * 1024))+)+VIDEO_FRAMES = int(os.environ.get("WA_ENRICH_VIDEO_FRAMES", "4"))+++# ---------------------------------------------------------------- lock ------+def _pid_alive(pid):+ try:+ os.kill(pid, 0)+ return True+ except OSError:+ return False+++def release_lock(signum=None, frame=None):+ shutil.rmtree(LOCKDIR, ignore_errors=True)+ if signum is not None:+ sys.exit(128 + signum)+++def acquire_lock():+ try:+ os.mkdir(LOCKDIR)+ except FileExistsError:+ try:+ with open(PIDFILE) as f:+ old = int(f.read().strip())+ if _pid_alive(old):+ return False+ except (FileNotFoundError, ValueError, OSError):+ pass+ shutil.rmtree(LOCKDIR, ignore_errors=True)+ try:+ os.mkdir(LOCKDIR)+ except FileExistsError:+ return False+ with open(PIDFILE, "w") as f:+ f.write(str(os.getpid()))+ return True+++# ------------------------------------------------------------- helpers ------+def download(msg_id, chat_jid):+ body = json.dumps({"message_id": msg_id, "chat_jid": chat_jid}).encode()+ req = urllib.request.Request(+ BRIDGE, data=body, headers={"Content-Type": "application/json"}+ )+ with urllib.request.urlopen(req, timeout=120) as r:+ d = json.load(r)+ return d.get("path") if d.get("success") else None+++def _openai_chat(key, messages, max_tokens=400):+ payload = json.dumps(+ {"model": VISION_MODEL, "messages": messages, "max_tokens": max_tokens}+ ).encode()+ out = subprocess.run(+ [+ "curl",+ "-s",+ "-m",+ "180",+ "https://api.openai.com/v1/chat/completions",+ "-H",+ f"Authorization: Bearer {key}",+ "-H",+ "Content-Type: application/json",+ "-d",+ "@-",+ ],+ input=payload.decode(),+ capture_output=True,+ text=True,+ timeout=200,+ )+ try:+ return json.loads(out.stdout)["choices"][0]["message"]["content"].strip()+ except Exception:+ return None+++def whisper(path, key):+ out = subprocess.run(+ [+ "curl",+ "-s",+ "-m",+ "180",+ "https://api.openai.com/v1/audio/transcriptions",+ "-H",+ f"Authorization: Bearer {key}",+ "-F",+ f"file=@{path}",+ "-F",+ "model=whisper-1",+ ],+ capture_output=True,+ text=True,+ timeout=200,+ )+ try:+ return json.loads(out.stdout).get("text", "").strip()+ except Exception:+ return None+++def _ffprobe_duration(path):+ try:+ out = subprocess.run(+ [+ "ffprobe",+ "-v",+ "error",+ "-show_entries",+ "format=duration",+ "-of",+ "default=nw=1:nk=1",+ path,+ ],+ capture_output=True,+ text=True,+ timeout=30,+ )+ return float(out.stdout.strip())+ except Exception:+ return 0.0+++def _extract_frames(path, n, workdir):+ dur = _ffprobe_duration(path)+ frames = []+ if dur <= 0:+ # single grab at 1s+ fp = os.path.join(workdir, "f0.jpg")+ subprocess.run(+ [+ "ffmpeg",+ "-y",+ "-ss",+ "1",+ "-i",+ path,+ "-frames:v",+ "1",+ "-vf",+ "scale=512:-1",+ fp,+ ],+ capture_output=True,+ timeout=60,+ )+ if os.path.exists(fp):+ frames.append(fp)+ return frames+ for i in range(n):+ t = dur * (i + 0.5) / n+ fp = os.path.join(workdir, f"f{i}.jpg")+ subprocess.run(+ [+ "ffmpeg",+ "-y",+ "-ss",+ f"{t:.2f}",+ "-i",+ path,+ "-frames:v",+ "1",+ "-vf",+ "scale=512:-1",+ fp,+ ],+ capture_output=True,+ timeout=60,+ )+ if os.path.exists(fp):+ frames.append(fp)+ return frames+++def _extract_audio(path, workdir):+ ap = os.path.join(workdir, "audio.mp3")+ r = subprocess.run(+ [+ "ffmpeg",+ "-y",+ "-i",+ path,+ "-vn",+ "-ac",+ "1",+ "-ar",+ "16000",+ "-b:a",+ "64k",+ ap,+ ],+ capture_output=True,+ timeout=180,+ )+ return ap if os.path.exists(ap) and os.path.getsize(ap) > 0 else None+++def _b64_data_uri(path):+ with open(path, "rb") as f:+ return "data:image/jpeg;base64," + base64.b64encode(f.read()).decode()+++# ----------------------------------------------------------- analyzers ------+def analyze_image(path, key):+ desc = _openai_chat(+ key,+ [+ {+ "role": "user",+ "content": [+ {+ "type": "text",+ "text": "Describe this image concisely for someone triaging a WhatsApp "+ "inbox. Note the main subject, context, and transcribe any visible "+ "text verbatim. 1-3 sentences.",+ },+ {"type": "image_url", "image_url": {"url": _b64_data_uri(path)}},+ ],+ }+ ],+ )+ return f"[image] {desc}" if desc else None+++def analyze_video(path, key):+ with tempfile.TemporaryDirectory() as wd:+ frames = _extract_frames(path, VIDEO_FRAMES, wd)+ visual = None+ if frames:+ content = [+ {+ "type": "text",+ "text": "These are sequential keyframes from a WhatsApp video. In 2-4 "+ "sentences describe what the video shows (subject, setting, "+ "motion/edit style), and transcribe any on-screen text verbatim.",+ }+ ]+ for fp in frames:+ content.append(+ {"type": "image_url", "image_url": {"url": _b64_data_uri(fp)}}+ )+ visual = _openai_chat(+ key, [{"role": "user", "content": content}], max_tokens=500+ )+ speech = None+ ap = _extract_audio(path, wd)+ if ap:+ speech = whisper(ap, key)+ parts = []+ if visual:+ parts.append(visual)+ if speech:+ parts.append(f"speech/audio: {speech}")+ if not parts:+ return None+ return "[video] " + " | ".join(parts)+++def analyze_audio(path, key):+ text = whisper(path, key)+ return f"[voice] {text}" if text else None+++def analyze_document(path, key, filename):+ text = ""+ if path.lower().endswith(".pdf") and shutil.which("pdftotext"):+ out = subprocess.run(+ ["pdftotext", "-l", "3", path, "-"],+ capture_output=True,+ text=True,+ timeout=60,+ )+ text = out.stdout.strip()+ if not text:+ try:+ with open(path, "rb") as f:+ raw = f.read(4096)+ text = "".join(+ ch+ for ch in raw.decode("utf-8", "ignore")+ if ch.isprintable() or ch in "\n\t "+ )+ except Exception:+ text = ""+ excerpt = " ".join(text.split())[:600]+ if not excerpt:+ return f"[document] {filename}"+ return f"[document] {filename}: {excerpt}"+++ANALYZERS = {+ "audio": lambda p, k, fn: analyze_audio(p, k),+ "video": lambda p, k, fn: analyze_video(p, k),+ "image": lambda p, k, fn: analyze_image(p, k),+ "document": lambda p, k, fn: analyze_document(p, k, fn),+}+++# ---------------------------------------------------------------- main ------+def main():+ ap = argparse.ArgumentParser()+ ap.add_argument("--days", type=int, default=2)+ ap.add_argument("--max", type=int, default=40)+ ap.add_argument("--types", default="audio,video,image,document")+ args = ap.parse_args()++ want = [t.strip() for t in args.types.split(",") if t.strip() in ANALYZERS]+ if not want:+ print("enrich: no valid --types", flush=True)+ sys.exit(1)++ key = os.environ.get("OPENAI_API_KEY")+ if not key:+ print("enrich: OPENAI_API_KEY not set — abort", flush=True)+ sys.exit(1)+ if not os.path.isfile(DB):+ print(f"enrich: db not found {DB} — abort", flush=True)+ sys.exit(2)+ if not acquire_lock():+ print("enrich: another run holds the lock — exit", flush=True)+ sys.exit(0)++ signal.signal(signal.SIGTERM, release_lock)+ signal.signal(signal.SIGINT, release_lock)++ try:+ con = sqlite3.connect(DB, timeout=15)+ con.execute("PRAGMA busy_timeout=8000;")+ placeholders = ",".join("?" for _ in want)+ rows = con.execute(+ f"SELECT id, chat_jid, is_from_me, media_type, COALESCE(filename,'') "+ f"FROM messages "+ f"WHERE media_type IN ({placeholders}) "+ f"AND (content='' OR content IS NULL) "+ f"AND timestamp >= datetime('now', ?) "+ f"ORDER BY timestamp DESC LIMIT ?",+ (*want, f"-{args.days} days", args.max),+ ).fetchall()+ if not rows:+ print(+ f"enrich: nothing to do (last {args.days}d, types={','.join(want)})",+ flush=True,+ )+ return+ print(+ f"enrich: {len(rows)} media item(s) (last {args.days}d, cap {args.max})",+ flush=True,+ )+ ok = fail = 0+ for i, (mid, jid, mine, mtype, fname) in enumerate(rows, 1):+ try:+ path = download(mid, jid)+ if not path or not os.path.exists(path):+ fail += 1+ print(f" [{i}] {mid[:8]} {mtype} download-fail", flush=True)+ continue+ if mtype == "video" and os.path.getsize(path) > MAX_VIDEO_BYTES:+ fail += 1+ print(f" [{i}] {mid[:8]} video too large, skip", flush=True)+ continue+ result = ANALYZERS[mtype](path, key, fname)+ if not result:+ fail += 1+ print(f" [{i}] {mid[:8]} {mtype} analyze-fail", flush=True)+ continue+ # truncated tag into content (idempotent); full text into sidecar+ content_val = result if len(result) <= 1000 else result[:997] + "..."+ con.execute(+ "UPDATE messages SET content=? WHERE id=? AND chat_jid=? "+ "AND (content='' OR content IS NULL)",+ (content_val, mid, jid),+ )+ con.commit()+ try:+ side = os.path.join(os.path.dirname(path), f"{mid}.enrich.json")+ with open(side, "w") as f:+ json.dump(+ {+ "id": mid,+ "chat_jid": jid,+ "media_type": mtype,+ "full": result,+ },+ f,+ )+ except Exception:+ pass+ ok += 1+ print(+ f" [{i}] {mid[:8]} {mtype} ({'me' if mine else 'them'}) ok",+ flush=True,+ )+ except Exception as e:+ fail += 1+ print(f" [{i}] {mid[:8]} {mtype} err {e}", flush=True)+ time.sleep(0.2)+ con.close()+ print(f"enrich: done — {ok} enriched, {fail} failed", flush=True)+ finally:+ release_lock()+++if __name__ == "__main__":+ main()diff --git a/claude-ops/scripts/whatsapp/systemd/whatsapp-enrich.service b/claude-ops/scripts/whatsapp/systemd/whatsapp-enrich.servicenew file mode 100644index 00000000..62d34366--- /dev/null+++ b/claude-ops/scripts/whatsapp/systemd/whatsapp-enrich.service@@ -0,0 +1,21 @@+[Unit]+Description=WhatsApp media enricher — fills empty video/image/document rows with vision+Whisper descriptions (oneshot)+Documentation=https://github.com/lharries/whatsapp-mcp+After=whatsapp-bridge.service+Requires=whatsapp-bridge.service+PartOf=whatsapp-bridge.service++[Service]+Type=oneshot+# Batch may run many vision/Whisper calls; disable the default ~90s start timeout.+TimeoutStartSec=0+# Relative media paths from /api/download resolve against the bridge cwd.+WorkingDirectory=__INSTALL_DIR__/whatsapp-bridge+# OPENAI_API_KEY (and other secrets) for the metered vision/Whisper API.+EnvironmentFile=-%h/.config/systemd/env/mcp-secrets.env+# Wait up to 10s for the bridge :8080 before downloading media.+ExecStartPre=/bin/sh -c 'for i in 1 2 3 4 5; do ss -ltn 2>/dev/null | grep -q ":8080" && exit 0; sleep 2; done; exit 0'+ExecStart=/usr/bin/python3 __INSTALL_DIR__/transcriber/enrich_media.py --types video,image,document --days 2 --max 40+StandardOutput=journal+StandardError=journal+SyslogIdentifier=whatsapp-enrichdiff --git a/claude-ops/scripts/whatsapp/systemd/whatsapp-enrich.timer b/claude-ops/scripts/whatsapp/systemd/whatsapp-enrich.timernew file mode 100644index 00000000..19e8eade--- /dev/null+++ b/claude-ops/scripts/whatsapp/systemd/whatsapp-enrich.timer@@ -0,0 +1,14 @@+[Unit]+Description=Enrich new WhatsApp video/image/document media every 10 min + on boot (ops-inbox integration)+Documentation=https://github.com/lharries/whatsapp-mcp++[Timer]+# First fire 3 min after boot/session start (after the bridge connects + backfills), then every 10 min.+OnBootSec=3min+OnUnitActiveSec=10min+AccuracySec=30s+Persistent=true+Unit=whatsapp-enrich.service++[Install]+WantedBy=timers.targetdiff --git a/claude-ops/skills/ops-inbox/SKILL.md b/claude-ops/skills/ops-inbox/SKILL.mdindex 4c397899..30fb3c5f 100644--- a/claude-ops/skills/ops-inbox/SKILL.md+++ b/claude-ops/skills/ops-inbox/SKILL.md@@ -97,6 +97,8 @@ Before executing, load available context:**0c. Voice notes are first-class.** Incoming voice notes (`media_type='audio'`, empty `content`) are auto-transcribed into `content` as `[voice] <text>` by the `whatsapp-transcribe.timer` (systemd-user, every 10 min, OpenAI `whisper-1`) — and `wa-inbox-fresh.sh` triggers a transcribe pass on every scan. So a voice note shows up in NEEDS_REPLY / thread scans exactly like a text message; treat a `[voice] …` body as the sender's words. Transcription is idempotent (only ever fills empty audio rows, never clobbers real text) and capped per run, so it never re-bills or stacks.+ **0c-bis. ALL media is now first-class, not just voice.** Beyond voice→`[voice]` (transcribe above), incoming **video / image / document** media (empty `content`) is auto-enriched into `content` as `[video] …` / `[image] …` / `[document] …` by `transcriber/enrich_media.py` (vision for stills/video frames + Whisper for any audio track) on the `whatsapp-enrich.timer` (systemd-user, every 10 min) — and `wa-inbox-fresh.sh` queues an enrich pass on every scan. So an image, clip, or PDF shows up in NEEDS_REPLY / thread scans with a real, readable body, exactly like text. Enrichment is idempotent (only fills empty media rows) and capped per run. The bridge also **self-heals media that 403/404/410s** (stale `directPath`, common for larger media) by asking the sender's phone to re-upload via `SendMediaRetryReceipt` (`apply-patches.py` Fix M), so large media never silently drops.+**0d. The scan engine self-refreshes + self-reconciles on EVERY run — this is automatic, you do not orchestrate it.** `bin/ops-inbox-scan` (the primary classifier, step "Scan engine" below) now does the refresh/pull itself, BLOCKING and bounded, before it classifies — so the data is converged by the time you read its JSON, regardless of whether the background `ops-inbox-autosync` hook has finished. On each invocation the scan:- **Refreshes (frontfill/backfill):** if the bridge is reachable on `:8080`, it fires `POST /api/backfill` + `link_contacts.py`, then **waits (bounded ~18s) for the newest stored message timestamp to stop advancing** so the classify pass reads a settled store. This is the blocking guarantee the background hook alone does NOT give. Skip with `OIS_NO_REFRESH=1` (set automatically on repeat calls in one session to avoid re-waiting).- **Reconciles outbound sends (Sam directive 2026-06-05 "include all things I sent to all people"):** it reads the bridge's outbound-send journal (`journalctl --user -u whatsapp-bridge.service`, or the bridge log file on non-systemd hosts) into a `{recipient_jid → latest_send_epoch}` map, and **demotes any NEEDS_REPLY thread whose last inbound is older than a send to any of that person's JIDs** (`reconciled` flag set, moved to WAITING). This catches replies that went out via `/api/send` or a phone send that has not yet landed in `messages.db` — the single most common false-NEEDS_REPLY. Only epoch-stamped send lines drive demotion (a send that genuinely predates the inbound never demotes).@@ -195,7 +197,7 @@ lsof -i :8080 | grep -q LISTEN && echo "bridge up" || journalctl --user -u whats```bashbash "$CLAUDE_PLUGIN_ROOT/scripts/install-whatsapp-bridge-linux.sh" --wa-phone <E.164>```-This clones lharries/whatsapp-mcp into `~/.local/share/whatsapp-mcp`, applies the in-repo claude-ops patches (Fix A/B pair-phone hardening, auto-backfill on Connected, `POST /api/backfill` REST endpoint, crash-safe `requestHistorySync`, Python LID↔phone↔contact resolver), drops the systemd-user units (`whatsapp-bridge.service`, `whatsapp-backfill.{service,timer}`, `whatsapp-transcribe.{service,timer}`), installs the voice-note transcriber (`transcriber/transcribe_voice_notes.py`) and the pre-scan freshness gate (`~/bin/wa-inbox-fresh.sh`), enables linger, and emits the pairing code via `journalctl --user -u whatsapp-bridge -f`. Idempotent: re-running is safe and updates patches in place. Pass `--no-transcribe-timer` to skip voice-note transcription. The transcribe service reads `OPENAI_API_KEY` from `~/.config/systemd/env/mcp-secrets.env`.+This clones lharries/whatsapp-mcp into `~/.local/share/whatsapp-mcp`, applies the in-repo claude-ops patches (Fix A/B pair-phone hardening, auto-backfill on Connected, `POST /api/backfill` REST endpoint, crash-safe `requestHistorySync`, Python LID↔phone↔contact resolver), drops the systemd-user units (`whatsapp-bridge.service`, `whatsapp-backfill.{service,timer}`, `whatsapp-transcribe.{service,timer}`), installs the voice-note transcriber (`transcriber/transcribe_voice_notes.py`), the media enricher (`transcriber/enrich_media.py`, with `whatsapp-enrich.{service,timer}`) and the pre-scan freshness gate (`~/bin/wa-inbox-fresh.sh`), enables linger, and emits the pairing code via `journalctl --user -u whatsapp-bridge -f`. Idempotent: re-running is safe and updates patches in place. Pass `--no-transcribe-timer` to skip voice-note transcription, `--no-enrich-timer` to skip video/image/document enrichment. The transcribe and enrich services read `OPENAI_API_KEY` from `~/.config/systemd/env/mcp-secrets.env`. The media-retry self-heal (Fix M) is part of the bridge patch set, no extra flag.**MCP tools** (use these instead of any wacli CLI command):