Write HTML. Render video. Built for agents.
JavaScript aiai-agentsanimationclaude-codeffmpegframeworkgsaphtmlhtml-to-videomcpmotion-graphicspuppeteerrenderingtypescriptvideo-editor
feat(cli): send missing features and host-app friction as ratingless feedback (#5140)
Miguel Ángel80c2547eac05
diff --git a/docs/guides/feedback.mdx b/docs/guides/feedback.mdxindex 2aaead6282..3d823288f7 100644--- a/docs/guides/feedback.mdx+++ b/docs/guides/feedback.mdx@@ -21,6 +21,12 @@ npx hyperframes feedback --rating 7 --comment "The render finished, but the captThe rating is from 0 to 10. The comment is optional, but a specific example is usually more useful than a score alone.+A comment with no rating is a report with no score, for something you needed that HyperFrames cannot do yet. It never counts in the rating:++```bash+npx hyperframes feedback --comment "MISSING FEATURE: trim a clip from the timeline | WORKAROUND: none"+```+## Report a reproducible bugUse `--file-issue` when the project can be shared publicly:diff --git a/docs/packages/cli.mdx b/docs/packages/cli.mdxindex d69e1638f7..b3e218930b 100644--- a/docs/packages/cli.mdx+++ b/docs/packages/cli.mdx@@ -1248,10 +1248,11 @@ the CLI attaches your email (below).npx hyperframes feedback --rating 10npx hyperframes feedback --rating 7 --comment "render succeeded but GSAP timeline didn't animate"npx hyperframes feedback --rating 3 --comment "GSAP timeline froze on seek" --file-issue+npx hyperframes feedback --comment "MISSING FEATURE: trim a clip from the timeline | WORKAROUND: none"```-`--rating` is required, 0–10. `--comment` adds free text. `--source person`-or `--source agent` says who wrote a rating report (a `--search-miss` report+`--rating` is 0–10. `--comment` adds free text; a `--comment` with no `--rating` is a report with no score (a missing feature, or friction in the app that launched the CLI), kept out of the rating metric. `--source person`+or `--source agent` says who wrote a rating or comment report (a `--search-miss` reportcarries neither), so the team can tell a person's feedback from an agent's. `--file-issue`also opens a GitHub issue, `--dir` picks the project published as its repro(default: current directory), and `--yes, -y` skips the consent prompt fordiff --git a/packages/cli/src/commands/coreSkillContent.test.ts b/packages/cli/src/commands/coreSkillContent.test.tsindex c8eab7ab8d..f6223d579b 100644--- a/packages/cli/src/commands/coreSkillContent.test.ts+++ b/packages/cli/src/commands/coreSkillContent.test.ts@@ -46,6 +46,15 @@ describe("hyperframes-core contract docs", () => {expect(renderReference).toContain("WORKAROUND:");});+ it("sends missing features and host-app friction as ratingless comments", () => {+ const skill = read("skills", "hyperframes-cli", "SKILL.md");++ expect(skill).toContain(+ 'npx hyperframes feedback --comment "MISSING FEATURE: <what the person asked for, in their words, with names, clients, figures and paths left out> | WORKAROUND:',+ );+ expect(skill).toMatch(/HYPERFRAMES_CLIENT[^\n]*npx hyperframes feedback --comment "HOST APP:/);+ });+it("mandates a composition-structure block for visual-defect feedback", () => {const skill = read("skills", "hyperframes-cli", "SKILL.md");const renderReference = read("skills", "hyperframes-cli", "references", "preview-render.md");diff --git a/packages/cli/src/commands/feedback.test.ts b/packages/cli/src/commands/feedback.test.tsindex 98688b11c9..4ef4814272 100644--- a/packages/cli/src/commands/feedback.test.ts+++ b/packages/cli/src/commands/feedback.test.ts@@ -8,8 +8,10 @@ const mocks = vi.hoisted(() => ({}));vi.mock("../telemetry/client.js", () => ({ shouldTrack: () => true, flush: mocks.flush }));const trackRenderFeedback = vi.hoisted(() => vi.fn());+const trackFeedbackComment = vi.hoisted(() => vi.fn());vi.mock("../telemetry/events.js", () => ({trackCatalogSearchMiss: vi.fn(),+ trackFeedbackComment,trackRenderFeedback,}));vi.mock("../telemetry/feedback.js", () => ({ getDoctorSummary: async () => "os=test" }));@@ -58,3 +60,39 @@ it("refuses a source that is neither person nor agent, and sends nothing", asyncawait expect(feedback.run?.({ args: { rating: "6", source: "bot" } } as never)).rejects.toThrow();expect(mocks.submitFeedback).not.toHaveBeenCalled();});++it("sends a comment with no rating as its own report, never as a rating", async () => {+ vi.spyOn(console, "log").mockImplementation(() => {});+ trackRenderFeedback.mockClear();+ const comment = "MISSING FEATURE: trim a clip | WORKAROUND: none";+ const { default: feedback } = await import("./feedback.js");+ await feedback.run?.({ args: { comment, source: "agent" } } as never);+ expect(trackRenderFeedback).not.toHaveBeenCalled();+ expect(trackFeedbackComment).toHaveBeenLastCalledWith(expect.objectContaining({ comment }));+ expect(mocks.submitFeedback).toHaveBeenLastCalledWith(+ expect.objectContaining({ rating: undefined, comment }),+ );+});++it.each([+ ["neither a rating nor a comment", {}],+ ["a blank comment with no rating", { comment: " " }],+ ["--file-issue with no rating", { comment: "MISSING FEATURE: x", "file-issue": true }],+])("refuses %s, and sends nothing", async (_, args) => {+ vi.spyOn(console, "error").mockImplementation(() => {});+ mocks.submitFeedback.mockClear();+ const { default: feedback } = await import("./feedback.js");+ await expect(feedback.run?.({ args } as never)).rejects.toThrow();+ expect(mocks.submitFeedback).not.toHaveBeenCalled();+});++it("keeps the rating error for an unreadable rating, so a retry keeps the rating", async () => {+ const errors = vi.spyOn(console, "error").mockImplementation(() => {});+ const { default: feedback } = await import("./feedback.js");+ await expect(+ feedback.run?.({ args: { rating: "8/10", comment: "x" } } as never),+ ).rejects.toThrow();+ expect(String(errors.mock.calls.at(-1)?.[0])).toContain(+ "Rating must be an integer between 0 and 10",+ );+});diff --git a/packages/cli/src/commands/feedback.ts b/packages/cli/src/commands/feedback.tsindex fdedeb1387..6dcaa926cf 100644--- a/packages/cli/src/commands/feedback.ts+++ b/packages/cli/src/commands/feedback.ts@@ -5,7 +5,11 @@ import { defineCommand } from "citty";import * as clack from "@clack/prompts";import open from "open";import type { Example } from "./_examples.js";-import { trackCatalogSearchMiss, trackRenderFeedback } from "../telemetry/events.js";+import {+ trackCatalogSearchMiss,+ trackFeedbackComment,+ trackRenderFeedback,+} from "../telemetry/events.js";import { shouldTrack } from "../telemetry/client.js";import { getDoctorSummary } from "../telemetry/feedback.js";import { readConfig, type RecentRenderRecord } from "../telemetry/config.js";@@ -21,6 +25,10 @@ import { lintFeedbackComment, type FeedbackLintInput } from "../utils/feedbackLiexport const examples: Example[] = [["Submit render feedback", 'hyperframes feedback --rating 8 --comment "fast but font missing"'],["Quick rating only", "hyperframes feedback --rating 10"],+ [+ "Report something you needed that is missing (no rating)",+ 'hyperframes feedback --comment "MISSING FEATURE: trim a clip from the timeline | WORKAROUND: none"',+ ],["Report a catalog gap after a fruitless search",'hyperframes feedback --search-miss "typewriter that deletes" --wanted "text that types then backspaces"',@@ -38,8 +46,8 @@ function normalizeComment(raw?: string): string | undefined {/*** Compact PostHog join keys appended to the environment string that rides* along with the forwarded report (and therefore lands verbatim in the wild- * feedback channel): `fid` = this submission's PostHog `cli_render_feedback`- * `feedback_id`; `tid` = the install's telemetry distinct_id; `renders` =+ * feedback channel): `fid` = the PostHog `feedback_id` (`cli_render_feedback` or+ * `cli_feedback_comment`); `tid` = the install's telemetry distinct_id; `renders` =* recent `render_job_id`s (newest last, `!` suffix = the render failed).* Together they turn a wild report into an exact telemetry lookup instead of* a hardware-fingerprint hunt.@@ -155,18 +163,62 @@ async function fileGithubIssue(opts: {await openAndPrintIssue(url);}+// A comment with no rating is its own report (a missing feature, host-app+// friction): it scores nothing, so it must stay out of the rating metric.+function reportRating(+ raw: string | undefined,+ comment: string | undefined,+ fileIssue: boolean,+): number | undefined {+ const rating = raw === undefined ? undefined : parseFeedbackRating(raw);+ if (rating === null) {+ console.error(c.error("Rating must be an integer between 0 and 10"));+ failCommand();+ }+ if (rating === undefined && !comment?.trim()) {+ console.error(c.error("Give a --rating (an integer from 0 to 10), a --comment, or both"));+ failCommand();+ }+ if (rating === undefined && fileIssue) {+ console.error(c.error("--file-issue needs a --rating"));+ failCommand();+ }+ return rating;+}++function trackFeedback(report: {+ rating: number | undefined;+ comment: string | undefined;+ doctorSummary: string;+ feedbackId: string;+ recentRenderIds: string[] | undefined;+}): void {+ const { rating, comment, ...keys } = report;+ if (rating === undefined) {+ if (comment) trackFeedbackComment({ comment, ...keys });+ return;+ }+ // Soft-warn (never blocks) when the comment for a non-clean report is+ // missing the mandated reproduction-packet markers. Prints before the+ // submission ack so the reporter sees the nudge while their run is fresh.+ printFeedbackLintWarnings({ rating, comment });+ // The standalone command runs separately from `render`, so it has no real+ // elapsed time to report. Omit it rather than recording a fake duration.+ trackRenderFeedback({ rating, comment, ...keys });+}+export default defineCommand({meta: { name: "feedback", description: "Submit feedback about your experience" },args: {rating: {type: "string",- // Required for a rating report, but --search-miss is a different report- // with no rating to give, so the check moved into the run body.+ // Optional: --search-miss and a lone --comment are reports with no+ // rating to give, so the check lives in the run body.description: "Likelihood to recommend (0=not likely, 10=extremely likely)",},comment: {type: "string",- description: "Optional details about your experience",+ description: "Details about your experience; alone, a report with no rating",},source: {type: "string",@@ -220,13 +272,8 @@ export default defineCommand({return;}- // `--rating` is no longer required at the arg level, so an absent one- // reaches here as undefined rather than being rejected by the parser.- const rating = args.rating === undefined ? null : parseFeedbackRating(args.rating);- if (rating === null) {- console.error(c.error("Rating must be an integer between 0 and 10"));- failCommand();- }+ const comment = normalizeComment(args.comment);+ const rating = reportRating(args.rating, comment, args["file-issue"] === true);const source = parseFeedbackSource(args.source);if (source === null) {@@ -239,7 +286,6 @@ export default defineCommand({return;}- const comment = normalizeComment(args.comment);const doctorSummary = await getDoctorSummary();// Join keys tying this report to the install's PostHog rows — see@@ -254,14 +300,7 @@ export default defineCommand({});const envWithJoinKeys = doctorSummary ? `${doctorSummary} ${joinKeys}` : joinKeys;- // Soft-warn (never blocks) when the comment for a non-clean report is- // missing the mandated reproduction-packet markers. Prints before the- // submission ack so the reporter sees the nudge while their run is fresh.- printFeedbackLintWarnings({ rating, comment });-- // The standalone command runs separately from `render`, so it has no real- // elapsed time to report. Omit it rather than recording a fake duration.- trackRenderFeedback({+ trackFeedback({rating,comment,doctorSummary,@@ -281,7 +320,7 @@ export default defineCommand({email: feedbackEmail(),});- if (args["file-issue"] === true) {+ if (args["file-issue"] === true && rating !== undefined) {await fileGithubIssue({rating,comment,diff --git a/packages/cli/src/telemetry/events.test.ts b/packages/cli/src/telemetry/events.test.tsindex 7930c84772..3db004d99f 100644--- a/packages/cli/src/telemetry/events.test.ts+++ b/packages/cli/src/telemetry/events.test.ts@@ -36,6 +36,7 @@ const {trackCliError,trackFigmaImport,trackRenderFeedback,+ trackFeedbackComment,trackRenderPreflightRejected,trackAuthLoginStarted,trackAuthLoginCompleted,@@ -1069,6 +1070,17 @@ describe("render telemetry events", () => {});});+describe("trackFeedbackComment", () => {+ it("is its own event with no rating, so it never counts in the rating metric", () => {+ trackEvent.mockClear();+ trackFeedbackComment({ comment: "MISSING FEATURE: trim", feedbackId: "f1" });++ const [name, props] = trackEvent.mock.calls[0] as [string, Record<string, unknown>];+ expect(name).toBe("cli_feedback_comment");+ expect(props).toEqual({ comment: "MISSING FEATURE: trim", feedback_id: "f1" });+ });+});+describe("trackRenderFeedback", () => {beforeEach(() => {trackEvent.mockClear();diff --git a/packages/cli/src/telemetry/events.ts b/packages/cli/src/telemetry/events.tsindex 923631c959..32f890beeb 100644--- a/packages/cli/src/telemetry/events.ts+++ b/packages/cli/src/telemetry/events.ts@@ -993,6 +993,22 @@ export function trackSkillsInstallSkipped(props: { reason: string }): void {trackEvent("cli skill install skipped", { reason: props.reason });}+export function trackFeedbackComment(props: {+ comment: string;+ doctorSummary?: string;+ feedbackId?: string;+ recentRenderIds?: string[];+}): void {+ trackEvent("cli_feedback_comment", {+ comment: props.comment,+ ...(props.doctorSummary ? { doctor_summary: props.doctorSummary } : {}),+ ...(props.feedbackId ? { feedback_id: props.feedbackId } : {}),+ ...(props.recentRenderIds?.length+ ? { recent_render_ids: props.recentRenderIds.join(",") }+ : {}),+ });+}+export function trackRenderFeedback(props: {rating: number;renderDurationMs?: number;@@ -1001,7 +1017,8 @@ export function trackRenderFeedback(props: {/*** Join key shared with the forwarded feedback report (Slack/backend): the* same uuid rides in the report's env string as `fid=…`, so a wild report- * resolves to exactly one PostHog `cli_render_feedback` event and vice versa.+ * resolves to exactly one PostHog `cli_render_feedback` or+ * `cli_feedback_comment` event and vice versa.*/feedbackId?: string;/** render_job_id values of this install's recent renders (newest last). */diff --git a/packages/cli/src/utils/submitFeedback.test.ts b/packages/cli/src/utils/submitFeedback.test.tsindex 61d2d5d7b5..021f98383f 100644--- a/packages/cli/src/utils/submitFeedback.test.ts+++ b/packages/cli/src/utils/submitFeedback.test.ts@@ -18,6 +18,18 @@ describe("submitFeedback", () => {vi.unstubAllGlobals();});+ it("leaves the rating and its scale out of a report that has none", async () => {+ const fetchMock = vi.fn<typeof fetch>(async () => new Response(null, { status: 202 }));+ vi.stubGlobal("fetch", fetchMock);++ await submitFeedback({ comment: "MISSING FEATURE: trim", cliVersion: "1.2.3" });++ const body = JSON.parse(String(fetchMock.mock.calls[0]?.[1]?.body));+ expect(body).not.toHaveProperty("rating");+ expect(body).not.toHaveProperty("rating_scale");+ expect(body.comment).toBe("MISSING FEATURE: trim");+ });+it("posts feedback to the backend endpoint", async () => {const fetchMock = vi.fn<typeof fetch>(async () => new Response(null, { status: 202 }));vi.stubGlobal("fetch", fetchMock);diff --git a/packages/cli/src/utils/submitFeedback.ts b/packages/cli/src/utils/submitFeedback.tsindex f170184ab5..64d9674152 100644--- a/packages/cli/src/utils/submitFeedback.ts+++ b/packages/cli/src/utils/submitFeedback.ts@@ -15,7 +15,7 @@ function cap(value: string | undefined, max: number): string | undefined {}export async function submitFeedback(input: {- rating: number;+ rating?: number;comment?: string;cliVersion: string;env?: string;@@ -28,7 +28,7 @@ export async function submitFeedback(input: {method: "POST",body: JSON.stringify({rating: input.rating,- rating_scale: FEEDBACK_RATING_SCALE,+ rating_scale: input.rating === undefined ? undefined : FEEDBACK_RATING_SCALE,comment: cap(input.comment, MAX_COMMENT),cli_version: cap(input.cliVersion, MAX_CLI_VERSION),env: cap(input.env, MAX_ENV),diff --git a/skills-manifest.json b/skills-manifest.jsonindex 5f40e23b96..00ac68f09b 100644--- a/skills-manifest.json+++ b/skills-manifest.json@@ -30,7 +30,7 @@"files": 7},"hyperframes-cli": {- "hash": "cf031cc9764e451f",+ "hash": "599eba83e1bf7245","files": 11},"hyperframes-core": {diff --git a/skills/hyperframes-cli/SKILL.md b/skills/hyperframes-cli/SKILL.mdindex b16fe4a47f..fd74ca4f75 100644--- a/skills/hyperframes-cli/SKILL.md+++ b/skills/hyperframes-cli/SKILL.md@@ -142,6 +142,11 @@ npx hyperframes feedback --rating <0-10> --comment "<specific result or frictionKeep clean-run feedback concise. For any bug or friction, capture a **reproduction packet** before submitting; do not send only a symptom summary. Include the rerunnable command (relative to the project directory — feedback is submitted to a public channel, so do **not** paste absolute paths, home-directory prefixes, or user/machine identifiers), expected versus actual behavior, exact error (also strip absolute paths from stack traces — keep basename + line, drop the leading directory), whether output completed/fell back/failed, workaround, and repro-project status. For a rating ≤ 7 that describes a visual defect (black frame, flicker, corrupt output, wrong frame, blank output, other visual anomaly), also include a `COMPOSITION_STRUCTURE:` block — a privacy-preserving structural anatomy (element census + attribute presence + timeline shape) so maintainers can pattern-match against known bug families without the composition ZIP. Agents auto-fill this via the composition-census helper; the human user does not fill it by hand. If the issue did not reproduce again, say so and still include the last failing command and logs. Use `--file-issue` only with consent: it publishes a minimal reproduction to a public URL. The required packet format and privacy warning live in `references/preview-render.md`.+Two more reports are a `--comment` with no `--rating`, so they never count in the rating metric. Send them unless telemetry is disabled or the user opted out, under the same privacy rules:++- **Missing feature.** Whenever the person asks for something HyperFrames or the host app cannot do, or can only do with a workaround, send one report per ask: `npx hyperframes feedback --comment "MISSING FEATURE: <what the person asked for, in their words, with names, clients, figures and paths left out> | WORKAROUND: <what you did instead, or none>"`. The report is about the capability they wanted, never their content or anything private. A catalog search that found nothing goes to `--search-miss` instead.+- **Host app.** When `HYPERFRAMES_CLIENT` is set, the CLI is running inside a host app. When that app itself gets in the way (a panel, button, preview or export that misbehaves), send `npx hyperframes feedback --comment "HOST APP: <what happened>"` with the reproduction packet.+## Read the matching reference before running a commandThe following references and owning skills are mandatory command contracts, not optional background reading. Before running a command in the table, read its matching row.diff --git a/skills/hyperframes-cli/references/preview-render.md b/skills/hyperframes-cli/references/preview-render.mdindex 111a864409..b665c04e61 100644--- a/skills/hyperframes-cli/references/preview-render.md+++ b/skills/hyperframes-cli/references/preview-render.md@@ -157,7 +157,7 @@ npx hyperframes feedback --rating 10 # clean run, nnpx hyperframes feedback --rating 6 --comment "bg <video> renders grey in multi-scene; worked around with --format png-sequence"```-`--rating` is an integer from 0-10 (required); `--comment` is free text. Feedback is anonymous and attaches a `doctorSummary` (OS/Node/CPU/mem/ffmpeg) automatically, so don't repeat those fields. A clean run needs only a short result. Before sending any bug, workaround, or confusing behavior, collect this compact reproduction packet:+`--rating` is an integer from 0-10; `--comment` is free text. A `--comment` with no `--rating` is a ratingless report (`MISSING FEATURE:` and `HOST APP:` in `SKILL.md`) that never counts in the rating metric. Feedback is anonymous and attaches a `doctorSummary` (OS/Node/CPU/mem/ffmpeg) automatically, so don't repeat those fields. A clean run needs only a short result. Before sending any bug, workaround, or confusing behavior, collect this compact reproduction packet:```textREPRO COMMAND: <HF_*/PRODUCER_* env> npx hyperframes <exact command> # run from the project directory; do NOT paste absolute paths