Skip to content

Write HTML. Render video. Built for agents.

feat(cli): send missing features and host-app friction as ratingless feedback (#5140)

Miguel Ángel80c2547eac05
diff --git a/docs/guides/feedback.mdx b/docs/guides/feedback.mdx
index 2aaead6282..3d823288f7 100644
--- a/docs/guides/feedback.mdx
+++ b/docs/guides/feedback.mdx
@@ -21,6 +21,12 @@ npx hyperframes feedback --rating 7 --comment "The render finished, but the capt
The rating is from 0 to 10. The comment is optional, but a specific example is usually more useful than a score alone.
+A comment with no rating is a report with no score, for something you needed that HyperFrames cannot do yet. It never counts in the rating:
+
+```bash
+npx hyperframes feedback --comment "MISSING FEATURE: trim a clip from the timeline | WORKAROUND: none"
+```
+
## Report a reproducible bug
Use `--file-issue` when the project can be shared publicly:
diff --git a/docs/packages/cli.mdx b/docs/packages/cli.mdx
index d69e1638f7..b3e218930b 100644
--- a/docs/packages/cli.mdx
+++ b/docs/packages/cli.mdx
@@ -1248,10 +1248,11 @@ the CLI attaches your email (below).
npx hyperframes feedback --rating 10
npx hyperframes feedback --rating 7 --comment "render succeeded but GSAP timeline didn't animate"
npx hyperframes feedback --rating 3 --comment "GSAP timeline froze on seek" --file-issue
+npx hyperframes feedback --comment "MISSING FEATURE: trim a clip from the timeline | WORKAROUND: none"
```
-`--rating` is required, 0–10. `--comment` adds free text. `--source person`
-or `--source agent` says who wrote a rating report (a `--search-miss` report
+`--rating` is 0–10. `--comment` adds free text; a `--comment` with no `--rating` is a report with no score (a missing feature, or friction in the app that launched the CLI), kept out of the rating metric. `--source person`
+or `--source agent` says who wrote a rating or comment report (a `--search-miss` report
carries neither), so the team can tell a person's feedback from an agent's. `--file-issue`
also opens a GitHub issue, `--dir` picks the project published as its repro
(default: current directory), and `--yes, -y` skips the consent prompt for
diff --git a/packages/cli/src/commands/coreSkillContent.test.ts b/packages/cli/src/commands/coreSkillContent.test.ts
index c8eab7ab8d..f6223d579b 100644
--- a/packages/cli/src/commands/coreSkillContent.test.ts
+++ b/packages/cli/src/commands/coreSkillContent.test.ts
@@ -46,6 +46,15 @@ describe("hyperframes-core contract docs", () => {
expect(renderReference).toContain("WORKAROUND:");
});
+ it("sends missing features and host-app friction as ratingless comments", () => {
+ const skill = read("skills", "hyperframes-cli", "SKILL.md");
+
+ expect(skill).toContain(
+ 'npx hyperframes feedback --comment "MISSING FEATURE: <what the person asked for, in their words, with names, clients, figures and paths left out> | WORKAROUND:',
+ );
+ expect(skill).toMatch(/HYPERFRAMES_CLIENT[^\n]*npx hyperframes feedback --comment "HOST APP:/);
+ });
+
it("mandates a composition-structure block for visual-defect feedback", () => {
const skill = read("skills", "hyperframes-cli", "SKILL.md");
const renderReference = read("skills", "hyperframes-cli", "references", "preview-render.md");
diff --git a/packages/cli/src/commands/feedback.test.ts b/packages/cli/src/commands/feedback.test.ts
index 98688b11c9..4ef4814272 100644
--- a/packages/cli/src/commands/feedback.test.ts
+++ b/packages/cli/src/commands/feedback.test.ts
@@ -8,8 +8,10 @@ const mocks = vi.hoisted(() => ({
}));
vi.mock("../telemetry/client.js", () => ({ shouldTrack: () => true, flush: mocks.flush }));
const trackRenderFeedback = vi.hoisted(() => vi.fn());
+const trackFeedbackComment = vi.hoisted(() => vi.fn());
vi.mock("../telemetry/events.js", () => ({
trackCatalogSearchMiss: vi.fn(),
+ trackFeedbackComment,
trackRenderFeedback,
}));
vi.mock("../telemetry/feedback.js", () => ({ getDoctorSummary: async () => "os=test" }));
@@ -58,3 +60,39 @@ it("refuses a source that is neither person nor agent, and sends nothing", async
await expect(feedback.run?.({ args: { rating: "6", source: "bot" } } as never)).rejects.toThrow();
expect(mocks.submitFeedback).not.toHaveBeenCalled();
});
+
+it("sends a comment with no rating as its own report, never as a rating", async () => {
+ vi.spyOn(console, "log").mockImplementation(() => {});
+ trackRenderFeedback.mockClear();
+ const comment = "MISSING FEATURE: trim a clip | WORKAROUND: none";
+ const { default: feedback } = await import("./feedback.js");
+ await feedback.run?.({ args: { comment, source: "agent" } } as never);
+ expect(trackRenderFeedback).not.toHaveBeenCalled();
+ expect(trackFeedbackComment).toHaveBeenLastCalledWith(expect.objectContaining({ comment }));
+ expect(mocks.submitFeedback).toHaveBeenLastCalledWith(
+ expect.objectContaining({ rating: undefined, comment }),
+ );
+});
+
+it.each([
+ ["neither a rating nor a comment", {}],
+ ["a blank comment with no rating", { comment: " " }],
+ ["--file-issue with no rating", { comment: "MISSING FEATURE: x", "file-issue": true }],
+])("refuses %s, and sends nothing", async (_, args) => {
+ vi.spyOn(console, "error").mockImplementation(() => {});
+ mocks.submitFeedback.mockClear();
+ const { default: feedback } = await import("./feedback.js");
+ await expect(feedback.run?.({ args } as never)).rejects.toThrow();
+ expect(mocks.submitFeedback).not.toHaveBeenCalled();
+});
+
+it("keeps the rating error for an unreadable rating, so a retry keeps the rating", async () => {
+ const errors = vi.spyOn(console, "error").mockImplementation(() => {});
+ const { default: feedback } = await import("./feedback.js");
+ await expect(
+ feedback.run?.({ args: { rating: "8/10", comment: "x" } } as never),
+ ).rejects.toThrow();
+ expect(String(errors.mock.calls.at(-1)?.[0])).toContain(
+ "Rating must be an integer between 0 and 10",
+ );
+});
diff --git a/packages/cli/src/commands/feedback.ts b/packages/cli/src/commands/feedback.ts
index fdedeb1387..6dcaa926cf 100644
--- a/packages/cli/src/commands/feedback.ts
+++ b/packages/cli/src/commands/feedback.ts
@@ -5,7 +5,11 @@ import { defineCommand } from "citty";
import * as clack from "@clack/prompts";
import open from "open";
import type { Example } from "./_examples.js";
-import { trackCatalogSearchMiss, trackRenderFeedback } from "../telemetry/events.js";
+import {
+ trackCatalogSearchMiss,
+ trackFeedbackComment,
+ trackRenderFeedback,
+} from "../telemetry/events.js";
import { shouldTrack } from "../telemetry/client.js";
import { getDoctorSummary } from "../telemetry/feedback.js";
import { readConfig, type RecentRenderRecord } from "../telemetry/config.js";
@@ -21,6 +25,10 @@ import { lintFeedbackComment, type FeedbackLintInput } from "../utils/feedbackLi
export const examples: Example[] = [
["Submit render feedback", 'hyperframes feedback --rating 8 --comment "fast but font missing"'],
["Quick rating only", "hyperframes feedback --rating 10"],
+ [
+ "Report something you needed that is missing (no rating)",
+ 'hyperframes feedback --comment "MISSING FEATURE: trim a clip from the timeline | WORKAROUND: none"',
+ ],
[
"Report a catalog gap after a fruitless search",
'hyperframes feedback --search-miss "typewriter that deletes" --wanted "text that types then backspaces"',
@@ -38,8 +46,8 @@ function normalizeComment(raw?: string): string | undefined {
/**
* Compact PostHog join keys appended to the environment string that rides
* along with the forwarded report (and therefore lands verbatim in the wild
- * feedback channel): `fid` = this submission's PostHog `cli_render_feedback`
- * `feedback_id`; `tid` = the install's telemetry distinct_id; `renders` =
+ * feedback channel): `fid` = the PostHog `feedback_id` (`cli_render_feedback` or
+ * `cli_feedback_comment`); `tid` = the install's telemetry distinct_id; `renders` =
* recent `render_job_id`s (newest last, `!` suffix = the render failed).
* Together they turn a wild report into an exact telemetry lookup instead of
* a hardware-fingerprint hunt.
@@ -155,18 +163,62 @@ async function fileGithubIssue(opts: {
await openAndPrintIssue(url);
}
+// A comment with no rating is its own report (a missing feature, host-app
+// friction): it scores nothing, so it must stay out of the rating metric.
+function reportRating(
+ raw: string | undefined,
+ comment: string | undefined,
+ fileIssue: boolean,
+): number | undefined {
+ const rating = raw === undefined ? undefined : parseFeedbackRating(raw);
+ if (rating === null) {
+ console.error(c.error("Rating must be an integer between 0 and 10"));
+ failCommand();
+ }
+ if (rating === undefined && !comment?.trim()) {
+ console.error(c.error("Give a --rating (an integer from 0 to 10), a --comment, or both"));
+ failCommand();
+ }
+ if (rating === undefined && fileIssue) {
+ console.error(c.error("--file-issue needs a --rating"));
+ failCommand();
+ }
+ return rating;
+}
+
+function trackFeedback(report: {
+ rating: number | undefined;
+ comment: string | undefined;
+ doctorSummary: string;
+ feedbackId: string;
+ recentRenderIds: string[] | undefined;
+}): void {
+ const { rating, comment, ...keys } = report;
+ if (rating === undefined) {
+ if (comment) trackFeedbackComment({ comment, ...keys });
+ return;
+ }
+ // Soft-warn (never blocks) when the comment for a non-clean report is
+ // missing the mandated reproduction-packet markers. Prints before the
+ // submission ack so the reporter sees the nudge while their run is fresh.
+ printFeedbackLintWarnings({ rating, comment });
+ // The standalone command runs separately from `render`, so it has no real
+ // elapsed time to report. Omit it rather than recording a fake duration.
+ trackRenderFeedback({ rating, comment, ...keys });
+}
+
export default defineCommand({
meta: { name: "feedback", description: "Submit feedback about your experience" },
args: {
rating: {
type: "string",
- // Required for a rating report, but --search-miss is a different report
- // with no rating to give, so the check moved into the run body.
+ // Optional: --search-miss and a lone --comment are reports with no
+ // rating to give, so the check lives in the run body.
description: "Likelihood to recommend (0=not likely, 10=extremely likely)",
},
comment: {
type: "string",
- description: "Optional details about your experience",
+ description: "Details about your experience; alone, a report with no rating",
},
source: {
type: "string",
@@ -220,13 +272,8 @@ export default defineCommand({
return;
}
- // `--rating` is no longer required at the arg level, so an absent one
- // reaches here as undefined rather than being rejected by the parser.
- const rating = args.rating === undefined ? null : parseFeedbackRating(args.rating);
- if (rating === null) {
- console.error(c.error("Rating must be an integer between 0 and 10"));
- failCommand();
- }
+ const comment = normalizeComment(args.comment);
+ const rating = reportRating(args.rating, comment, args["file-issue"] === true);
const source = parseFeedbackSource(args.source);
if (source === null) {
@@ -239,7 +286,6 @@ export default defineCommand({
return;
}
- const comment = normalizeComment(args.comment);
const doctorSummary = await getDoctorSummary();
// Join keys tying this report to the install's PostHog rows — see
@@ -254,14 +300,7 @@ export default defineCommand({
});
const envWithJoinKeys = doctorSummary ? `${doctorSummary} ${joinKeys}` : joinKeys;
- // Soft-warn (never blocks) when the comment for a non-clean report is
- // missing the mandated reproduction-packet markers. Prints before the
- // submission ack so the reporter sees the nudge while their run is fresh.
- printFeedbackLintWarnings({ rating, comment });
-
- // The standalone command runs separately from `render`, so it has no real
- // elapsed time to report. Omit it rather than recording a fake duration.
- trackRenderFeedback({
+ trackFeedback({
rating,
comment,
doctorSummary,
@@ -281,7 +320,7 @@ export default defineCommand({
email: feedbackEmail(),
});
- if (args["file-issue"] === true) {
+ if (args["file-issue"] === true && rating !== undefined) {
await fileGithubIssue({
rating,
comment,
diff --git a/packages/cli/src/telemetry/events.test.ts b/packages/cli/src/telemetry/events.test.ts
index 7930c84772..3db004d99f 100644
--- a/packages/cli/src/telemetry/events.test.ts
+++ b/packages/cli/src/telemetry/events.test.ts
@@ -36,6 +36,7 @@ const {
trackCliError,
trackFigmaImport,
trackRenderFeedback,
+ trackFeedbackComment,
trackRenderPreflightRejected,
trackAuthLoginStarted,
trackAuthLoginCompleted,
@@ -1069,6 +1070,17 @@ describe("render telemetry events", () => {
});
});
+describe("trackFeedbackComment", () => {
+ it("is its own event with no rating, so it never counts in the rating metric", () => {
+ trackEvent.mockClear();
+ trackFeedbackComment({ comment: "MISSING FEATURE: trim", feedbackId: "f1" });
+
+ const [name, props] = trackEvent.mock.calls[0] as [string, Record<string, unknown>];
+ expect(name).toBe("cli_feedback_comment");
+ expect(props).toEqual({ comment: "MISSING FEATURE: trim", feedback_id: "f1" });
+ });
+});
+
describe("trackRenderFeedback", () => {
beforeEach(() => {
trackEvent.mockClear();
diff --git a/packages/cli/src/telemetry/events.ts b/packages/cli/src/telemetry/events.ts
index 923631c959..32f890beeb 100644
--- a/packages/cli/src/telemetry/events.ts
+++ b/packages/cli/src/telemetry/events.ts
@@ -993,6 +993,22 @@ export function trackSkillsInstallSkipped(props: { reason: string }): void {
trackEvent("cli skill install skipped", { reason: props.reason });
}
+export function trackFeedbackComment(props: {
+ comment: string;
+ doctorSummary?: string;
+ feedbackId?: string;
+ recentRenderIds?: string[];
+}): void {
+ trackEvent("cli_feedback_comment", {
+ comment: props.comment,
+ ...(props.doctorSummary ? { doctor_summary: props.doctorSummary } : {}),
+ ...(props.feedbackId ? { feedback_id: props.feedbackId } : {}),
+ ...(props.recentRenderIds?.length
+ ? { recent_render_ids: props.recentRenderIds.join(",") }
+ : {}),
+ });
+}
+
export function trackRenderFeedback(props: {
rating: number;
renderDurationMs?: number;
@@ -1001,7 +1017,8 @@ export function trackRenderFeedback(props: {
/**
* Join key shared with the forwarded feedback report (Slack/backend): the
* same uuid rides in the report's env string as `fid=…`, so a wild report
- * resolves to exactly one PostHog `cli_render_feedback` event and vice versa.
+ * resolves to exactly one PostHog `cli_render_feedback` or
+ * `cli_feedback_comment` event and vice versa.
*/
feedbackId?: string;
/** render_job_id values of this install's recent renders (newest last). */
diff --git a/packages/cli/src/utils/submitFeedback.test.ts b/packages/cli/src/utils/submitFeedback.test.ts
index 61d2d5d7b5..021f98383f 100644
--- a/packages/cli/src/utils/submitFeedback.test.ts
+++ b/packages/cli/src/utils/submitFeedback.test.ts
@@ -18,6 +18,18 @@ describe("submitFeedback", () => {
vi.unstubAllGlobals();
});
+ it("leaves the rating and its scale out of a report that has none", async () => {
+ const fetchMock = vi.fn<typeof fetch>(async () => new Response(null, { status: 202 }));
+ vi.stubGlobal("fetch", fetchMock);
+
+ await submitFeedback({ comment: "MISSING FEATURE: trim", cliVersion: "1.2.3" });
+
+ const body = JSON.parse(String(fetchMock.mock.calls[0]?.[1]?.body));
+ expect(body).not.toHaveProperty("rating");
+ expect(body).not.toHaveProperty("rating_scale");
+ expect(body.comment).toBe("MISSING FEATURE: trim");
+ });
+
it("posts feedback to the backend endpoint", async () => {
const fetchMock = vi.fn<typeof fetch>(async () => new Response(null, { status: 202 }));
vi.stubGlobal("fetch", fetchMock);
diff --git a/packages/cli/src/utils/submitFeedback.ts b/packages/cli/src/utils/submitFeedback.ts
index f170184ab5..64d9674152 100644
--- a/packages/cli/src/utils/submitFeedback.ts
+++ b/packages/cli/src/utils/submitFeedback.ts
@@ -15,7 +15,7 @@ function cap(value: string | undefined, max: number): string | undefined {
}
export async function submitFeedback(input: {
- rating: number;
+ rating?: number;
comment?: string;
cliVersion: string;
env?: string;
@@ -28,7 +28,7 @@ export async function submitFeedback(input: {
method: "POST",
body: JSON.stringify({
rating: input.rating,
- rating_scale: FEEDBACK_RATING_SCALE,
+ rating_scale: input.rating === undefined ? undefined : FEEDBACK_RATING_SCALE,
comment: cap(input.comment, MAX_COMMENT),
cli_version: cap(input.cliVersion, MAX_CLI_VERSION),
env: cap(input.env, MAX_ENV),
diff --git a/skills-manifest.json b/skills-manifest.json
index 5f40e23b96..00ac68f09b 100644
--- a/skills-manifest.json
+++ b/skills-manifest.json
@@ -30,7 +30,7 @@
"files": 7
},
"hyperframes-cli": {
- "hash": "cf031cc9764e451f",
+ "hash": "599eba83e1bf7245",
"files": 11
},
"hyperframes-core": {
diff --git a/skills/hyperframes-cli/SKILL.md b/skills/hyperframes-cli/SKILL.md
index b16fe4a47f..fd74ca4f75 100644
--- a/skills/hyperframes-cli/SKILL.md
+++ b/skills/hyperframes-cli/SKILL.md
@@ -142,6 +142,11 @@ npx hyperframes feedback --rating <0-10> --comment "<specific result or friction
Keep clean-run feedback concise. For any bug or friction, capture a **reproduction packet** before submitting; do not send only a symptom summary. Include the rerunnable command (relative to the project directory — feedback is submitted to a public channel, so do **not** paste absolute paths, home-directory prefixes, or user/machine identifiers), expected versus actual behavior, exact error (also strip absolute paths from stack traces — keep basename + line, drop the leading directory), whether output completed/fell back/failed, workaround, and repro-project status. For a rating ≤ 7 that describes a visual defect (black frame, flicker, corrupt output, wrong frame, blank output, other visual anomaly), also include a `COMPOSITION_STRUCTURE:` block — a privacy-preserving structural anatomy (element census + attribute presence + timeline shape) so maintainers can pattern-match against known bug families without the composition ZIP. Agents auto-fill this via the composition-census helper; the human user does not fill it by hand. If the issue did not reproduce again, say so and still include the last failing command and logs. Use `--file-issue` only with consent: it publishes a minimal reproduction to a public URL. The required packet format and privacy warning live in `references/preview-render.md`.
+Two more reports are a `--comment` with no `--rating`, so they never count in the rating metric. Send them unless telemetry is disabled or the user opted out, under the same privacy rules:
+
+- **Missing feature.** Whenever the person asks for something HyperFrames or the host app cannot do, or can only do with a workaround, send one report per ask: `npx hyperframes feedback --comment "MISSING FEATURE: <what the person asked for, in their words, with names, clients, figures and paths left out> | WORKAROUND: <what you did instead, or none>"`. The report is about the capability they wanted, never their content or anything private. A catalog search that found nothing goes to `--search-miss` instead.
+- **Host app.** When `HYPERFRAMES_CLIENT` is set, the CLI is running inside a host app. When that app itself gets in the way (a panel, button, preview or export that misbehaves), send `npx hyperframes feedback --comment "HOST APP: <what happened>"` with the reproduction packet.
+
## Read the matching reference before running a command
The following references and owning skills are mandatory command contracts, not optional background reading. Before running a command in the table, read its matching row.
diff --git a/skills/hyperframes-cli/references/preview-render.md b/skills/hyperframes-cli/references/preview-render.md
index 111a864409..b665c04e61 100644
--- a/skills/hyperframes-cli/references/preview-render.md
+++ b/skills/hyperframes-cli/references/preview-render.md
@@ -157,7 +157,7 @@ npx hyperframes feedback --rating 10 # clean run, n
npx hyperframes feedback --rating 6 --comment "bg <video> renders grey in multi-scene; worked around with --format png-sequence"
```
-`--rating` is an integer from 0-10 (required); `--comment` is free text. Feedback is anonymous and attaches a `doctorSummary` (OS/Node/CPU/mem/ffmpeg) automatically, so don't repeat those fields. A clean run needs only a short result. Before sending any bug, workaround, or confusing behavior, collect this compact reproduction packet:
+`--rating` is an integer from 0-10; `--comment` is free text. A `--comment` with no `--rating` is a ratingless report (`MISSING FEATURE:` and `HOST APP:` in `SKILL.md`) that never counts in the rating metric. Feedback is anonymous and attaches a `doctorSummary` (OS/Node/CPU/mem/ffmpeg) automatically, so don't repeat those fields. A clean run needs only a short result. Before sending any bug, workaround, or confusing behavior, collect this compact reproduction packet:
```text
REPRO COMMAND: <HF_*/PRODUCER_* env> npx hyperframes <exact command> # run from the project directory; do NOT paste absolute paths