Desktop app that records your on-screen work session and uses the GitHub Copilot SDK to reconstruct it as an intent + ordered steps, then builds a reusable Skill or Automation for Microsoft Autopilots like Scout, Microsoft Copilot Cowork, or Copilot Studio.
TypeScript agent-skillsai-agentsautomationcopilotcopilot-clicopilot-coworkcopilot-studioelectronmicrosoft-scoutscreen-recording
Standardize skill steps as numbered title cards; drop "asks first" (#37)
adileidff9dd09ce30
diff --git a/common/skill.ts b/common/skill.tsindex 71b7352..6871624 100644--- a/common/skill.ts+++ b/common/skill.ts@@ -132,7 +132,7 @@ export type SkillInput = z.infer<typeof SkillInputSchema>;* A generalized step is either a **calculation** (reads, derives, decides, or formats* — no external side effect) or an **action** (changes the world: submit, send, create,* delete). Splitting them keeps the plan honest about side effects; the actions are the- * risky surface, and destructive/sending ones set {@link PlanStep.pausesForConfirmation}.+ * risky surface.*/export const PlanStepKind = z.enum(["calculation", "action"]);export type PlanStepKind = z.infer<typeof PlanStepKind>;@@ -142,12 +142,12 @@ export const PlanStepSchema = z.preprocess((v) => (typeof v === "string" ? { kind: "action", text: v } : v),z.object({kind: PlanStepKind,+ /** Short title/label for the step, e.g. "List open PRs". */+ title: z.string().default(""),/** Imperative, generalized description of the step. */text: z.string(),/** The native tool/skill this step uses, if any (e.g. "workiq_search_chats"). */tool: z.string().default(""),- /** Actions that send/create/delete should pause for the user's OK before running. */- pausesForConfirmation: z.boolean().default(false),}),);export type PlanStep = z.infer<typeof PlanStepSchema>;diff --git a/electron/skillbuilder/builder.ts b/electron/skillbuilder/builder.tsindex 5b77ef2..5c8ea00 100644--- a/electron/skillbuilder/builder.ts+++ b/electron/skillbuilder/builder.ts@@ -32,7 +32,7 @@ const TURN_TIMEOUT_MS = 180_000;const KICKOFF_PROMPT ="Read get_analysis (and get_timeline where the tool mapping needs evidence), then call " +"propose_plan with how you'll generalize this task, its inputs (fixed / provided / locate), and its " +- "ordered calculation and action steps (each with the native tool it uses). " ++ "ordered steps (each a short title + description, with the native tool it uses). " +"Stop after propose_plan so the user can review it.";const CREATE_PROMPT =@@ -317,9 +317,9 @@ function renderPlanForPrompt(plan: SkillPlan): string {if (plan.steps.length) {lines.push("", "Steps (in order):");plan.steps.forEach((s, idx) => {- const bits = [`${idx + 1}. (${s.kind}) ${s.text}`];+ const head = [s.title, s.text].filter(Boolean).join(" — ");+ const bits = [`${idx + 1}. (${s.kind}) ${head}`];if (s.tool) bits.push(`[tool: ${s.tool}]`);- if (s.pausesForConfirmation) bits.push("[pause for confirmation]");lines.push(bits.join(" "));});}diff --git a/electron/skillbuilder/instructions.ts b/electron/skillbuilder/instructions.tsindex c4b1537..a0e5630 100644--- a/electron/skillbuilder/instructions.ts+++ b/electron/skillbuilder/instructions.ts@@ -76,15 +76,14 @@ user can override any of them in plain language.## Steps: separate calculations from actions-Break the generalized procedure into ordered **steps**, each tagged \`kind\`:+Break the generalized procedure into ordered **steps**, each with a short **title**, a+**description**, and a \`kind\`:- **calculation** — reads, derives, filters, decides, or formats. No external side effect(e.g. "read the sheet", "keep the rows still open", "compute the total").- **action** — changes the world: submits a form, sends a message, creates/edits/deletes afile or record, posts, pays. These are the risky surface — keep them explicit.-Put the native tool each step uses in its \`tool\`. For any action that sends, creates, or-deletes on the user's behalf — or is otherwise destructive — set \`pausesForConfirmation:-true\` so the skill checks with the user first. Order matters: interleave calculations and+Put the native tool each step uses in its \`tool\`. Order matters: interleave calculations andactions in the real sequence the task runs.## Write a good SKILL.md (authoring principles)@@ -107,7 +106,6 @@ should be written, not as a transcript of this one recording:template or a tiny Input/Output example only where it removes ambiguity.- **No surprises.** The skill must do exactly what its description says — no hiddenside effects, destructive steps, or data exfiltration the user wouldn't expect.- Destructive or send/create actions should pause for the user's confirmation.## Your tools@@ -116,8 +114,8 @@ should be written, not as a transcript of this one recording:counts) behind those steps. Use it to ground the native-tool mapping in real evidence.- **propose_plan({ name, title, description, summary, generalization, inputs, steps,allowedTools })** — your reviewable plan. Each input has a \`source\` (fixed / provided /- locate); each step has a \`kind\` (calculation / action), its \`tool\`, and an optional- \`pausesForConfirmation\`. Call once per turn, then stop.+ locate); each step has a short \`title\`, a \`text\` description, a \`kind\` (calculation /+ action), and its \`tool\`. Call once per turn, then stop.- **submit_skill({ name, description, allowedTools, body })** — the final skill. \`body\`is the SKILL.md instructions (imperative, generalized, native-tool-first). Call thisonly after the user approves the plan.diff --git a/electron/skillbuilder/tools.ts b/electron/skillbuilder/tools.tsindex 76114df..2a04ef0 100644--- a/electron/skillbuilder/tools.ts+++ b/electron/skillbuilder/tools.ts@@ -33,7 +33,7 @@ export function createSkillBuilderTools(ctx: SkillToolContext): Tool[] {const proposePlan: Tool = {name: "propose_plan",description:- "Propose your reviewable plan for the skill: how you'll generalize the task, the inputs it needs and where each comes from (fixed / provided / locate), the ordered steps tagged as calculations or actions (each with the native tool it uses), and the allowed-tools. Call this once per turn, then STOP so the user can review or refine it. Do NOT write the skill body yet.",+ "Propose your reviewable plan for the skill: how you'll generalize the task, the inputs it needs and where each comes from (fixed / provided / locate), the ordered steps (each with a short title, a description, and the native tool it uses), and the allowed-tools. Call this once per turn, then STOP so the user can review or refine it. Do NOT write the skill body yet.",parameters: {type: "object",properties: {@@ -75,7 +75,7 @@ export function createSkillBuilderTools(ctx: SkillToolContext): Tool[] {steps: {type: "array",description:- "The generalized procedure as ordered, typed steps. Tag each as a calculation (reads/derives/decides/formats — no external side effect) or an action (changes the world: submit/send/create/delete).",+ "The generalized procedure as ordered, typed steps. Give each a short title and a description, and tag it as a calculation (reads/derives/decides/formats — no external side effect) or an action (changes the world: submit/send/create/delete).",items: {type: "object",properties: {@@ -84,6 +84,10 @@ export function createSkillBuilderTools(ctx: SkillToolContext): Tool[] {enum: ["calculation", "action"],description: "calculation = no external effect; action = a side effect.",},+ title: {+ type: "string",+ description: "Short title/label for the step, e.g. \"List open PRs\".",+ },text: {type: "string",description: "Imperative, generalized description of the step.",@@ -93,13 +97,8 @@ export function createSkillBuilderTools(ctx: SkillToolContext): Tool[] {description:"The native tool/skill this step uses, e.g. \"workiq_search_chats\" or \"Bash(gh *)\".",},- pausesForConfirmation: {- type: "boolean",- description:- "For actions that send/create/delete: pause for the user's OK before running.",- },},- required: ["kind", "text"],+ required: ["kind", "title", "text"],additionalProperties: false,},},diff --git a/evals/README.md b/evals/README.mdindex 78d593a..d72ecdb 100644--- a/evals/README.md+++ b/evals/README.md@@ -208,13 +208,11 @@ each scenario asserts the shape of the proposed `SkillPlan`:vocabulary is applied, not just present.- **typed steps** — `minCalculations` / `minActions` require the procedure to besplit into `calculation` (no side effect) and `action` (changes the world) steps.-- **confirmation** — `requiresConfirmation` requires at least one action to set- `pausesForConfirmation`, i.e. the send/create/delete surface pauses for the user.**Coverage.** Five scenarios. Two target **Scout**: `price-tracker-skill` (a canonical-page URL → **fixed** input, `web_fetch` + the `xlsx` skill, calculations then a-confirmed append) and `github-issue-triage-skill` (the gh-vs-browser case as a skill —-must use `gh`, forbid the browser, and gate the mutating comment/label actions). Three+page URL → **fixed** input, `web_fetch` + the `xlsx` skill, calculations then an+append) and `github-issue-triage-skill` (the gh-vs-browser case as a skill —+must use `gh`, forbid the browser, and drive the mutating comment/label actions). Threetarget **Cowork** (Microsoft 365 Copilot), whose catalogue has **no browser automation**— each asserts the right M365 `server/Tool` is reached for while playwright/`click` andthe web hosts are forbidden: `cowork-teams-digest` (read a channel then post viadiff --git a/evals/skillbuilder/run.ts b/evals/skillbuilder/run.tsindex 1cc3b64..9150e48 100644--- a/evals/skillbuilder/run.ts+++ b/evals/skillbuilder/run.ts@@ -128,8 +128,7 @@ function printResult(r: Result): void {}for (const [i, s] of (r.plan?.steps ?? []).entries()) {const tool = s.tool ? ` {${s.tool}}` : "";- const gate = s.pausesForConfirmation ? " ⏸ confirm" : "";- console.error(` ${i + 1}. [${s.kind}] ${s.text}${tool}${gate}`);+ console.error(` ${i + 1}. [${s.kind}] ${s.text}${tool}`);}console.error(` score: ${Math.round((r.score?.score ?? 0) * 100)}% · ${r.ok ? "PASS" : "FAIL"} · ${(r.durationMs / 1000).toFixed(1)}s`);for (const c of r.score?.checks ?? []) {diff --git a/evals/skillbuilder/scenario.ts b/evals/skillbuilder/scenario.tsindex 9a3f894..5da4c70 100644--- a/evals/skillbuilder/scenario.ts+++ b/evals/skillbuilder/scenario.ts@@ -34,8 +34,6 @@ export interface SkillRubric {minCalculations?: number;/** The plan must contain at least this many `action` steps. Default 1. */minActions?: number;- /** When true, at least one action step must set `pausesForConfirmation` (a send/create/delete). */- requiresConfirmation?: boolean;}export interface SkillBuilderScenario {diff --git a/evals/skillbuilder/scenarios.ts b/evals/skillbuilder/scenarios.tsindex ab1484a..40ed5ef 100644--- a/evals/skillbuilder/scenarios.ts+++ b/evals/skillbuilder/scenarios.ts@@ -95,7 +95,6 @@ const priceTracker: SkillBuilderScenario = {minInputs: 1,minCalculations: 1,minActions: 1,- requiresConfirmation: false,},};@@ -176,7 +175,6 @@ const githubIssueTriage: SkillBuilderScenario = {minInputs: 1,minCalculations: 1,minActions: 1,- requiresConfirmation: true,},};@@ -255,7 +253,6 @@ const coworkTeamsDigest: SkillBuilderScenario = {minInputs: 1,minCalculations: 1,minActions: 1,- requiresConfirmation: true,},};@@ -327,7 +324,6 @@ const coworkOutlookReply: SkillBuilderScenario = {minInputs: 1,minCalculations: 1,minActions: 1,- requiresConfirmation: true,},};@@ -401,7 +397,6 @@ const coworkCalendarSchedule: SkillBuilderScenario = {minInputs: 1,minCalculations: 1,minActions: 1,- requiresConfirmation: true,},};diff --git a/evals/skillbuilder/score.ts b/evals/skillbuilder/score.tsindex c5eaf50..73fabd4 100644--- a/evals/skillbuilder/score.ts+++ b/evals/skillbuilder/score.ts@@ -87,16 +87,6 @@ export function scoreSkill(plan: SkillPlan, rubric: SkillRubric): SkillScoreResudetail: `found ${actions.length}`,});- // Confirmation on the risky surface: at least one action pauses for the user.- if (rubric.requiresConfirmation) {- const gated = actions.filter((s) => s.pausesForConfirmation);- checks.push({- name: "an action pauses for confirmation",- pass: gated.length > 0,- detail: gated.length ? `${gated.length} gated action(s)` : "no action set pausesForConfirmation",- });- }-// A forbidden-token hit (wrong tool) fails outright; otherwise every check must pass.const forbiddenHit = checks.some((c) => c.name.startsWith("avoids ") && !c.pass);const passed = checks.filter((c) => c.pass).length;diff --git a/src/App.css b/src/App.cssindex 4f61ec2..3a2198c 100644--- a/src/App.css+++ b/src/App.css@@ -2389,26 +2389,6 @@ button:focus-visible,gap: 8px;}-.step-kind {- font-family: var(--mono);- font-size: 9.5px;- text-transform: uppercase;- letter-spacing: 0.05em;- border-radius: 5px;- padding: 1px 6px;- flex: none;-}--.step-kind-calculation {- color: var(--muted);- background: var(--surface-2);-}--.step-kind-action {- color: var(--brand-ink);- background: var(--signal-dim);-}-.step-text {flex: 1 1 auto;min-width: 0;@@ -2584,14 +2564,6 @@ button:focus-visible,padding: 10px 12px;}-/* Skill steps carry a quiet left accent by kind. */-.step-tile.step-action {- border-left: 3px solid var(--brand);-}-.step-tile.step-calculation {- border-left: 3px solid var(--line-strong);-}-.tile-head {display: flex;align-items: flex-start;@@ -2816,7 +2788,7 @@ button:focus-visible,background: rgba(var(--rec-rgb), 0.08);}-/* --- skill-step foot: native tool + "asks first" toggle ------------------ */+/* --- skill-step foot: native tool ---------------------------------------- */.tile-foot {display: flex;align-items: center;@@ -2825,31 +2797,6 @@ button:focus-visible,padding-left: 2px;}-.confirm-toggle {- flex: none;- border: 1px solid var(--line);- border-radius: 20px;- background: transparent;- color: var(--faint);- font-family: var(--mono);- font-size: 10px;- text-transform: uppercase;- letter-spacing: 0.04em;- padding: 2px 9px;- cursor: pointer;- opacity: 0;- transition: opacity 0.12s ease, border-color 0.12s ease, color 0.12s ease, background 0.12s ease;-}-.tile:hover .confirm-toggle {- opacity: 1;-}-.confirm-toggle.on {- opacity: 1;- color: var(--warn);- border-color: var(--warn);- background: rgba(240, 160, 30, 0.08);-}-/* --- "+ Add …" — a quiet, intentional affordance ------------------------- */.tile-add {align-self: stretch;diff --git a/src/plan-edit.tsx b/src/plan-edit.tsxindex fc8d79c..39de1d4 100644--- a/src/plan-edit.tsx+++ b/src/plan-edit.tsx@@ -27,7 +27,7 @@ export function moveItem<T>(arr: T[], i: number, dir: -1 | 1): T[] {return out;}-/* --- shared vocab (source + step-kind) ----------------------------------- */+/* --- shared vocab (input source) ----------------------------------------- */export const SOURCE_LABEL: Record<SkillInput["source"], string> = {fixed: "Fixed value",@@ -47,15 +47,6 @@ const SOURCE_HINT: Record<SkillInput["source"], string> = {locate: "How the agent should find it on your device",};-const STEP_KIND_LABEL: Record<PlanStep["kind"], string> = {- calculation: "Calculates",- action: "Does",-};--const STEP_KIND_OPTIONS: { value: PlanStep["kind"]; label: string }[] = [- { value: "calculation", label: "Calculates" },- { value: "action", label: "Does" },-];/* --- EditableText: reads as plain text, becomes an input on click --------- */@@ -332,7 +323,7 @@ export function InputTiles({);}-/* --- Skill steps (ordered; typed calculation / action) -------------------- */+/* --- Skill steps (ordered; numbered title + description) ------------------ */export function SkillStepTiles({steps,@@ -347,21 +338,15 @@ export function SkillStepTiles({return (<div className="tiles">{steps.map((s, i) => (- <div key={i} className={`tile step-tile step-${s.kind}`}>+ <div key={i} className="tile step-tile"><div className="tile-head">- <EditablePill- value={s.kind}- options={STEP_KIND_OPTIONS}- onChange={(v) => patch(i, { kind: v })}- pill={<span className={`step-kind step-kind-${s.kind}`}>{STEP_KIND_LABEL[s.kind]}</span>}- />+ <span className="tile-num">{i + 1}</span><EditableTextclassName="ed-strong"- value={s.text}- placeholder="What happens in this step?"- ariaLabel="Step description"- multiline- onChange={(v) => patch(i, { text: v })}+ value={s.title}+ placeholder="Step title"+ ariaLabel="Step title"+ onChange={(v) => patch(i, { title: v })}/><RowControlsindex={i}@@ -370,6 +355,15 @@ export function SkillStepTiles({onRemove={() => onChange(removeAt(steps, i))}/></div>+ <EditableText+ as="p"+ multiline+ className="ed-detail"+ value={s.text}+ placeholder="What happens in this step?"+ ariaLabel="Step description"+ onChange={(v) => patch(i, { text: v })}+ /><div className="tile-foot"><EditableTextclassName="ed-tool"@@ -379,20 +373,12 @@ export function SkillStepTiles({hideEmptyonChange={(v) => patch(i, { tool: v })}/>- <button- type="button"- className={`confirm-toggle${s.pausesForConfirmation ? " on" : ""}`}- title="Pause for your OK before running this step"- onClick={() => patch(i, { pausesForConfirmation: !s.pausesForConfirmation })}- >- asks first- </button></div></div>))}<AddTilelabel="Add step"- onAdd={() => onChange([...steps, { kind: "action", text: "", tool: "", pausesForConfirmation: false }])}+ onAdd={() => onChange([...steps, { kind: "action", title: "", text: "", tool: "" }])}/></div>);