Skip to content

armgit / klad

Public Imported from GitHub
0

A drawing app built for agent/human collaboration

Rust

The agent in the editor

Indicode BV442163ba7cdf
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 9c0231f..b30595f 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -12,6 +12,8 @@ The first downloadable builds.
- `klad --version` and About show the commit a build was made from.
- In-betweens blend colours evenly (in OKLab), so light can follow a turning limb when the second key is coloured for it.
- `klad arcs` follows the part of a foot on the ground: a heel-to-toe roll is no longer reported as a slide.
+- The Agent pane: ask for a change in plain words and watch it land; the agent sees the drawing, reads your selection and notes, and edits through the same commands as a terminal agent. Your Claude API key stays in the Keychain; a spend limit per session; a switch per drawing that keeps it to looking; one click undoes its whole turn.
+- "Report a problem" copies what an issue needs.
- Bones: a layer or group gets a skeleton (`klad bone`, the bone tool J), weights from the bones or painted (`klad weights`), and poses that bend the drawing (`klad pose`, pose mode); in-betweens of two poses turn the bones.
- Keys: layers, groups and shapes move, turn, scale and fade between frames with eases (`klad key`, the keys section of the inspector).
- Godot: `--godot-rig` exports a Skeleton2D with Polygon2D parts and the poses as animations; `--godot-scene` a parallax scene with a Parallax2D per depth plane.
diff --git a/README.md b/README.md
index 75412dd..a95f501 100644
--- a/README.md
+++ b/README.md
@@ -40,7 +40,7 @@ Early, and open for testing. What works today:
- The `klad` CLI on macOS, Linux and Windows: layers, brush strokes, vector shapes, fills, inking, painting from shapes, light, effects, depth and fog, text in about fifty open fonts that come with it (and any other Google Fonts family on request), frames and clips, in-betweens, exports.
- The editor on macOS, and in a first version on Windows and Linux: a pen with pressure and tilt (pressure only on Windows; on Linux the mouse for now), brushes, layers, masks, a text tool, the timeline, onion skins, notes for the agent, and a start window for new and recent drawings.
-- Coming next: an agent inside the editor you sign in to, so you never need a terminal.
+- An agent inside the editor (the Agent pane) that works on the open drawing with your Claude API key, so you never need a terminal. Signing in with a ChatGPT account comes later.
![The start window](docs/images/start-window.png)
diff --git a/crates/app/src/agent/claude.rs b/crates/app/src/agent/claude.rs
new file mode 100644
index 0000000..53383d3
--- /dev/null
+++ b/crates/app/src/agent/claude.rs
@@ -0,0 +1,372 @@
+use super::{Delta, Provider, Reply, Request, Usage};
+use anyhow::{Context, Result, anyhow, bail};
+use serde_json::{Value, json};
+use std::io::{BufRead, BufReader, Write};
+use std::process::{Command, Stdio};
+
+pub const API: &str = "https://api.anthropic.com/v1";
+pub const VERSION: &str = "2026-10-21";
+
+pub struct Claude {
+ pub key: String,
+}
+
+fn updates(model: &str) -> bool {
+ ["claude-fable-5", "claude-opus-5-5", "claude-sonnet-5-5"]
+ .iter()
+ .any(|m| model.starts_with(m))
+}
+
+fn fallbacks(model: &str) -> bool {
+ ["claude-fable-5-1", "claude-opus-5", "claude-sonnet-5-5"]
+ .iter()
+ .any(|m| model.starts_with(m))
+}
+
+pub fn body(request: &Request) -> Value {
+ let mut tools = request.tools.clone();
+ if let Some(last) = tools.last_mut() {
+ last["cache_control"] = json!({"type": "ephemeral"});
+ }
+ let mut body = json!({
+ "model": request.model,
+ "max_tokens": 64000,
+ "stream": true,
+ "system": [{"type": "text", "text": request.system, "cache_control": {"type": "ephemeral"}}],
+ "tools": tools,
+ "tool_choice": {"type": "auto"},
+ "messages": request.messages,
+ "cache_control": {"type": "ephemeral"},
+ "output_config": {"effort": "medium"},
+ });
+ if updates(&request.model) {
+ body["thinking"] = json!({"type": "adaptive", "display": "updates"});
+ }
+ if fallbacks(&request.model) {
+ body["fallbacks"] = json!("default");
+ }
+ body
+}
+
+pub fn betas(model: &str) -> String {
+ let mut list = Vec::new();
+ if updates(model) {
+ list.push("thinking-display-updates-2026-08-18");
+ }
+ if fallbacks(model) {
+ list.push("server-side-fallback-2026-07-01");
+ }
+ list.join(",")
+}
+
+fn config(key: &str, extra: &[String]) -> String {
+ let quoted = |s: &str| s.replace('\\', "\\\\").replace('"', "\\\"");
+ let mut lines = vec![
+ format!("header = \"x-api-key: {}\"", quoted(key)),
+ format!("header = \"anthropic-version: {VERSION}\""),
+ ];
+ lines.extend(extra.iter().cloned());
+ lines.join("\n") + "\n"
+}
+
+fn curl(key: &str, url: &str, extra: &[String], body: Option<&std::path::Path>) -> Result<std::process::Child> {
+ let mut command = Command::new("curl");
+ command.args(["-sS", "-N", "--no-buffer", "--fail-with-body", "--config", "-", url]);
+ if let Some(path) = body {
+ command.args(["-X", "POST", "-H", "content-type: application/json", "--data-binary"]);
+ command.arg(format!("@{}", path.display()));
+ }
+ let mut child = command
+ .stdin(Stdio::piped())
+ .stdout(Stdio::piped())
+ .stderr(Stdio::piped())
+ .spawn()
+ .context("starting curl, which klad uses to reach the API")?;
+ child
+ .stdin
+ .take()
+ .context("curl's input")?
+ .write_all(config(key, extra).as_bytes())?;
+ Ok(child)
+}
+
+fn api_error(text: &str) -> String {
+ serde_json::from_str::<Value>(text)
+ .ok()
+ .and_then(|v| v["error"]["message"].as_str().map(str::to_string))
+ .unwrap_or_else(|| text.trim().chars().take(400).collect())
+}
+
+pub fn models(key: &str) -> Result<Vec<String>> {
+ let out = curl(key, &format!("{API}/models?limit=100"), &[], None)?.wait_with_output()?;
+ let text = String::from_utf8_lossy(&out.stdout).to_string();
+ if !out.status.success() {
+ bail!("{}", api_error(&text));
+ }
+ let v: Value = serde_json::from_str(&text)?;
+ Ok(v["data"]
+ .as_array()
+ .map(|a| a.iter().filter_map(|m| m["id"].as_str().map(str::to_string)).collect())
+ .unwrap_or_default())
+}
+
+#[derive(Default)]
+pub struct Stream {
+ blocks: Vec<Value>,
+ partial: Vec<String>,
+ pub stop: String,
+ pub usage: Usage,
+ pub error: Option<String>,
+}
+
+impl Stream {
+ pub fn line(&mut self, data: &str, delta: &mut dyn FnMut(Delta)) {
+ let Ok(event) = serde_json::from_str::<Value>(data) else {
+ return;
+ };
+ let usage = |u: &Value, into: &mut Usage| {
+ let n = |k: &str| u[k].as_u64();
+ if let Some(v) = n("input_tokens") {
+ into.input = v;
+ }
+ if let Some(v) = n("output_tokens") {
+ into.output = v;
+ }
+ if let Some(v) = n("cache_read_input_tokens") {
+ into.cache_read = v;
+ }
+ if let Some(v) = n("cache_creation_input_tokens") {
+ into.cache_write = v;
+ }
+ };
+ match event["type"].as_str().unwrap_or_default() {
+ "message_start" => usage(&event["message"]["usage"], &mut self.usage),
+ "content_block_start" => {
+ let i = event["index"].as_u64().unwrap_or(0) as usize;
+ while self.blocks.len() <= i {
+ self.blocks.push(Value::Null);
+ self.partial.push(String::new());
+ }
+ self.blocks[i] = event["content_block"].clone();
+ }
+ "content_block_delta" => {
+ let i = event["index"].as_u64().unwrap_or(0) as usize;
+ let (Some(block), d) = (self.blocks.get_mut(i), &event["delta"]) else {
+ return;
+ };
+ let append = |b: &mut Value, k: &str, t: &str| {
+ let now = b[k].as_str().unwrap_or_default().to_string();
+ b[k] = json!(now + t);
+ };
+ match d["type"].as_str().unwrap_or_default() {
+ "text_delta" => {
+ let t = d["text"].as_str().unwrap_or_default();
+ append(block, "text", t);
+ delta(Delta::Text(t.to_string()));
+ }
+ "thinking_delta" => {
+ let t = d["thinking"].as_str().unwrap_or_default();
+ append(block, "thinking", t);
+ delta(Delta::Thinking(t.to_string()));
+ }
+ "signature_delta" => append(block, "signature", d["signature"].as_str().unwrap_or_default()),
+ "input_json_delta" => self.partial[i].push_str(d["partial_json"].as_str().unwrap_or_default()),
+ _ => {}
+ }
+ }
+ "content_block_stop" => {
+ let i = event["index"].as_u64().unwrap_or(0) as usize;
+ if let Some(block) = self.blocks.get_mut(i)
+ && block["type"] == "tool_use"
+ {
+ let text = self.partial[i].trim();
+ block["input"] = match text.is_empty() {
+ true => json!({}),
+ false => serde_json::from_str::<Value>(text)
+ .ok()
+ .filter(Value::is_object)
+ .unwrap_or_else(|| json!({"__invalid": true})),
+ };
+ }
+ }
+ "message_delta" => {
+ if let Some(s) = event["delta"]["stop_reason"].as_str() {
+ self.stop = s.to_string();
+ }
+ usage(&event["usage"], &mut self.usage);
+ }
+ "error" => {
+ self.error = Some(
+ event["error"]["message"]
+ .as_str()
+ .unwrap_or("the API stopped the reply")
+ .into(),
+ )
+ }
+ _ => {}
+ }
+ }
+
+ pub fn content(&self) -> Vec<Value> {
+ self.blocks
+ .iter()
+ .filter(|b| !b.is_null())
+ .map(|b| {
+ let mut b = b.clone();
+ if b["type"] == "tool_use" && b["input"].get("__invalid").is_some() {
+ b["input"] = json!({});
+ }
+ b
+ })
+ .collect()
+ }
+
+ pub fn calls(&self) -> Vec<Value> {
+ self.blocks
+ .iter()
+ .filter(|b| b["type"] == "tool_use")
+ .cloned()
+ .collect()
+ }
+}
+
+impl Provider for Claude {
+ fn send(&mut self, request: &Request, delta: &mut dyn FnMut(Delta)) -> Result<Reply> {
+ let file = std::env::temp_dir().join(format!("klad-agent-{}-{}.json", std::process::id(), rand_tag()));
+ std::fs::write(&file, serde_json::to_vec(&body(request))?)?;
+ let beta = betas(&request.model);
+ let extra: Vec<String> = match beta.is_empty() {
+ true => Vec::new(),
+ false => vec![format!("header = \"anthropic-beta: {beta}\"")],
+ };
+ let mut child = curl(&self.key, &format!("{API}/messages"), &extra, Some(&file))?;
+ let mut stream = Stream::default();
+ let mut raw = String::new();
+ let stdout = child.stdout.take().context("curl's output")?;
+ BufReader::new(stdout)
+ .lines()
+ .map_while(Result::ok)
+ .for_each(|line| match line.strip_prefix("data:") {
+ Some(data) => stream.line(data.trim(), delta),
+ None if !line.starts_with("event:") => {
+ raw.push_str(&line);
+ raw.push('\n');
+ }
+ None => {}
+ });
+ let status = child.wait()?;
+ let _ = std::fs::remove_file(&file);
+ if let Some(e) = stream.error {
+ return Err(anyhow!("the API: {e}"));
+ }
+ if !status.success() && stream.stop.is_empty() {
+ let mut err = String::new();
+ if let Some(mut e) = child.stderr.take() {
+ let _ = std::io::Read::read_to_string(&mut e, &mut err);
+ }
+ bail!(
+ "the API: {}",
+ api_error(if raw.trim().is_empty() { &err } else { &raw })
+ );
+ }
+ let invalid = stream
+ .calls()
+ .iter()
+ .filter(|c| c["input"].get("__invalid").is_some())
+ .filter_map(|c| c["id"].as_str().map(str::to_string))
+ .collect();
+ let content = stream.content();
+ if stream.stop == "refusal" {
+ delta(Delta::Text("(the model declined this request)".into()));
+ }
+ Ok(Reply {
+ content,
+ stop: stream.stop,
+ usage: stream.usage,
+ invalid,
+ })
+ }
+}
+
+fn rand_tag() -> u128 {
+ std::time::SystemTime::now()
+ .duration_since(std::time::UNIX_EPOCH)
+ .map_or(0, |d| d.as_nanos())
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+
+ #[test]
+ fn a_streamed_reply_with_thinking_text_and_a_tool_call_comes_back_whole() {
+ let lines = [
+ r#"{"type":"message_start","message":{"usage":{"input_tokens":1200,"cache_read_input_tokens":900,"cache_creation_input_tokens":0}}}"#,
+ r#"{"type":"content_block_start","index":0,"content_block":{"type":"thinking","thinking":"","signature":""}}"#,
+ r#"{"type":"content_block_delta","index":0,"delta":{"type":"thinking_delta","thinking":"Looking first."}}"#,
+ r#"{"type":"content_block_delta","index":0,"delta":{"type":"signature_delta","signature":"sig"}}"#,
+ r#"{"type":"content_block_stop","index":0}"#,
+ r#"{"type":"content_block_start","index":1,"content_block":{"type":"text","text":""}}"#,
+ r#"{"type":"content_block_delta","index":1,"delta":{"type":"text_delta","text":"Let me "}}"#,
+ r#"{"type":"content_block_delta","index":1,"delta":{"type":"text_delta","text":"look."}}"#,
+ r#"{"type":"content_block_stop","index":1}"#,
+ r#"{"type":"content_block_start","index":2,"content_block":{"type":"tool_use","id":"t1","name":"look","input":{}}}"#,
+ r#"{"type":"content_block_delta","index":2,"delta":{"type":"input_json_delta","partial_json":"{\"sca"}}"#,
+ r#"{"type":"content_block_delta","index":2,"delta":{"type":"input_json_delta","partial_json":"le\": 0.5}"}}"#,
+ r#"{"type":"content_block_stop","index":2}"#,
+ r#"{"type":"message_delta","delta":{"stop_reason":"tool_use"},"usage":{"output_tokens":42}}"#,
+ ];
+ let mut stream = Stream::default();
+ let mut said = Vec::new();
+ lines.iter().for_each(|l| stream.line(l, &mut |d| said.push(d)));
+ let content = stream.content();
+ assert_eq!(
+ content[0],
+ json!({"type": "thinking", "thinking": "Looking first.", "signature": "sig"})
+ );
+ assert_eq!(content[1]["text"], "Let me look.");
+ assert_eq!(content[2]["input"], json!({"scale": 0.5}));
+ assert_eq!(stream.stop, "tool_use");
+ assert_eq!(
+ stream.usage,
+ Usage {
+ input: 1200,
+ output: 42,
+ cache_read: 900,
+ cache_write: 0
+ }
+ );
+ assert_eq!(said[1], Delta::Text("Let me ".into()));
+ let request = Request {
+ model: "claude-opus-5-5".into(),
+ system: "draw".into(),
+ tools: super::super::tools(true),
+ messages: vec![json!({"role": "user", "content": "hi"})],
+ };
+ let b = body(&request);
+ assert_eq!(
+ b["tools"].as_array().unwrap().last().unwrap()["cache_control"]["type"],
+ "ephemeral"
+ );
+ assert_eq!(b["system"][0]["cache_control"]["type"], "ephemeral");
+ assert_eq!(b["thinking"]["display"], "updates");
+ assert_eq!(b["fallbacks"], "default");
+ assert!(betas("claude-opus-5-5").contains("server-side-fallback-2026-07-01"));
+ assert!(
+ !config("sk-ant-x", &[]).is_empty() && !format!("{:?}", b).contains("sk-ant"),
+ "the key is never in the body"
+ );
+ let mut broken = Stream::default();
+ [
+ r#"{"type":"content_block_start","index":0,"content_block":{"type":"tool_use","id":"t9","name":"apply","input":{}}}"#,
+ r#"{"type":"content_block_delta","index":0,"delta":{"type":"input_json_delta","partial_json":"{\"ops\": \"cut"}}"#,
+ r#"{"type":"content_block_stop","index":0}"#,
+ ]
+ .iter()
+ .for_each(|l| broken.line(l, &mut |_| {}));
+ assert!(
+ broken.calls()[0]["input"].get("__invalid").is_some(),
+ "a cut-off input is caught, not run"
+ );
+ }
+}
diff --git a/crates/app/src/agent/mod.rs b/crates/app/src/agent/mod.rs
new file mode 100644
index 0000000..1979d42
--- /dev/null
+++ b/crates/app/src/agent/mod.rs
@@ -0,0 +1,705 @@
+pub mod claude;
+pub mod secret;
+
+use anyhow::{Context, Result, anyhow, bail};
+use serde_json::{Value, json};
+use std::io::Write;
+use std::path::{Component, Path, PathBuf};
+use std::process::{Command, Stdio};
+use std::sync::Arc;
+use std::sync::atomic::{AtomicBool, Ordering};
+use std::sync::mpsc::Sender;
+
+pub const DEFAULT_MODEL: &str = "claude-opus-5-5";
+
+const NO_BUNDLE: [&str; 3] = ["fonts", "licences", "skill"];
+const REFUSED: [&str; 10] = [
+ "open", "install", "watch", "bench", "demo", "stress", "scribble", "new", "apply", "help",
+];
+const READ_ONLY: [&str; 16] = [
+ "ls",
+ "show",
+ "frames",
+ "bones",
+ "keys",
+ "brushes",
+ "notes",
+ "selection",
+ "strokes",
+ "hash",
+ "fonts",
+ "arcs",
+ "compare",
+ "log",
+ "skill",
+ "palette",
+];
+
+#[derive(Clone, Copy, Debug, Default, PartialEq)]
+pub struct Usage {
+ pub input: u64,
+ pub output: u64,
+ pub cache_read: u64,
+ pub cache_write: u64,
+}
+
+impl Usage {
+ pub fn add(&mut self, other: Usage) {
+ self.input += other.input;
+ self.output += other.output;
+ self.cache_read += other.cache_read;
+ self.cache_write += other.cache_write;
+ }
+
+ pub fn cost(&self, model: &str) -> f64 {
+ let (input, output) = price(model);
+ (self.input as f64 * input
+ + self.output as f64 * output
+ + self.cache_read as f64 * input * 0.05
+ + self.cache_write as f64 * input * 1.25)
+ / 1_000_000.0
+ }
+}
+
+pub fn price(model: &str) -> (f64, f64) {
+ match model {
+ m if m.starts_with("claude-fable") || m.starts_with("claude-mythos") => (10.0, 50.0),
+ m if m.starts_with("claude-opus-5-5") => (4.0, 20.0),
+ m if m.starts_with("claude-opus") => (5.0, 25.0),
+ m if m.starts_with("claude-sonnet-4") => (3.0, 15.0),
+ m if m.starts_with("claude-sonnet") => (2.0, 10.0),
+ m if m.starts_with("claude-haiku-5") => (0.1, 0.5),
+ m if m.starts_with("claude-haiku") => (1.0, 5.0),
+ _ => (5.0, 25.0),
+ }
+}
+
+#[derive(Clone, Debug, PartialEq)]
+pub enum Delta {
+ Text(String),
+ Thinking(String),
+}
+
+pub struct Request {
+ pub model: String,
+ pub system: String,
+ pub tools: Vec<Value>,
+ pub messages: Vec<Value>,
+}
+
+pub struct Reply {
+ pub content: Vec<Value>,
+ pub stop: String,
+ pub usage: Usage,
+ pub invalid: Vec<String>,
+}
+
+pub trait Provider: Send {
+ fn send(&mut self, request: &Request, delta: &mut dyn FnMut(Delta)) -> Result<Reply>;
+}
+
+#[derive(Clone, Debug, PartialEq)]
+pub enum Event {
+ Text(String),
+ Thinking(String),
+ Tool {
+ name: String,
+ input: Value,
+ },
+ Done {
+ name: String,
+ text: String,
+ image: Option<Vec<u8>>,
+ failed: bool,
+ edited: bool,
+ },
+ Usage(Usage),
+ Finished {
+ stop: String,
+ },
+ Failed(String),
+}
+
+pub struct Drawing {
+ pub bundle: PathBuf,
+ pub binary: PathBuf,
+ pub editable: bool,
+}
+
+impl Drawing {
+ fn folder(&self) -> PathBuf {
+ self.bundle
+ .parent()
+ .map_or_else(|| PathBuf::from("."), Path::to_path_buf)
+ }
+
+ fn exports(&self) -> PathBuf {
+ self.folder().join("exports")
+ }
+}
+
+fn normal(path: &Path) -> PathBuf {
+ path.components().fold(PathBuf::new(), |mut out, c| {
+ match c {
+ Component::ParentDir => {
+ out.pop();
+ }
+ Component::CurDir => {}
+ other => out.push(other.as_os_str()),
+ }
+ out
+ })
+}
+
+fn looks_like_path(arg: &str) -> bool {
+ arg.contains('/')
+ || arg.contains('\\')
+ || arg.starts_with('~')
+ || [
+ ".png", ".jpg", ".jpeg", ".json", ".tres", ".tscn", ".ttf", ".otf", ".draw", ".svg",
+ ]
+ .iter()
+ .any(|e| arg.to_ascii_lowercase().ends_with(e))
+}
+
+pub fn allowed(drawing: &Drawing, arg: &str) -> Result<()> {
+ if !looks_like_path(arg) || arg.trim_start().starts_with(['M', 'm']) && arg.contains(' ') {
+ return Ok(());
+ }
+ if arg.starts_with('~') {
+ bail!("{arg}: paths stay inside the drawing or its exports folder");
+ }
+ let resolved = normal(&drawing.folder().join(arg));
+ let roots = [normal(&drawing.bundle), normal(&drawing.exports())];
+ match roots.iter().any(|r| resolved.starts_with(r)) {
+ true => Ok(()),
+ false => bail!(
+ "{arg}: paths stay inside the drawing ({}) or the exports folder next to it",
+ drawing.bundle.display()
+ ),
+ }
+}
+
+pub fn tools(editable: bool) -> Vec<Value> {
+ let mut list = vec![
+ json!({
+ "name": "look",
+ "description": "Render the open drawing and see it. Use this after every change you make, and zoom in on details with region and scale.",
+ "eager_input_streaming": true,
+ "input_schema": {"type": "object", "properties": {
+ "frame": {"type": "integer", "description": "Frame to render (default 0)"},
+ "region": {"type": "string", "description": "x,y,w,h in canvas px to render only that part"},
+ "scale": {"type": "number", "description": "Render scale; default fits the longest side in 1024 px"},
+ "overlay": {"type": "string", "description": "ids, bbox, grid, parts, bones or weights:<bone>"}
+ }}
+ }),
+ json!({
+ "name": "read",
+ "description": "Read what the user selected or pinned, the layer tree, or a layer's strokes, as text.",
+ "eager_input_streaming": true,
+ "input_schema": {"type": "object", "properties": {
+ "what": {"type": "string", "enum": ["selection", "notes", "ls", "strokes", "frames", "bones", "keys"]},
+ "layer": {"type": "string", "description": "For strokes: the layer id or name"}
+ }, "required": ["what"]}
+ }),
+ ];
+ if editable {
+ list.push(json!({
+ "name": "klad",
+ "description": "Run a klad command on the open drawing, as from a terminal. args starts with the subcommand; the drawing's path is put after it for you. Example: [\"stroke\", \"l4\", \"--brush\", \"ink\", \"--size\", \"8\", \"--path\", \"M100 100 L 200 140\"]. Files you write go into the drawing or the exports folder next to it.",
+ "eager_input_streaming": true,
+ "input_schema": {"type": "object", "properties": {
+ "args": {"type": "array", "items": {"type": "string"}}
+ }, "required": ["args"]}
+ }));
+ list.push(json!({
+ "name": "apply",
+ "description": "Apply many edits at once: JSON op lines, as klad apply takes them. One transaction.",
+ "eager_input_streaming": true,
+ "input_schema": {"type": "object", "properties": {
+ "ops": {"type": "string", "description": "One JSON op per line"}
+ }, "required": ["ops"]}
+ }));
+ }
+ list
+}
+
+pub fn system(drawing: &Drawing) -> String {
+ format!(
+ "You are working inside the klad editor, on the drawing the user has open: {}. The user watches every change land on their canvas and can undo your whole turn in one step. Use look to see the drawing before and after you change it, read for what the user selected or pinned, klad and apply to change it. Keep to what the user asked; say briefly what you did.\n\n{}",
+ drawing.bundle.display(),
+ include_str!("../../../../docs/skill/SKILL.md")
+ )
+}
+
+struct Ran {
+ text: String,
+ image: Option<Vec<u8>>,
+ failed: bool,
+ edited: bool,
+}
+
+fn run(drawing: &Drawing, args: &[String], stdin: Option<&str>) -> Result<(bool, String)> {
+ let mut child = Command::new(&drawing.binary)
+ .args(args)
+ .current_dir(drawing.folder())
+ .env("KLAD_AUTHOR", "agent")
+ .stdin(if stdin.is_some() { Stdio::piped() } else { Stdio::null() })
+ .stdout(Stdio::piped())
+ .stderr(Stdio::piped())
+ .spawn()
+ .with_context(|| format!("starting {}", drawing.binary.display()))?;
+ if let (Some(text), Some(mut pipe)) = (stdin, child.stdin.take()) {
+ pipe.write_all(text.as_bytes())?;
+ }
+ let out = child.wait_with_output()?;
+ let text = String::from_utf8_lossy(&out.stdout).to_string() + &String::from_utf8_lossy(&out.stderr);
+ let mut text = text.trim().to_string();
+ if text.len() > 12_000 {
+ text.truncate(12_000);
+ text.push_str("\n... (cut; ask for less, for example one layer or a region)");
+ }
+ Ok((out.status.success(), text))
+}
+
+fn bundle_arg(drawing: &Drawing) -> String {
+ drawing.bundle.to_string_lossy().to_string()
+}
+
+fn execute(drawing: &Drawing, name: &str, input: &Value) -> Result<Ran> {
+ let s = |k: &str| input.get(k).and_then(Value::as_str).map(str::to_string);
+ match name {
+ "look" => {
+ let out = std::env::temp_dir().join(format!("klad-agent-look-{}.png", std::process::id()));
+ let mut args = vec![
+ "render".to_string(),
+ bundle_arg(drawing),
+ "--out".into(),
+ out.to_string_lossy().to_string(),
+ ];
+ if let Some(f) = input.get("frame").and_then(Value::as_u64) {
+ args.extend(["--frame".into(), f.to_string()]);
+ }
+ if let Some(r) = s("region") {
+ args.extend(["--region".into(), r]);
+ }
+ let scale = input.get("scale").and_then(Value::as_f64).or_else(|| {
+ let doc = std::fs::read_to_string(drawing.bundle.join("doc.json")).ok()?;
+ let v: Value = serde_json::from_str(&doc).ok()?;
+ let longest = v["canvas"]["width"].as_f64()?.max(v["canvas"]["height"].as_f64()?);
+ (s("region").is_none()).then(|| (1024.0 / longest).min(1.0))
+ });
+ if let Some(sc) = scale {
+ args.extend(["--scale".into(), format!("{sc:.3}")]);
+ }
+ if let Some(o) = s("overlay") {
+ args.extend(["--overlay".into(), o]);
+ }
+ let (ok, text) = run(drawing, &args, None)?;
+ let image = ok.then(|| std::fs::read(&out).ok()).flatten();
+ let _ = std::fs::remove_file(&out);
+ Ok(Ran {
+ text,
+ image,
+ failed: !ok,
+ edited: false,
+ })
+ }
+ "read" => {
+ let what = s("what").unwrap_or_default();
+ let mut args = match what.as_str() {
+ "selection" | "notes" | "ls" | "frames" | "bones" | "keys" => vec![what.clone(), bundle_arg(drawing)],
+ "strokes" => vec![
+ "strokes".into(),
+ bundle_arg(drawing),
+ s("layer").context("strokes needs a layer")?,
+ ],
+ other => bail!("read: no {other:?}; selection, notes, ls, strokes, frames, bones or keys"),
+ };
+ if what == "strokes" {
+ args.push("--json".into());
+ }
+ let (ok, text) = run(drawing, &args, None)?;
+ Ok(Ran {
+ text,
+ image: None,
+ failed: !ok,
+ edited: false,
+ })
+ }
+ "klad" if drawing.editable => {
+ let args: Vec<String> = input
+ .get("args")
+ .and_then(Value::as_array)
+ .context("klad needs args: a list of strings")?
+ .iter()
+ .map(|a| {
+ a.as_str()
+ .map(str::to_string)
+ .ok_or_else(|| anyhow!("every arg is a string"))
+ })
+ .collect::<Result<_>>()?;
+ let sub = args.first().cloned().context("klad needs a subcommand")?;
+ if REFUSED.contains(&sub.as_str()) {
+ bail!("klad {sub} is not available here; for many edits at once use the apply tool");
+ }
+ args.iter().try_for_each(|a| allowed(drawing, a))?;
+ let mut full = vec![sub.clone()];
+ if !NO_BUNDLE.contains(&sub.as_str()) {
+ full.push(bundle_arg(drawing));
+ }
+ full.extend(args[1..].iter().cloned());
+ let (ok, text) = run(drawing, &full, None)?;
+ Ok(Ran {
+ text,
+ image: None,
+ failed: !ok,
+ edited: ok && !READ_ONLY.contains(&sub.as_str()),
+ })
+ }
+ "apply" if drawing.editable => {
+ let ops = s("ops").context("apply needs ops: JSON lines")?;
+ let (ok, text) = run(drawing, &["apply".into(), bundle_arg(drawing)], Some(&ops))?;
+ Ok(Ran {
+ text,
+ image: None,
+ failed: !ok,
+ edited: ok,
+ })
+ }
+ "klad" | "apply" => bail!("the user has not let the agent edit this drawing; it can look and suggest only"),
+ other => bail!("no tool {other}"),
+ }
+}
+
+pub fn base64(bytes: &[u8]) -> String {
+ const TABLE: &[u8; 64] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
+ bytes
+ .chunks(3)
+ .flat_map(|c| {
+ let n = (c[0] as u32) << 16 | (*c.get(1).unwrap_or(&0) as u32) << 8 | *c.get(2).unwrap_or(&0) as u32;
+ let out = [18, 12, 6, 0].map(|s| TABLE[(n >> s & 63) as usize] as char);
+ let keep = c.len() + 1;
+ (0..4).map(move |i| if i < keep { out[i] } else { '=' })
+ })
+ .collect()
+}
+
+pub struct Limits {
+ pub spend: f64,
+ pub spent: f64,
+}
+
+pub fn turn(
+ provider: &mut dyn Provider,
+ drawing: &Drawing,
+ model: &str,
+ history: &mut Vec<Value>,
+ limits: Limits,
+ stop: &Arc<AtomicBool>,
+ events: &Sender<Event>,
+) -> Result<usize> {
+ let request_tools = tools(drawing.editable);
+ let system = system(drawing);
+ let mut spent = limits.spent;
+ let mut edits = 0;
+ loop {
+ if limits.spend > 0.0 && spent >= limits.spend {
+ let _ = events.send(Event::Finished {
+ stop: format!("the spend limit of ${:.2} for this session is reached", limits.spend),
+ });
+ return Ok(edits);
+ }
+ let request = Request {
+ model: model.to_string(),
+ system: system.clone(),
+ tools: request_tools.clone(),
+ messages: history.clone(),
+ };
+ let reply = provider.send(&request, &mut |d| {
+ let _ = events.send(match d {
+ Delta::Text(t) => Event::Text(t),
+ Delta::Thinking(t) => Event::Thinking(t),
+ });
+ })?;
+ spent += reply.usage.cost(model);
+ let _ = events.send(Event::Usage(reply.usage));
+ history.push(json!({"role": "assistant", "content": reply.content}));
+ if reply.stop != "tool_use" {
+ let _ = events.send(Event::Finished { stop: reply.stop });
+ return Ok(edits);
+ }
+ let calls: Vec<(String, String, Value)> = reply
+ .content
+ .iter()
+ .filter(|b| b["type"] == "tool_use")
+ .map(|b| {
+ (
+ b["id"].as_str().unwrap_or_default().to_string(),
+ b["name"].as_str().unwrap_or_default().to_string(),
+ b["input"].clone(),
+ )
+ })
+ .collect();
+ let mut results = Vec::new();
+ for (id, name, input) in calls {
+ if stop.load(Ordering::Relaxed) {
+ results.push(json!({"type": "tool_result", "tool_use_id": id, "content": "stopped by the user", "is_error": true}));
+ continue;
+ }
+ let _ = events.send(Event::Tool {
+ name: name.clone(),
+ input: input.clone(),
+ });
+ let ran = match reply.invalid.contains(&id) {
+ true => Err(anyhow!("the tool input was not valid JSON (cut off?); send it again")),
+ false => execute(drawing, &name, &input),
+ };
+ let (text, image, failed, edited) = match ran {
+ Ok(r) => (r.text, r.image, r.failed, r.edited),
+ Err(e) => (format!("{e:#}"), None, true, false),
+ };
+ edits += edited as usize;
+ let _ = events.send(Event::Done {
+ name: name.clone(),
+ text: text.clone(),
+ image: image.clone(),
+ failed,
+ edited,
+ });
+ let mut content =
+ vec![json!({"type": "text", "text": if text.is_empty() { "done".to_string() } else { text }})];
+ if let Some(png) = &image {
+ content.insert(0, json!({"type": "image", "source": {"type": "base64", "media_type": "image/png", "data": base64(png)}}));
+ }
+ results.push(json!({"type": "tool_result", "tool_use_id": id, "content": content, "is_error": failed}));
+ }
+ history.push(json!({"role": "user", "content": results}));
+ if stop.load(Ordering::Relaxed) {
+ let _ = events.send(Event::Finished { stop: "stopped".into() });
+ return Ok(edits);
+ }
+ }
+}
+
+pub struct Scripted {
+ pub replies: Vec<Vec<Value>>,
+}
+
+impl Provider for Scripted {
+ fn send(&mut self, _request: &Request, delta: &mut dyn FnMut(Delta)) -> Result<Reply> {
+ if self.replies.is_empty() {
+ bail!("the script has no more replies");
+ }
+ let content = self.replies.remove(0);
+ content
+ .iter()
+ .filter_map(|b| b["text"].as_str())
+ .for_each(|t| delta(Delta::Text(t.to_string())));
+ let stop = match content.iter().any(|b| b["type"] == "tool_use") {
+ true => "tool_use",
+ false => "end_turn",
+ };
+ Ok(Reply {
+ content,
+ stop: stop.into(),
+ usage: Usage {
+ input: 1000,
+ output: 100,
+ ..Usage::default()
+ },
+ invalid: Vec::new(),
+ })
+ }
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+ use klad_core::bundle::Bundle;
+ use klad_core::doc::{Document, Layer};
+ use std::sync::mpsc::channel;
+
+ fn binary() -> PathBuf {
+ let exe = std::env::current_exe().unwrap();
+ exe.parent().and_then(Path::parent).unwrap().join("klad")
+ }
+
+ fn drawing(name: &str, editable: bool) -> Drawing {
+ let dir = std::env::temp_dir().join(format!("klad-agent-{name}-{}", std::process::id()));
+ let _ = std::fs::remove_dir_all(&dir);
+ std::fs::create_dir_all(&dir).unwrap();
+ let mut doc = Document::new(320, 200);
+ doc.canvas.background = Some("#ffffff".into());
+ doc.layers = vec![Layer::paint("l1", "paint", "c1")];
+ Bundle::create(dir.join("art.draw"), doc).unwrap();
+ Drawing {
+ bundle: dir.join("art.draw"),
+ binary: binary(),
+ editable,
+ }
+ }
+
+ fn call(id: &str, name: &str, input: Value) -> Value {
+ json!({"type": "tool_use", "id": id, "name": name, "input": input})
+ }
+
+ #[test]
+ fn a_scripted_turn_draws_looks_and_undoes_in_its_steps() {
+ let d = drawing("turn", true);
+ let mut script = Scripted {
+ replies: vec![
+ vec![
+ json!({"type": "text", "text": "Drawing a line."}),
+ call(
+ "t1",
+ "apply",
+ json!({"ops": r##"{"op":"stroke","layer":"l1","brush":"ink","size":10,"color":"#202020","path":"M40 100 L 280 100"}"##}),
+ ),
+ ],
+ vec![
+ call("t2", "look", json!({})),
+ call("t3", "klad", json!({"args": ["export", "-o", "/etc/stolen.png"]})),
+ ],
+ vec![json!({"type": "text", "text": "Done: one line across the middle."})],
+ ],
+ };
+ let (tx, rx) = channel();
+ let mut history = vec![json!({"role": "user", "content": "draw a line"})];
+ let stop = Arc::new(AtomicBool::new(false));
+ let edits = turn(
+ &mut script,
+ &d,
+ DEFAULT_MODEL,
+ &mut history,
+ Limits { spend: 0.0, spent: 0.0 },
+ &stop,
+ &tx,
+ )
+ .unwrap();
+ assert_eq!(edits, 1);
+ let events: Vec<Event> = rx.try_iter().collect();
+ let looked = events.iter().find_map(|e| match e {
+ Event::Done {
+ name, image: Some(png), ..
+ } if name == "look" => Some(png.clone()),
+ _ => None,
+ });
+ assert!(
+ looked.is_some_and(|p| p.starts_with(b"\x89PNG")),
+ "look returns the render as a picture"
+ );
+ let refused = events.iter().find_map(|e| match e {
+ Event::Done {
+ name,
+ text,
+ failed: true,
+ ..
+ } if name == "klad" => Some(text.clone()),
+ _ => None,
+ });
+ assert!(
+ refused.is_some_and(|t| t.contains("paths stay inside the drawing")),
+ "{events:?}"
+ );
+ assert!(matches!(events.last(), Some(Event::Finished { stop }) if stop == "end_turn"));
+ assert_eq!(
+ history.len(),
+ 6,
+ "user, assistant, results, assistant, results, assistant"
+ );
+ let bundle = Bundle::open(&d.bundle).unwrap();
+ assert_eq!(bundle.read_marks("l1", "c1").unwrap().len(), 1);
+ let (ok, _) = run(&d, &["undo".into(), bundle_arg(&d)], None).unwrap();
+ assert!(ok);
+ assert!(
+ Bundle::open(&d.bundle)
+ .unwrap()
+ .read_marks("l1", "c1")
+ .unwrap()
+ .is_empty()
+ );
+ let _ = std::fs::remove_dir_all(d.bundle.parent().unwrap());
+ }
+
+ #[test]
+ fn a_drawing_the_agent_may_not_edit_offers_only_looking_and_the_spend_limit_stops_it() {
+ let d = drawing("locked", false);
+ assert_eq!(
+ tools(false)
+ .iter()
+ .map(|t| t["name"].as_str().unwrap())
+ .collect::<Vec<_>>(),
+ ["look", "read"]
+ );
+ let mut script = Scripted {
+ replies: vec![
+ vec![call("t1", "apply", json!({"ops": "{}"}))],
+ vec![json!({"type": "text", "text": "ok"})],
+ ],
+ };
+ let (tx, rx) = channel();
+ let mut history = vec![json!({"role": "user", "content": "go"})];
+ let stop = Arc::new(AtomicBool::new(false));
+ turn(
+ &mut script,
+ &d,
+ DEFAULT_MODEL,
+ &mut history,
+ Limits { spend: 0.0, spent: 0.0 },
+ &stop,
+ &tx,
+ )
+ .unwrap();
+ let events: Vec<Event> = rx.try_iter().collect();
+ assert!(
+ events
+ .iter()
+ .any(|e| matches!(e, Event::Done { failed: true, text, .. } if text.contains("look and suggest only")))
+ );
+ let mut more = Scripted {
+ replies: vec![vec![json!({"type": "text", "text": "never"})]],
+ };
+ let (tx, rx) = channel();
+ turn(
+ &mut more,
+ &d,
+ DEFAULT_MODEL,
+ &mut history,
+ Limits { spend: 1.0, spent: 1.5 },
+ &stop,
+ &tx,
+ )
+ .unwrap();
+ assert!(
+ rx.try_iter()
+ .any(|e| matches!(e, Event::Finished { stop } if stop.contains("spend limit")))
+ );
+ assert_eq!(more.replies.len(), 1, "no request is made past the limit");
+ let _ = std::fs::remove_dir_all(d.bundle.parent().unwrap());
+ }
+
+ #[test]
+ fn paths_stay_in_the_drawing_and_base64_is_standard() {
+ let d = Drawing {
+ bundle: PathBuf::from("/w/art.draw"),
+ binary: PathBuf::new(),
+ editable: true,
+ };
+ assert!(allowed(&d, "art.draw/brushes/x.json").is_ok());
+ assert!(allowed(&d, "exports/hero.png").is_ok());
+ assert!(allowed(&d, "../other.draw/doc.json").is_err());
+ assert!(allowed(&d, "/etc/passwd").is_err());
+ assert!(allowed(&d, "exports/../../x.png").is_err());
+ assert!(allowed(&d, "M10 20 L 30 40").is_ok());
+ assert_eq!(base64(b"klad"), "a2xhZA==");
+ assert_eq!(base64(b"kla"), "a2xh");
+ let u = Usage {
+ input: 1_000_000,
+ output: 100_000,
+ cache_read: 0,
+ cache_write: 0,
+ };
+ assert!((u.cost("claude-opus-5-5") - 6.0).abs() < 1e-9);
+ }
+}
diff --git a/crates/app/src/agent/secret.rs b/crates/app/src/agent/secret.rs
new file mode 100644
index 0000000..2c3f6e0
--- /dev/null
+++ b/crates/app/src/agent/secret.rs
@@ -0,0 +1,106 @@
+use anyhow::{Context, Result, bail};
+use std::io::Write;
+use std::path::PathBuf;
+use std::process::{Command, Stdio};
+
+const SERVICE: &str = "klad: Claude API key";
+const ACCOUNT: &str = "klad";
+
+fn file() -> Option<PathBuf> {
+ if let Some(path) = std::env::var_os("KLAD_AGENT_KEY_FILE") {
+ return Some(PathBuf::from(path));
+ }
+ crate::places::config().map(|c| c.join("klad").join("agent-key"))
+}
+
+fn keychain() -> bool {
+ cfg!(target_os = "macos") && std::env::var_os("KLAD_AGENT_KEY_FILE").is_none()
+}
+
+pub fn store(key: &str) -> Result<()> {
+ let key = key.trim();
+ if key.is_empty() || key.contains(['\n', '"', ' ']) {
+ bail!("that does not look like an API key");
+ }
+ if keychain() {
+ let mut child = Command::new("security")
+ .arg("-i")
+ .stdin(Stdio::piped())
+ .stdout(Stdio::null())
+ .stderr(Stdio::piped())
+ .spawn()
+ .context("starting security for the Keychain")?;
+ writeln!(
+ child.stdin.take().context("security's input")?,
+ "add-generic-password -U -a \"{ACCOUNT}\" -s \"{SERVICE}\" -w \"{key}\""
+ )?;
+ let out = child.wait_with_output()?;
+ if !out.status.success() {
+ bail!(
+ "the Keychain refused the key: {}",
+ String::from_utf8_lossy(&out.stderr).trim()
+ );
+ }
+ return Ok(());
+ }
+ let path = file().context("no settings folder for the key")?;
+ if let Some(dir) = path.parent() {
+ std::fs::create_dir_all(dir)?;
+ }
+ std::fs::write(&path, key)?;
+ #[cfg(unix)]
+ {
+ use std::os::unix::fs::PermissionsExt;
+ std::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o600))?;
+ }
+ Ok(())
+}
+
+pub fn load() -> Option<String> {
+ if keychain() {
+ let out = Command::new("security")
+ .args(["find-generic-password", "-a", ACCOUNT, "-s", SERVICE, "-w"])
+ .stderr(Stdio::null())
+ .output()
+ .ok()?;
+ let key = String::from_utf8_lossy(&out.stdout).trim().to_string();
+ return (out.status.success() && !key.is_empty()).then_some(key);
+ }
+ std::fs::read_to_string(file()?)
+ .ok()
+ .map(|k| k.trim().to_string())
+ .filter(|k| !k.is_empty())
+}
+
+pub fn forget() -> Result<()> {
+ if keychain() {
+ let _ = Command::new("security")
+ .args(["delete-generic-password", "-a", ACCOUNT, "-s", SERVICE])
+ .output();
+ return Ok(());
+ }
+ if let Some(path) = file().filter(|p| p.exists()) {
+ std::fs::remove_file(path)?;
+ }
+ Ok(())
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+
+ #[test]
+ #[cfg(unix)]
+ fn a_key_is_kept_out_of_settings_in_a_file_only_its_owner_reads() {
+ use std::os::unix::fs::PermissionsExt;
+ let path = std::env::temp_dir().join(format!("klad-key-{}", std::process::id()));
+ unsafe { std::env::set_var("KLAD_AGENT_KEY_FILE", &path) };
+ store("sk-ant-test-123").unwrap();
+ assert_eq!(load().as_deref(), Some("sk-ant-test-123"));
+ assert_eq!(std::fs::metadata(&path).unwrap().permissions().mode() & 0o777, 0o600);
+ assert!(store("not a key").is_err());
+ forget().unwrap();
+ assert!(load().is_none());
+ unsafe { std::env::remove_var("KLAD_AGENT_KEY_FILE") };
+ }
+}
diff --git a/crates/app/src/commands.rs b/crates/app/src/commands.rs
index d32adc5..e57782c 100644
--- a/crates/app/src/commands.rs
+++ b/crates/app/src/commands.rs
@@ -114,6 +114,7 @@ pub enum Cmd {
InspectorPane,
NotesPane,
HistoryPane,
+ AgentPane,
Timeline,
Numbers,
Settings,
@@ -149,6 +150,7 @@ pub enum Cmd {
OpenRecent,
CopyPath,
InstallCommand,
+ ReportProblem,
PaintOver,
CompareReference,
About,
@@ -473,7 +475,7 @@ use Repeat as R;
use Trigger as T;
impl Cmd {
- pub const ALL: [Cmd; 155] = [
+ pub const ALL: [Cmd; 157] = [
Cmd::Brush,
Cmd::Eraser,
Cmd::Fill,
@@ -584,6 +586,7 @@ impl Cmd {
Cmd::InspectorPane,
Cmd::NotesPane,
Cmd::HistoryPane,
+ Cmd::AgentPane,
Cmd::Timeline,
Cmd::Numbers,
Cmd::Settings,
@@ -619,6 +622,7 @@ impl Cmd {
Cmd::OpenRecent,
Cmd::CopyPath,
Cmd::InstallCommand,
+ Cmd::ReportProblem,
Cmd::About,
Cmd::Export,
Cmd::AddFont,
@@ -1623,6 +1627,15 @@ impl Cmd {
R::No,
"show or hide it in the dock",
),
+ Cmd::AgentPane => spec(
+ "panel.agent",
+ "agent panel",
+ icon::SPARKLE,
+ G::Panels,
+ T::None,
+ R::No,
+ "ask the agent for a change in plain words and watch it work on this drawing",
+ ),
Cmd::HistoryPane => spec(
"panel.history",
"history panel",
@@ -2001,6 +2014,15 @@ impl Cmd {
R::No,
"repaint what is below this layer with brush strokes that follow its forms, and carry them through an animation",
),
+ Cmd::ReportProblem => spec(
+ "file.report",
+ "report a problem",
+ icon::LOG,
+ G::File,
+ T::None,
+ R::No,
+ "copies the version, the system and the last messages, to paste into an issue on GitHub",
+ ),
Cmd::InstallCommand => spec(
"file.install-command",
"install the klad command",
diff --git a/crates/app/src/editor.rs b/crates/app/src/editor.rs
index 62ac61f..713524a 100644
--- a/crates/app/src/editor.rs
+++ b/crates/app/src/editor.rs
@@ -1,3 +1,4 @@
+mod agentpane;
mod between;
mod bones;
mod brushwin;
@@ -614,6 +615,7 @@ pub(crate) struct Editor {
compare: Option<compare::Compared>,
bones: bones::BoneTool,
key_ease: klad_core::keys::Ease,
+ agent: agentpane::AgentPane,
bindings: crate::commands::Bindings,
capturing: Option<Cmd>,
key_filter: String,
@@ -775,6 +777,7 @@ impl Editor {
compare: None,
bones: bones::BoneTool::default(),
key_ease: klad_core::keys::Ease::Linear,
+ agent: agentpane::AgentPane::default(),
bindings: crate::commands::Bindings::default(),
capturing: None,
key_filter: String::new(),
diff --git a/crates/app/src/editor/agentpane.rs b/crates/app/src/editor/agentpane.rs
new file mode 100644
index 0000000..f8dc0a2
--- /dev/null
+++ b/crates/app/src/editor/agentpane.rs
@@ -0,0 +1,448 @@
+use super::Editor;
+use crate::agent::{self, Drawing, Event, Limits, Provider, Usage};
+use crate::messages::Tone;
+use crate::theme::{self, Hint};
+use anyhow::Result;
+use egui::RichText;
+use klad_core::history::Stack;
+use serde_json::{Value, json};
+use std::sync::Arc;
+use std::sync::atomic::{AtomicBool, Ordering};
+use std::sync::mpsc::{Receiver, channel};
+
+pub(super) enum Entry {
+ User(String),
+ Text(String),
+ Thinking(String),
+ Tool {
+ name: String,
+ input: Value,
+ text: String,
+ image: Option<egui::TextureHandle>,
+ failed: bool,
+ done: bool,
+ },
+ Note(String),
+}
+
+struct Running {
+ events: Receiver<Event>,
+ done: Receiver<(Vec<Value>, Result<usize, String>)>,
+ stop: Arc<AtomicBool>,
+}
+
+#[derive(Default)]
+pub(super) struct AgentPane {
+ pub input: String,
+ pub entries: Vec<Entry>,
+ running: Option<Running>,
+ history: Vec<Value>,
+ pub session: Usage,
+ pub turn: Usage,
+ pub last_edits: usize,
+ pub key: String,
+ pub models: Vec<String>,
+ pub script: Option<Vec<Vec<Value>>>,
+}
+
+fn summary(name: &str, input: &Value) -> String {
+ match name {
+ "klad" => input["args"]
+ .as_array()
+ .map(|a| a.iter().filter_map(Value::as_str).collect::<Vec<_>>().join(" "))
+ .unwrap_or_default(),
+ "apply" => format!(
+ "{} ops",
+ input["ops"]
+ .as_str()
+ .map_or(0, |o| o.lines().filter(|l| !l.trim().is_empty()).count())
+ ),
+ "look" => ["frame", "region", "scale", "overlay"]
+ .iter()
+ .filter_map(|k| {
+ input
+ .get(*k)
+ .map(|v| format!("{k} {}", v.to_string().trim_matches('"')))
+ })
+ .collect::<Vec<_>>()
+ .join(", "),
+ "read" => input["what"].as_str().unwrap_or_default().to_string(),
+ _ => String::new(),
+ }
+}
+
+fn tokens(n: u64) -> String {
+ match n {
+ n if n >= 10_000 => format!("{:.0}k", n as f64 / 1000.0),
+ n if n >= 1000 => format!("{:.1}k", n as f64 / 1000.0),
+ n => n.to_string(),
+ }
+}
+
+impl Editor {
+ pub(super) fn agent_running(&self) -> bool {
+ self.agent.running.is_some()
+ }
+
+ fn editable(&self) -> bool {
+ let path = self.scene.bundle.root.display().to_string();
+ !self.settings.agent.locked.contains(&path)
+ }
+
+ pub(super) fn ask_agent(&mut self, text: String) -> Result<()> {
+ if self.agent_running() || text.trim().is_empty() {
+ return Ok(());
+ }
+ let provider: Box<dyn Provider> = match self.agent.script.take() {
+ Some(replies) => Box::new(agent::Scripted { replies }),
+ None => {
+ let key = agent::secret::load()
+ .ok_or_else(|| anyhow::anyhow!("add a Claude API key first: the settings window, under agent"))?;
+ Box::new(agent::claude::Claude { key })
+ }
+ };
+ let binary =
+ crate::start::draw_binary().ok_or_else(|| anyhow::anyhow!("the klad command is not next to the editor"))?;
+ let drawing = Drawing {
+ bundle: self.scene.bundle.root.clone(),
+ binary,
+ editable: self.editable(),
+ };
+ let model = self.settings.agent.model.clone();
+ let limits = Limits {
+ spend: self.settings.agent.limit as f64,
+ spent: self.agent.session.cost(&model),
+ };
+ let mut history = std::mem::take(&mut self.agent.history);
+ self.agent.entries.push(Entry::User(text.clone()));
+ self.agent.turn = Usage::default();
+ let (tx, events) = channel();
+ let (finished, done) = channel();
+ let stop = Arc::new(AtomicBool::new(false));
+ let halt = stop.clone();
+ let ctx = self.ctx.clone();
+ std::thread::Builder::new().name("klad agent".into()).spawn(move || {
+ let mut provider = provider;
+ let read = |what: &str| {
+ std::process::Command::new(&drawing.binary)
+ .args([what, &drawing.bundle.to_string_lossy()])
+ .output()
+ .ok()
+ .map(|o| String::from_utf8_lossy(&o.stdout).trim().to_string())
+ .unwrap_or_default()
+ };
+ let (selection, notes) = (read("selection"), read("notes"));
+ let mut message = text;
+ [
+ ("What the user has selected", selection),
+ ("Notes pinned on the canvas", notes),
+ ]
+ .into_iter()
+ .filter(|(_, v)| !v.is_empty() && !v.starts_with("nothing") && !v.starts_with("no "))
+ .for_each(|(k, v)| message.push_str(&format!("\n\n{k}:\n{v}")));
+ history.push(json!({"role": "user", "content": message}));
+ let result = agent::turn(provider.as_mut(), &drawing, &model, &mut history, limits, &halt, &tx)
+ .map_err(|e| format!("{e:#}"));
+ if let Err(e) = &result {
+ let _ = tx.send(Event::Failed(e.clone()));
+ if history.last().is_some_and(|m| m["role"] == "user") {
+ history.pop();
+ }
+ }
+ let _ = finished.send((history, result));
+ ctx.request_repaint();
+ })?;
+ self.agent.running = Some(Running { events, done, stop });
+ Ok(())
+ }
+
+ pub(super) fn poll_agent(&mut self) {
+ let Some(running) = self.agent.running.as_ref() else {
+ return;
+ };
+ let events: Vec<Event> = running.events.try_iter().collect();
+ let finished = running.done.try_recv().ok();
+ events.into_iter().for_each(|e| match e {
+ Event::Text(t) => match self.agent.entries.last_mut() {
+ Some(Entry::Text(s)) => s.push_str(&t),
+ _ => self.agent.entries.push(Entry::Text(t)),
+ },
+ Event::Thinking(t) => match self.agent.entries.last_mut() {
+ Some(Entry::Thinking(s)) => s.push_str(&t),
+ _ => self.agent.entries.push(Entry::Thinking(t)),
+ },
+ Event::Tool { name, input } => self.agent.entries.push(Entry::Tool {
+ name,
+ input,
+ text: String::new(),
+ image: None,
+ failed: false,
+ done: false,
+ }),
+ Event::Done {
+ text: t,
+ image: png,
+ failed: f,
+ ..
+ } => {
+ let texture = png.as_deref().and_then(|bytes| {
+ let rgba = klad_render::assets::decode(bytes).ok()?;
+ let image = egui::ColorImage::from_rgba_unmultiplied(
+ [rgba.width as usize, rgba.height as usize],
+ &rgba.pixels,
+ );
+ Some(self.ctx.load_texture("agent look", image, egui::TextureOptions::LINEAR))
+ });
+ if let Some(Entry::Tool {
+ text,
+ image,
+ failed,
+ done,
+ ..
+ }) = self
+ .agent
+ .entries
+ .iter_mut()
+ .rev()
+ .find(|e| matches!(e, Entry::Tool { done: false, .. }))
+ {
+ *text = t;
+ *image = texture;
+ *failed = f;
+ *done = true;
+ }
+ }
+ Event::Usage(u) => {
+ self.agent.turn.add(u);
+ self.agent.session.add(u);
+ }
+ Event::Finished { stop } if stop != "end_turn" && stop != "tool_use" => {
+ self.agent.entries.push(Entry::Note(match stop.as_str() {
+ "max_tokens" => "the reply hit its length limit".into(),
+ "refusal" => "the model declined this request".into(),
+ other => other.to_string(),
+ }))
+ }
+ Event::Finished { .. } => {}
+ Event::Failed(e) => self.agent.entries.push(Entry::Note(format!("stopped: {e}"))),
+ });
+ if let Some((history, result)) = finished {
+ self.agent.history = history;
+ self.agent.last_edits = result.as_ref().map_or(0, |n| *n);
+ self.agent.running = None;
+ if let Err(e) = result {
+ self.say(Tone::Error, format!("the agent: {e}"));
+ }
+ self.check(|ed| ed.reload());
+ }
+ }
+
+ pub(super) fn undo_agent_turn(&mut self) -> Result<()> {
+ let n = std::mem::take(&mut self.agent.last_edits);
+ (0..n).try_for_each(|_| self.step(Stack::Undo))?;
+ self.agent
+ .entries
+ .push(Entry::Note(format!("undid the agent's last turn ({n} edits)")));
+ Ok(())
+ }
+
+ pub(super) fn agent_pane(&mut self, ui: &mut egui::Ui) -> Result<()> {
+ self.poll_agent();
+ if !self.settings.agent.notice_seen {
+ ui.group(|ui| {
+ ui.label(
+ "The agent sends your messages, what you selected, your notes for it, renders of this drawing and the commands it runs to the model's provider (Anthropic, with your own API key). It changes this drawing only while it may edit it, and you can undo a whole turn.",
+ );
+ if ui.button("understood").hint("shown once").clicked() {
+ self.settings.agent.notice_seen = true;
+ self.settings_dirty = true;
+ }
+ });
+ return Ok(());
+ }
+ let mut send = false;
+ egui::ScrollArea::vertical()
+ .max_height(ui.available_height() - 110.0)
+ .stick_to_bottom(true)
+ .auto_shrink([false, false])
+ .show(ui, |ui| {
+ if self.agent.entries.is_empty() {
+ ui.label(
+ RichText::new("Ask for a change in plain words: \"paint the rock as a round form lit from the left\", \"ink everything on the sketch layer\". Select strokes or pin notes first to point at things.")
+ .color(theme::WEAK),
+ );
+ }
+ self.agent.entries.iter().for_each(|e| match e {
+ Entry::User(t) => {
+ ui.label(RichText::new(t).strong());
+ }
+ Entry::Text(t) => {
+ ui.label(t);
+ }
+ Entry::Thinking(t) if !t.trim().is_empty() => {
+ ui.label(RichText::new(t).italics().small().color(theme::WEAK));
+ }
+ Entry::Thinking(_) => {}
+ Entry::Tool { name, input, text, image, failed, done } => {
+ let mark = match (done, failed) {
+ (false, _) => "...",
+ (true, true) => "failed",
+ (true, false) => "",
+ };
+ ui.collapsing(RichText::new(format!("{name} {} {mark}", summary(name, input))).small(), |ui| {
+ if let Some(t) = image {
+ let size = t.size_vec2();
+ let k = (ui.available_width() / size.x.max(1.0)).min(1.0);
+ ui.image((t.id(), size * k));
+ }
+ ui.label(RichText::new(text).small().monospace());
+ });
+ }
+ Entry::Note(t) => {
+ ui.label(RichText::new(t).small().color(theme::REFUSED));
+ }
+ });
+ });
+ ui.separator();
+ let r = ui.add(
+ egui::TextEdit::multiline(&mut self.agent.input)
+ .desired_rows(2)
+ .desired_width(f32::INFINITY)
+ .hint_text("what should the agent do?"),
+ );
+ if r.has_focus() && ui.input(|i| i.key_pressed(egui::Key::Enter) && i.modifiers.command) {
+ send = true;
+ }
+ ui.horizontal(|ui| {
+ match self.agent_running() {
+ true => {
+ if ui.button("stop").hint("ends the turn at the next tool call").clicked()
+ && let Some(r) = &self.agent.running
+ {
+ r.stop.store(true, Ordering::Relaxed);
+ }
+ }
+ false => {
+ if ui
+ .button("send")
+ .hint("cmd Return; your selection and notes go with it")
+ .clicked()
+ {
+ send = true;
+ }
+ }
+ }
+ let undo = self.agent.last_edits > 0 && !self.agent_running();
+ if ui
+ .add_enabled(undo, egui::Button::new("undo this turn"))
+ .hint("every change the agent's last turn made, in one go")
+ .clicked()
+ {
+ self.check(|e| e.undo_agent_turn());
+ }
+ let mut editable = self.editable();
+ if ui
+ .checkbox(&mut editable, "may edit")
+ .hint("off: the agent can look and suggest, not change this drawing")
+ .changed()
+ {
+ let path = self.scene.bundle.root.display().to_string();
+ self.settings.agent.locked.retain(|p| p != &path);
+ if !editable {
+ self.settings.agent.locked.push(path);
+ }
+ self.settings_dirty = true;
+ }
+ });
+ let model = self.settings.agent.model.clone();
+ ui.label(
+ RichText::new(format!(
+ "{model}: this turn {} in, {} out, ${:.2}; session ${:.2} of ${:.2}",
+ tokens(self.agent.turn.input + self.agent.turn.cache_read + self.agent.turn.cache_write),
+ tokens(self.agent.turn.output),
+ self.agent.turn.cost(&model),
+ self.agent.session.cost(&model),
+ self.settings.agent.limit
+ ))
+ .small()
+ .color(theme::WEAK),
+ );
+ if send {
+ let text = std::mem::take(&mut self.agent.input);
+ self.ask_agent(text)?;
+ }
+ if self.agent_running() {
+ ui.ctx().request_repaint_after(std::time::Duration::from_millis(100));
+ }
+ Ok(())
+ }
+
+ pub(super) fn agent_settings(&mut self, ui: &mut egui::Ui) {
+ ui.label(RichText::new("agent").strong());
+ let has_key = agent::secret::load().is_some();
+ ui.horizontal(|ui| {
+ let label = ui.label(if has_key {
+ "Claude API key (saved)"
+ } else {
+ "Claude API key"
+ });
+ ui.add(
+ egui::TextEdit::singleline(&mut self.agent.key)
+ .password(true)
+ .desired_width(180.0),
+ )
+ .labelled_by(label.id)
+ .hint("a key from the Claude Console; kept in the Keychain on a Mac, never in settings or the drawing");
+ if ui.button("save").hint("keep the key and check it").clicked() {
+ let key = std::mem::take(&mut self.agent.key);
+ match agent::secret::store(&key).and_then(|_| agent::claude::models(&key)) {
+ Ok(models) => {
+ self.say(Tone::Done, format!("key saved; {} models", models.len()));
+ self.agent.models = models;
+ }
+ Err(e) => self.say(Tone::Error, format!("the key: {e:#}")),
+ }
+ }
+ if has_key && ui.button("forget").hint("remove the key").clicked() {
+ let _ = agent::secret::forget();
+ self.say(Tone::Done, "the key is removed");
+ }
+ });
+ ui.horizontal(|ui| {
+ let models = match self.agent.models.is_empty() {
+ true => vec![
+ agent::DEFAULT_MODEL.to_string(),
+ "claude-sonnet-5-5".into(),
+ "claude-haiku-5-5".into(),
+ ],
+ false => self.agent.models.clone(),
+ };
+ egui::ComboBox::from_id_salt("agent model")
+ .selected_text(&self.settings.agent.model)
+ .show_ui(ui, |ui| {
+ models.iter().for_each(|m| {
+ if ui.selectable_label(self.settings.agent.model == *m, m).clicked() {
+ self.settings.agent.model = m.clone();
+ self.settings_dirty = true;
+ }
+ });
+ })
+ .response
+ .hint("the model the agent uses");
+ let mut limit = self.settings.agent.limit;
+ if ui
+ .add(
+ egui::DragValue::new(&mut limit)
+ .range(0.0..=100.0)
+ .speed(0.1)
+ .prefix("spend limit $"),
+ )
+ .hint("the agent stops before the next call once a session has cost this much; 0 has no limit")
+ .changed()
+ {
+ self.settings.agent.limit = limit;
+ self.settings_dirty = true;
+ }
+ });
+ }
+}
diff --git a/crates/app/src/editor/commands.rs b/crates/app/src/editor/commands.rs
index 0f89bea..51383f9 100644
--- a/crates/app/src/editor/commands.rs
+++ b/crates/app/src/editor/commands.rs
@@ -338,6 +338,7 @@ impl Editor {
Cmd::InspectorPane => layout.dock && layout.is_open(Pane::Inspector),
Cmd::NotesPane => layout.dock && layout.is_open(Pane::Notes),
Cmd::HistoryPane => layout.dock && layout.is_open(Pane::History),
+ Cmd::AgentPane => layout.dock && layout.is_open(Pane::Agent),
Cmd::Timeline => layout.timeline,
Cmd::Numbers => layout.numbers,
Cmd::Settings => self.windows.settings,
@@ -598,6 +599,14 @@ impl Editor {
Cmd::ShowParts => self.checks.parts = !self.checks.parts,
Cmd::ShowArcs => self.show_arcs()?,
Cmd::InstallCommand => self.start_job(install_job()),
+ Cmd::ReportProblem => {
+ let text = self.problem_report();
+ self.ctx.copy_text(text);
+ self.say(
+ Tone::Done,
+ "copied a problem report: paste it into an issue at github.com/klad-app/klad",
+ );
+ }
Cmd::CopyPath => {
let path = self.scene.bundle.root.display().to_string();
self.ctx.copy_text(path.clone());
@@ -770,6 +779,7 @@ impl Editor {
Cmd::InspectorPane => self.pane(Pane::Inspector),
Cmd::NotesPane => self.pane(Pane::Notes),
Cmd::HistoryPane => self.pane(Pane::History),
+ Cmd::AgentPane => self.pane(Pane::Agent),
Cmd::Timeline => {
self.settings.layout.timeline = !self.settings.layout.timeline;
self.settings_dirty = true;
@@ -933,3 +943,29 @@ pub(crate) fn install_job() -> Spec {
failure: |_, err| err.lines().last().unwrap_or_default().to_string(),
}
}
+
+impl Editor {
+ pub(crate) fn problem_report(&self) -> String {
+ let lines: Vec<String> = self
+ .messages
+ .log()
+ .rev()
+ .take(40)
+ .map(|m| format!("{:?}: {}", m.tone, m.text))
+ .collect::<Vec<_>>()
+ .into_iter()
+ .rev()
+ .collect();
+ format!(
+ "klad {}\n{} {}\ncanvas {}x{}, {} layers, {} frames\n\nlast messages:\n{}\n",
+ crate::VERSION,
+ std::env::consts::OS,
+ std::env::consts::ARCH,
+ self.scene.bundle.doc.canvas.width,
+ self.scene.bundle.doc.canvas.height,
+ self.scene.bundle.doc.all_layers().len(),
+ self.scene.bundle.doc.timeline.frames,
+ lines.join("\n")
+ )
+ }
+}
diff --git a/crates/app/src/editor/panels.rs b/crates/app/src/editor/panels.rs
index 585bedd..76d9d3f 100644
--- a/crates/app/src/editor/panels.rs
+++ b/crates/app/src/editor/panels.rs
@@ -265,6 +265,7 @@ impl Editor {
(Cmd::AddFont, "add a font"),
(Cmd::CopyPath, "copy path"),
(Cmd::InstallCommand, "install the klad command"),
+ (Cmd::ReportProblem, "report a problem"),
(Cmd::About, "about and licences"),
]
.into_iter()
@@ -315,6 +316,7 @@ impl Editor {
(Cmd::InspectorPane, Pane::Inspector),
(Cmd::NotesPane, Pane::Notes),
(Cmd::HistoryPane, Pane::History),
+ (Cmd::AgentPane, Pane::Agent),
]
.into_iter()
.for_each(|(cmd, pane)| {
@@ -439,6 +441,7 @@ impl Editor {
},
Pane::Notes => self.notes_pane(ui),
Pane::History => self.history_pane(ui),
+ Pane::Agent => self.agent_pane(ui),
}
}
@@ -848,6 +851,8 @@ impl Editor {
ui.separator();
ui.heading("Layout");
self.scale_section(ui);
+ ui.separator();
+ self.agent_settings(ui);
ui.horizontal_wrapped(|ui| {
self.command_check(ui, Cmd::Numbers, "numbers panel");
self.command_check(ui, Cmd::Timeline, "timeline");
diff --git a/crates/app/src/editor/ui_tests.rs b/crates/app/src/editor/ui_tests.rs
index ef60c3f..8f9fc5d 100644
--- a/crates/app/src/editor/ui_tests.rs
+++ b/crates/app/src/editor/ui_tests.rs
@@ -2612,3 +2612,59 @@ fn a_layer_is_keyed_from_the_inspector_and_eases_between_its_keys() {
);
let _ = std::fs::remove_dir_all(&path);
}
+
+#[test]
+fn the_agent_panel_runs_a_scripted_turn_streams_it_and_undoes_it_in_one_go() {
+ let path = sample("agent-panel");
+ let mut h = harness(editor(&path), vec2(1440.0, 1000.0));
+ [Pane::Tool, Pane::Layers, Pane::Inspector, Pane::Notes, Pane::History]
+ .into_iter()
+ .for_each(|p| h.state_mut().settings.layout.toggle(p));
+ h.state_mut().invoke(Cmd::AgentPane).unwrap();
+ h.run_steps(2);
+ assert!(h.query_by_label("understood").is_some(), "the notice comes first");
+ h.get_by_label("understood").click();
+ h.run_steps(2);
+ let before = h.state().scene.bundle.read_marks("l6", "c6").unwrap().len();
+ h.state_mut().agent.script = Some(vec![
+ vec![
+ serde_json::json!({"type": "text", "text": "Adding a stroke."}),
+ serde_json::json!({"type": "tool_use", "id": "t1", "name": "apply", "input": {"ops": r##"{"op":"stroke","layer":"l6","brush":"ink","size":12,"color":"#802020","path":"M50 300 L 450 320"}"##}}),
+ ],
+ vec![serde_json::json!({"type": "tool_use", "id": "t2", "name": "look", "input": {}})],
+ vec![serde_json::json!({"type": "text", "text": "One red stroke along the bottom."})],
+ ]);
+ h.state_mut().agent.input = "draw a red line at the bottom".into();
+ h.get_by_label("send").click();
+ h.run_steps(1);
+ settle(&mut h, |ed| !ed.agent_running());
+ h.run_steps(3);
+ assert!(h.query_by_label("One red stroke along the bottom.").is_some());
+ assert!(h.query_all(By::new().label_contains("apply 1 ops")).count() > 0);
+ assert!(h.query_all(By::new().label_contains("look")).count() > 0);
+ settle(&mut h, |ed| {
+ ed.scene.bundle.read_marks("l6", "c6").map_or(0, |m| m.len()) == before + 1
+ });
+ assert_eq!(h.state().agent.last_edits, 1);
+ h.get_by_label("undo this turn").click();
+ h.run_steps(2);
+ settle(&mut h, |ed| {
+ ed.scene.bundle.read_marks("l6", "c6").map_or(0, |m| m.len()) == before
+ });
+ let _ = std::fs::remove_dir_all(&path);
+}
+
+#[test]
+fn a_problem_report_holds_the_version_the_system_and_the_last_messages() {
+ let path = sample("report");
+ let mut h = harness(editor(&path), vec2(1280.0, 900.0));
+ h.state_mut().say(Tone::Error, "the fill leaked out of the lines");
+ let report = h.state().problem_report();
+ assert!(report.starts_with(&format!("klad {}", crate::VERSION)));
+ assert!(report.contains(std::env::consts::OS) && report.contains("the fill leaked out of the lines"));
+ assert!(
+ !report.contains(&path.display().to_string()),
+ "the drawing's path stays private"
+ );
+ let _ = std::fs::remove_dir_all(&path);
+}
diff --git a/crates/app/src/layout.rs b/crates/app/src/layout.rs
index b928640..773203e 100644
--- a/crates/app/src/layout.rs
+++ b/crates/app/src/layout.rs
@@ -19,6 +19,7 @@ pub enum Pane {
Inspector,
Notes,
History,
+ Agent,
}
impl Pane {
@@ -29,6 +30,7 @@ impl Pane {
Pane::Inspector => "Inspector",
Pane::Notes => "Notes for the agent",
Pane::History => "History",
+ Pane::Agent => "Agent",
}
}
@@ -39,6 +41,7 @@ impl Pane {
Pane::Inspector => 110.0,
Pane::Notes => 60.0,
Pane::History => 60.0,
+ Pane::Agent => 160.0,
}
}
@@ -49,6 +52,7 @@ impl Pane {
Pane::Inspector => 220.0,
Pane::Notes => 90.0,
Pane::History => 80.0,
+ Pane::Agent => 320.0,
}
}
}
@@ -92,11 +96,12 @@ impl Default for Layout {
(Pane::Inspector, false),
(Pane::Notes, true),
(Pane::History, true),
+ (Pane::Agent, false),
]
.into_iter()
.map(|(pane, collapsed)| PaneState {
pane,
- open: true,
+ open: pane != Pane::Agent,
collapsed,
body: pane.default_body(),
})
@@ -321,7 +326,8 @@ mod tests {
)
.unwrap()
.repaired();
- assert_eq!(old.panes.len(), 5);
+ assert_eq!(old.panes.len(), 6);
+ assert!(!old.is_open(Pane::Agent), "the agent pane comes closed until it is asked for");
assert_eq!(old.panes[0].pane, Pane::Layers);
assert_eq!(old.panes[0].body, Pane::Layers.min_body());
}
diff --git a/crates/app/src/lib.rs b/crates/app/src/lib.rs
index 471f537..e710d66 100644
--- a/crates/app/src/lib.rs
+++ b/crates/app/src/lib.rs
@@ -1,3 +1,4 @@
+mod agent;
mod color;
mod commands;
mod composite;
diff --git a/crates/app/src/settings.rs b/crates/app/src/settings.rs
index b85d5b2..993be62 100644
--- a/crates/app/src/settings.rs
+++ b/crates/app/src/settings.rs
@@ -83,6 +83,28 @@ pub struct Settings {
pub check_updates: bool,
#[serde(default, rename = "updatesChecked")]
pub updates_checked: u64,
+ #[serde(default)]
+ pub agent: AgentSettings,
+}
+
+#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
+#[serde(default, rename_all = "camelCase")]
+pub struct AgentSettings {
+ pub model: String,
+ pub limit: f32,
+ pub notice_seen: bool,
+ pub locked: Vec<String>,
+}
+
+impl Default for AgentSettings {
+ fn default() -> Self {
+ Self {
+ model: crate::agent::DEFAULT_MODEL.to_string(),
+ limit: 2.0,
+ notice_seen: false,
+ locked: Vec::new(),
+ }
+ }
}
fn yes() -> bool {
@@ -104,6 +126,7 @@ impl Default for Settings {
drawings: None,
check_updates: true,
updates_checked: 0,
+ agent: AgentSettings::default(),
}
}
}
diff --git a/docs/DESIGN.md b/docs/DESIGN.md
index fa66c49..5ec6562 100644
--- a/docs/DESIGN.md
+++ b/docs/DESIGN.md
@@ -1,4 +1,4 @@
-_Last updated: 2026-11-05 22:06_
+_Last updated: 2026-11-05 22:26_
# klad: design
@@ -243,6 +243,7 @@ Every layer has `blend`, `opacity`, `visible`, `locked`, optional `mask`, `clip`
- **Text (M5).** `{"type": "text", "text", "font", "x", "y", "size", "tracking", "leading", "align", "weight"}` with the node's fill and outline. The font is a file in the bundle's `fonts/` folder, so a drawing renders the same anywhere. Each line is shaped by `harfrust` (HarfBuzz's shaping in Rust: GPOS and `kern` kerning, ligatures, marks, scripts), `weight` sets a variable font's `wght` axis, and the outlines come from the file through `skrifa`, each glyph's contours merged with Skia's path ops so an outline never shows the overlaps variable fonts are built with; they are drawn as one path, never by the system's font engine, so the Mac and the container draw the same pixels. A missing font is refused with its name.
- **Fonts (M5, S5.26).** klad ships about fifty open families for game art in `fonts/` (sans, serif, mono, display, handwriting, pixel; OFL-1.1 or Apache-2.0, unchanged from the Google Fonts repository at a pinned commit, listed with their licences in the notices; `fonts/update.py` fetches them again). `klad_render::fonts` finds that folder next to the binary (also `../Resources/fonts` in a Mac app and the repository's `fonts/` for development builds; `KLAD_FONTS` overrides), and keeps families downloaded on request in a per-user folder (`~/Library/Application Support/klad/fonts`, `$XDG_DATA_HOME/klad/fonts`, `%APPDATA%\klad\fonts`). `fonts/catalogue.json` lists every open family on Google Fonts (about 1,900) with its group and folder at the same commit, so `klad fonts --search` works offline and a download (`klad fonts --get`, or naming a family anywhere a font is taken) fetches exactly that commit's files with `curl`. Choosing a font anywhere (the CLI's `font=`, `klad font`, the editor's picker) copies the file and its licence into the bundle; the node keeps the file name. This replaces M5 choice 3 (no fonts with the app). The editor's frame and plane caches fold `klad_render::RENDER_VERSION` into their keys, so pictures cached before a renderer change are not shown again. A light layer is no kind of its own: a paint layer set to `add` with a soft glow (a shortcut button for it only confused the layers pane).
- **Foliage (M5, S5.27).** `foliage` is a shape op (CLI only, principle 7): each closed shape becomes a clump built from leaf strokes on a jittered grid, each coloured from the clump lit as a round form (a normal from its bounding ellipse, mixed with the normal of an optional `mass` shape so clumps turn with the whole crown, a key light or a lamp, the underside darker) and stepped into three value families between `shade`, `color` and `lit` (OKLab; by default darker and cooler, lighter and warmer), with more variety in the light; then leaves along the outline, kept more where the outline faces the light or the sky, a few in `glow` on the shadow edge. Strokes are sorted dark to light so lit leaves land on top. It puts how foliage is painted (light families on the clump, the big mass before the texture, a broken silhouette) into one op, so an agent does not scatter leaves like confetti.
+- **The agent in the editor (M8).** The Agent pane (closed until asked for; `agent panel` in cmd K or the panels menu) runs a model with four tools over the same surface a terminal agent has (`app/src/agent`): `look` renders the drawing (`klad render`, the longest side fitted in 1024 px unless a region or scale is given) and returns the PNG as an image in the tool result; `read` returns the selection, notes, layer tree, a layer's strokes, frames, bones or keys; `klad` runs any subcommand with the drawing's path put after it, refusing `open`, `install`, `watch`, `bench`, `demo`, `stress`, `scribble`, `new` and `apply` and any argument that looks like a path outside the drawing or the `exports` folder next to it (lexically normalised, no `~`); `apply` takes op lines. With "may edit" off for a drawing (kept per drawing in settings), only `look` and `read` are offered. The turn loop (`agent::turn`) runs on a thread behind a `Provider` trait: it sends the history, streams text and progress notes to the pane, runs every tool call of a reply in order and returns all their results in one user message, stops at the next tool call when Stop is pressed, and refuses to start a request once the session's estimated cost reaches the spend limit. Each turn counts its edits; "undo this turn" steps the shared history back that many times. The user's message carries the selection and pinned notes. The Claude provider (`agent/claude.rs`) is raw HTTP through `curl` (no SDK for Rust): `POST /v1/messages` with `anthropic-version: 2026-10-21`, streaming SSE (text, thinking, signature and tool-input deltas assembled into whole blocks that go back unchanged, a tool input that does not parse is answered as an error instead of being run), `claude-opus-5-5` by default at effort medium, adaptive thinking with `display: "updates"` for progress notes, server-side `fallbacks: "default"` on the models that take it, `cache_control` on the system prompt (the agent skill) and the last tool plus automatic caching of the conversation, and `GET /v1/models` for the model list. The key never reaches the command line: curl reads its headers from stdin (`--config -`), the request body goes through a temporary file. The key is kept in the macOS Keychain (`security -i`, fed on stdin) or, elsewhere, in a file in the settings folder only its owner can read; never in settings, the drawing or a log. A notice says once what is sent. Tokens and an estimated cost show per turn and per session. A scripted provider drives the tests. "Report a problem" (document menu) copies the version, the system, the drawing's size and the last 40 messages, without its path.
- **Rigs (M6).** A paint layer or a group gets `bones` (format 9): each a name, an optional parent that comes before it, a head and a tail in canvas px. A group's skeleton poses every paint layer in it together, so a character's lines, colour and shading layers stay in step. Each bone has a weight map in `weights/<owner>.weights` (zstd: a JSON header with the grid size and bone names, then one byte per cell per bone, cells of 4 px), written by `Bundle::apply` with the edit and kept in its history step; weights are normalised across bones when read (bilinear between cells), and a place no bone weighs follows the nearest bone. Automatic weights are 1/(d + 0.15 L + 1)^3 by distance to each bone segment (L its length), so joints blend over a short stretch; `set` paints soft dabs (smoothstep falloff towards a value), `part` gives a named part's cells to one bone at 1 and the others 0, `smooth` blurs every map once. A pose (`pose` on a cel: its rest drawing's id, an angle per bone in degrees from rest, a root move) is linear blend skinning: each bone's rest matrix (head, angle) and posed matrix (its parent's posed matrix times the rest offset times its turn) give a skin matrix, and every stroke sample moves by the weighted sum; stroke sizes scale by the mean local stretch (the Jacobian's determinant, square-rooted); polygon fills move their points; flood fills are warped through a 4 px triangle mesh over their coverage (each posed triangle rasterised, sampling the rest coverage by barycentric position). The rest pose returns the marks untouched. `pose` (computed by the CLI, `cli/src/rig.rs`) makes or re-poses a drawing at `to` on every paint layer under the skeleton from the rest drawing shown at `from` (a posed drawing's own rest when `from` shows one). In-betweens of two poses of one rest drawing (or a pose and its rest) mix the angles and root by t instead of matching strokes (`rig::posed_pair`), and keep the mixed pose. The editor's bone tool (J) draws bones (from a bone's end makes a child), picks, renames and removes them, paints weights in its own process (`klad_core::rig::weights_for`) over a tint of the picked bone's map, and turns bones in pose mode, committing the pose through `klad pose` as a job; the drawing is not deformed live while turning, only the skeleton.
- **Keys (M6, S6.5).** Layers, groups and vector nodes keep `keys`: per property (`x`, `y`, `rotate`, `sx`, `sy`, `opacity`) a sorted list of frame, value and ease (linear, in, out, in-out, step, or a cubic curve x1,y1,x2,y2 solved by bisection). A segment follows its first key's ease; values hold before the first key and after the last. Layers and groups composite with their keyed opacity and, when moved, with their keyed transform about the canvas centre (drawn whole, not tile by tile, while moved); nodes take keyed transforms and opacity when rasterised (`nodes_at`). The editor's frame and plane caches hash the keyed values; its vector raster cache does not follow node keys yet, so keyed shapes move in `klad render` and the exports, not in the editor.
- **Godot scenes (M6, S6.6, S6.7).** `export --godot-scene` flattens each visible top-level layer (with its fog; the background with the first) into a texture on a `Parallax2D` with `scroll_scale` 1/(1 + depth), the factor the editor's depth preview (hold D) uses. `export --godot-rig` writes per skeleton: each part of the rest drawing (named parts of each paint layer's rest drawing, the loose marks as one) replayed alone, trimmed, on a `Polygon2D` with a 16 px triangle mesh over its pixels (one cell of margin), UVs, and per-vertex weights sampled from the maps; a `Skeleton2D` with a `Bone2D` per bone at rest (local position and angle against the parent, `rest`, `length`); and an `AnimationPlayer` with one animation per clip (or one for the timeline) whose value tracks hold each bone's rest angle plus its pose at the frame each posed drawing starts (stepped, so drawings hold as in klad); frames that are not poses of the rest drawing are listed for the frames export. Re-exports keep their uids. `samples/rig` builds a figure with all of it.