Skip to content

armgit / cua

Public Imported from GitHub
0

Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.

feat(telemetry): installer funnel events and an agent-used-a-Space event (CUA-1881) (#5457)

Francesco Bonacciba3e14fd8beb
diff --git a/docs/content/docs/cua-sdk/concepts/meta.json b/docs/content/docs/cua-sdk/concepts/meta.json
index a19fc7f501..6c31fe793b 100644
--- a/docs/content/docs/cua-sdk/concepts/meta.json
+++ b/docs/content/docs/cua-sdk/concepts/meta.json
@@ -4,6 +4,7 @@
"how-sandboxes-work",
"telemetry",
"telemetry-events",
+ "telemetry-counting",
"telemetry-error-kinds"
]
}
diff --git a/docs/content/docs/cua-sdk/concepts/telemetry-counting.mdx b/docs/content/docs/cua-sdk/concepts/telemetry-counting.mdx
new file mode 100644
index 0000000000..13baa22541
--- /dev/null
+++ b/docs/content/docs/cua-sdk/concepts/telemetry-counting.mdx
@@ -0,0 +1,61 @@
+---
+title: How events are counted
+description: How the installers, an AI agent using a Space, agent runs, teleport callers and session deliveries are counted.
+---
+
+Details behind some rows of [Telemetry events](/cua-sdk/concepts/telemetry-events).
+
+## Installers
+
+The installers (`install.sh`, `install.ps1`) send three events about their
+own run: `installer_started`, then `installer_completed` or
+`installer_failed` with the part that was running and a fixed reason word.
+They follow the same switches as every Cua program (`DO_NOT_TRACK`,
+`CUA_TELEMETRY`, the config file, CI) and send nothing on a dry run. Before
+the first one, the installer prints a short notice saying so. They create
+the same anonymous install id, so the app's later events continue the same
+install.
+
+`installer_failed` also carries `install_stage`, the part that was running
+(`platform`, `manifest`, `cli`, `runtime`, `app`, `driver`, `sign_in`, `host` or
+`other`), and `error_kind`, a fixed reason (`network`, `checksum`, `signature`,
+`unsupported_platform`, `permission_denied`, `missing_tool`, `cancelled` or
+`other`), never the message. Both are `none` on the other two steps. The
+installers send as product `cli`, with `$lib` `cua-installer`.
+
+## An AI agent using a Space
+
+`cua_agent_used_space` is sent when a tool call from an AI agent (through
+`cua mcp` or the daemon's MCP server) acts inside a Space and works: a
+command, a file, the screen, a service or an agent in the Space. It carries
+only the kind of tool, at most once per install per UTC day. Calls that only
+list, create or share Spaces do not count.
+
+## Agent runs and retention
+
+An agent run is counted by the install that started it: once when it
+starts, and once when any Cua program of that install first sees its first
+turn end, fail or stop. Until then a marker with the same fixed words waits
+in `~/.cua/telemetry/agent_runs/` (dropped after 7 days). Cua programs
+inside an agent run (`CUA_AGENT_RUN_ID` set) record no agent runs, so a
+Space never counts as an install. Retention counts installs with a
+`cua_app_active` event on day 1, 7 and 30 after their first one.
+
+## Teleport caller kinds
+
+`caller_kind` says who asked for a teleport: `cua_app_signed` (the Cua app or
+CLI, verified by its code signature), `embedded_sdk` (another app that embeds
+the cua SDK), `unsigned` (ad hoc or unsigned code) or `unknown` (the platform
+cannot verify signatures). With `caller_kind_source: broker_verified` it comes
+from the Keyvault broker's OS-verified caller identity; with `self_reported`
+the calling process reported it, and an SDK process only ever reports
+`embedded_sdk` or `unknown`. Paths, bundle ids and signing ids of callers are
+never sent.
+
+## Session deliveries
+
+A modified build can remove its own telemetry. The receive-side count
+(`cua_spacesd_session_deliveries`) is recorded by cua-spacesd inside the
+sandbox, and only when the host enabled telemetry for that sandbox
+(`CUA_SPACESD_TELEMETRY=1`, which the SDK sets on local sandboxes while
+telemetry is on).
diff --git a/docs/content/docs/cua-sdk/concepts/telemetry-events.mdx b/docs/content/docs/cua-sdk/concepts/telemetry-events.mdx
index 4e370a821c..a0a0ee4263 100644
--- a/docs/content/docs/cua-sdk/concepts/telemetry-events.mdx
+++ b/docs/content/docs/cua-sdk/concepts/telemetry-events.mdx
@@ -27,6 +27,8 @@ Every event carries: `telemetry_schema_version`, `event_version`, `product`
| `cua_spaces_feature_used` | `feature` (a fixed Spaces app feature) | Spaces app feature adoption. |
| `cua_stream_stats` | `transport`, `codec`, `decode` (`hw` or `sw`), `fps_bucket`, `latency_bucket`, `resolution`, `duration_bucket` | Streaming performance (50% sampled). |
| `cua_onboarding_step` | `step`, `outcome`, `onboarding_eligible` (`yes` or `no` on the Spaces app's `app_launched`: its first run was still due; `unknown` otherwise), `error_kind` (a failed `signed_in`: `timeout`, `cancelled`, `permission_denied`, `transport`, `unauthenticated` or `other`, never the message; `none` otherwise) | Install and onboarding funnel, and the Spaces checklist: `checklist_shown`, `checklist_<step>` (`add_machine`, `create_space`, `teleport`, `share`, `run_agent`), `checklist_completed`, `checklist_dismissed`, `checklist_reopened`. |
+| `cua_installer_step` | `step` (`installer_started`, `installer_completed`, `installer_failed`), `outcome`, `install_stage`, `error_kind`, `installer` (`sh` or `ps1`), `items` (what was asked for: `cli`, `spaces`, `cua_driver`, `host` joined with `+`) | Install funnel: how many installs start and finish, and where and why they fail ([values](/cua-sdk/concepts/telemetry-counting#installers)). |
+| `cua_agent_used_space` | `tool_group` (`shell`, `files`, `computer`, `services`, `agents` or `other`) | First success and agent retention: an AI agent's tool call acted inside a Space and worked. At most one per install per UTC day. |
| `cua_agent_run_started` | `harness` (cua-agents id or `other`), `location`, `entry` (`spaces_tool`, `cli`, `sdk`, `persistent`, `routine`), `outcome`, `error_kind` | Agent runs started, and failed starts. |
| `cua_agent_run_completed` | `harness`, `location`, `entry`, `outcome`, `error_kind`, `duration_bucket` | First agent run, and how reliably runs finish. Once per run. |
| `cua_bench_run_completed` | `taskset` (cua-bench dataset or `custom`), `score_bucket`, `task_count`, `outcome` | Benchmark use and coarse scores. |
@@ -50,20 +52,5 @@ Every event carries: `telemetry_schema_version`, `event_version`, `product`
| `cua_device_enroll` | `method` (`sign_in`, `approval`, `rekey`), `outcome` | How devices join your account's relay. |
| `cua_experiment` | `action` (`experiment_on` or `experiment_off`), `experiment` (`cua_volume`, `your_cloud`, `sharing`) | Which experiments people try, and turn off again. |
-## Teleport caller kinds
-
-`caller_kind` says who asked for a teleport: `cua_app_signed` (the Cua app or
-CLI, verified by its code signature), `embedded_sdk` (another app that embeds
-the cua SDK), `unsigned` (ad hoc or unsigned code) or `unknown` (the platform
-cannot verify signatures). With `caller_kind_source: broker_verified` it comes
-from the Keyvault broker's OS-verified caller identity; with `self_reported`
-the calling process reported it, and an SDK process only ever reports
-`embedded_sdk` or `unknown`. Paths, bundle ids and signing ids of callers are
-never sent.
-
-A modified build can remove its own telemetry. The receive-side count
-(`cua_spacesd_session_deliveries`) is recorded by cua-spacesd inside the
-sandbox, and only when the host enabled telemetry for that sandbox
-(`CUA_SPACESD_TELEMETRY=1`, which the SDK sets on local sandboxes while
-telemetry is on).
-
+Teleport caller kinds and how session deliveries are counted:
+[How events are counted](/cua-sdk/concepts/telemetry-counting).
diff --git a/docs/content/docs/cua-sdk/concepts/telemetry.mdx b/docs/content/docs/cua-sdk/concepts/telemetry.mdx
index e705f35831..4692e1132e 100644
--- a/docs/content/docs/cua-sdk/concepts/telemetry.mdx
+++ b/docs/content/docs/cua-sdk/concepts/telemetry.mdx
@@ -81,19 +81,16 @@ These are the questions the events answer, and nothing else:
- Are they still using Cua a day, a week and a month later?
The activation funnel joins events on the anonymous install id, in order:
-app launched (`cua_onboarding_step` `app_launched`), first run completed
+installer started and finished (`cua_installer_step`), app launched
+(`cua_onboarding_step` `app_launched`), first run completed
(`onboarding_completed`), signed in (`signed_in`), first Space created
(`first_space_created`), first Space ready (`first_space_ready`), first
-teleport (`first_teleport`) and first agent run (`first_agent_run`). The
-`first_*` steps are sent once per install.
-
-An agent run is counted by the install that started it: once when it
-starts, and once when any Cua program of that install first sees its first
-turn end, fail or stop. Until then a marker with the same fixed words waits
-in `~/.cua/telemetry/agent_runs/` (dropped after 7 days). Cua programs
-inside an agent run (`CUA_AGENT_RUN_ID` set) record no agent runs, so a
-Space never counts as an install. Retention counts installs with a
-`cua_app_active` event on day 1, 7 and 30 after their first one.
+teleport (`first_teleport`), first agent run (`first_agent_run`) and an
+AI agent using a Space (`cua_agent_used_space`). The `first_*` steps are sent
+once per install.
+
+How the installers, an AI agent using a Space, agent runs and retention
+are counted: [How events are counted](/cua-sdk/concepts/telemetry-counting).
## Never collected
diff --git a/docs/scripts/sandbox-navigation.test.ts b/docs/scripts/sandbox-navigation.test.ts
index 7819b7fc8f..ce0ace8dc2 100644
--- a/docs/scripts/sandbox-navigation.test.ts
+++ b/docs/scripts/sandbox-navigation.test.ts
@@ -49,7 +49,13 @@ test('the Cua SDK has its task guides and one concept page', async () => {
'omarchy',
'migrate-from-deprecated-packages',
]);
- assert.deepEqual((await metadata('cua-sdk/concepts')).pages, ['how-sandboxes-work', 'telemetry', 'telemetry-events']);
+ assert.deepEqual((await metadata('cua-sdk/concepts')).pages, [
+ 'how-sandboxes-work',
+ 'telemetry',
+ 'telemetry-events',
+ 'telemetry-counting',
+ 'telemetry-error-kinds',
+ ]);
for (const slug of guides.pages) {
assert.equal(await routeExists(`/cua-sdk/guides/${slug}`), true, slug);
}
diff --git a/libs/cua/crates/cua-spaces/src/mcp/mod.rs b/libs/cua/crates/cua-spaces/src/mcp/mod.rs
index bf8957406c..623204613d 100644
--- a/libs/cua/crates/cua-spaces/src/mcp/mod.rs
+++ b/libs/cua/crates/cua-spaces/src/mcp/mod.rs
@@ -391,7 +391,14 @@ impl McpServer {
));
}
let args = params.get("arguments").cloned().unwrap_or(json!({}));
- self.call_any(name, args).await.to_result()
+ let out = self.call_any(name, args).await;
+ // Only an MCP client (an AI agent) gets here; the app calls
+ // tools through `call`. Anonymous: the tool's kind, once per
+ // install per day (`cua_agent_used_space`).
+ if !out.is_error && cua_telemetry::events::space_tool_group(name).is_some() {
+ cua_telemetry::global().capture_agent_used_space(name);
+ }
+ out.to_result()
}
other => {
return Some(error(
diff --git a/libs/cua/crates/cua-spacesd-client/tests/environment_proxy.rs b/libs/cua/crates/cua-spacesd-client/tests/environment_proxy.rs
index c03b42b747..888fd70310 100644
--- a/libs/cua/crates/cua-spacesd-client/tests/environment_proxy.rs
+++ b/libs/cua/crates/cua-spacesd-client/tests/environment_proxy.rs
@@ -43,48 +43,62 @@ fn relay_rpc_uses_environment_proxy() {
"tls-web",
"tls-http",
] {
- let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
- let address = listener.local_addr().unwrap();
- drop(listener);
- let mut child = Command::new(std::env::current_exe().unwrap());
- child.args(["--exact", "relay_rpc_uses_environment_proxy", "--nocapture"]);
- for key in [
- "HTTP_PROXY",
- "http_proxy",
- "HTTPS_PROXY",
- "https_proxy",
- "ALL_PROXY",
- "all_proxy",
- "NO_PROXY",
- "no_proxy",
- "REQUEST_METHOD",
- ] {
- child.env_remove(key);
- }
- if mode != "direct" {
- child.env(
+ // The probe port is released before the child binds it again, so another
+ // process can take it in between. That is not a routing failure: retry the
+ // case on a fresh port.
+ let mut attempts_left = 5;
+ let output = loop {
+ let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
+ let address = listener.local_addr().unwrap();
+ drop(listener);
+ let mut child = Command::new(std::env::current_exe().unwrap());
+ child.args(["--exact", "relay_rpc_uses_environment_proxy", "--nocapture"]);
+ for key in [
+ "HTTP_PROXY",
+ "http_proxy",
+ "HTTPS_PROXY",
+ "https_proxy",
+ "ALL_PROXY",
+ "all_proxy",
+ "NO_PROXY",
+ "no_proxy",
+ "REQUEST_METHOD",
+ ] {
+ child.env_remove(key);
+ }
+ if mode != "direct" {
+ child.env(
+ if mode.starts_with("tls-") {
+ "HTTPS_PROXY"
+ } else {
+ "HTTP_PROXY"
+ },
+ format!("http://{address}"),
+ );
if mode.starts_with("tls-") {
- "HTTPS_PROXY"
- } else {
- "HTTP_PROXY"
- },
- format!("http://{address}"),
- );
- if mode.starts_with("tls-") {
- child.env("HTTP_PROXY", "http://invalid-http-proxy.invalid:9");
+ child.env("HTTP_PROXY", "http://invalid-http-proxy.invalid:9");
+ }
}
- }
- if mode == "config" {
- child.env("HTTP_PROXY", "http://user:[email protected]");
- }
- if mode == "bypass" {
- child.env("NO_PROXY", "127.0.0.0/8");
- }
- let output = child
- .env(MODE, mode)
- .env(ADDRESS, address.to_string())
- .output()
- .unwrap();
+ if mode == "config" {
+ child.env("HTTP_PROXY", "http://user:[email protected]");
+ }
+ if mode == "bypass" {
+ child.env("NO_PROXY", "127.0.0.0/8");
+ }
+ let output = child
+ .env(MODE, mode)
+ .env(ADDRESS, address.to_string())
+ .output()
+ .unwrap();
+ attempts_left -= 1;
+ if !output.status.success()
+ && attempts_left > 0
+ && String::from_utf8_lossy(&output.stderr).contains("AddrInUse")
+ {
+ continue;
+ }
+ break output;
+ };
if !output.status.success() {
failures.push(format!(
"{mode}: {}\n{}",
@@ -110,15 +124,17 @@ async fn head(socket: &mut TcpStream) -> String {
async fn run_case() {
let mode = std::env::var(MODE).unwrap();
+ // Bind the proxy address first: the mock server's own ephemeral bind must not
+ // be able to take the port the parent reserved for the proxy.
+ let proxy = TcpListener::bind(std::env::var(ADDRESS).unwrap())
+ .await
+ .unwrap();
let mock = MockServer::start(MockAuth {
token: Some(TOKEN.into()),
prefix: Some("/m/proxy-machine".into()),
..Default::default()
})
.await;
- let proxy = TcpListener::bind(std::env::var(ADDRESS).unwrap())
- .await
- .unwrap();
let upstream = mock.addr;
let attempts = Arc::new(AtomicUsize::new(0));
let accepted = Arc::new(tokio::sync::Notify::new());
diff --git a/libs/cua/crates/cua-telemetry/src/client.rs b/libs/cua/crates/cua-telemetry/src/client.rs
index 5e1187c665..21acd7b8bf 100644
--- a/libs/cua/crates/cua-telemetry/src/client.rs
+++ b/libs/cua/crates/cua-telemetry/src/client.rs
@@ -43,6 +43,8 @@ const FIRST_RUN_FILE: &str = "first_run_recorded";
const FUNNEL_DIR: &str = "funnel";
/// The UTC day (days since the epoch) `cua_app_active` was last sent.
const ACTIVE_DAY_FILE: &str = "active_day";
+/// The UTC day of the last `cua_agent_used_space`.
+const AGENT_USED_SPACE_FILE: &str = "agent_used_space_day";
/// The Spaces app's experiments that are on (`cua_volume+sharing`, `none`).
const EXPERIMENTS_FILE: &str = "experiments_on";
@@ -634,6 +636,46 @@ impl Telemetry {
r
}
+ /// Captures `cua_agent_used_space` when an AI agent's tool call `tool`
+ /// reached a Space and worked: at most once per install per UTC day.
+ /// Tools that do not act inside a Space are ignored.
+ pub fn capture_agent_used_space(&self, tool: &str) -> Captured {
+ let day = SystemTime::now()
+ .duration_since(UNIX_EPOCH)
+ .map(|d| d.as_secs() / 86_400)
+ .unwrap_or(0);
+ self.capture_agent_used_space_on(day, tool)
+ }
+
+ /// [`Self::capture_agent_used_space`] for a given UTC day (days since
+ /// the epoch).
+ pub fn capture_agent_used_space_on(&self, day: u64, tool: &str) -> Captured {
+ let Some(ev) = crate::events::agent_used_space(tool) else {
+ return Captured::Invalid(format!("{tool} does not act inside a Space"));
+ };
+ if !self.is_enabled() {
+ return Captured::Disabled;
+ }
+ // A sandbox's cua-spacesd is not an install, and a Cua program inside
+ // an agent run's Space is not the install the agent works for.
+ if self.product() == "spacesd" || self.in_agent_run() {
+ return Captured::Disabled;
+ }
+ let Some(dir) = self.state_dir() else {
+ return Captured::Disabled;
+ };
+ let marker = dir.join(FUNNEL_DIR).join(AGENT_USED_SPACE_FILE);
+ let today = format!("{day}\n");
+ if std::fs::read_to_string(&marker).is_ok_and(|s| s == today) {
+ return Captured::AlreadyRecorded;
+ }
+ let r = self.capture(ev);
+ if r == Captured::Queued {
+ let _ = fsutil::write_replace(&marker, today.as_bytes());
+ }
+ r
+ }
+
/// The Spaces app's experiments that are on now (fixed ids; anything
/// else is dropped): kept in the state directory, so the day's
/// `cua_app_active` carries them whichever product sends it. Nothing is
diff --git a/libs/cua/crates/cua-telemetry/src/events.rs b/libs/cua/crates/cua-telemetry/src/events.rs
index 9895711985..a57e164e9a 100644
--- a/libs/cua/crates/cua-telemetry/src/events.rs
+++ b/libs/cua/crates/cua-telemetry/src/events.rs
@@ -1159,3 +1159,86 @@ pub fn device_enroll(method: &str, outcome: Outcome) -> Option<Event> {
.s("outcome", outcome.as_str())
})
}
+
+/// `cua_installer_step`, as `install.sh` and `install.ps1` send it (they
+/// build the same payload themselves; this builder is the reference the
+/// tests hold them to). None for a step outside [`schema::INSTALLER_STEPS`].
+/// `stage` and `error_kind` are kept only on `installer_failed`.
+pub fn installer_step(
+ step: &str,
+ stage: &str,
+ error_kind: &str,
+ installer: &str,
+ items: &[&str],
+) -> Option<Event> {
+ let s = pick(schema::INSTALLER_STEPS, step, "");
+ if s.is_empty() {
+ return None;
+ }
+ let failed = s == "installer_failed";
+ let kind = if failed {
+ pick(schema::INSTALL_ERROR_KINDS, error_kind, "other")
+ } else {
+ "none"
+ };
+ let outcome = match (failed, kind) {
+ (false, _) => Outcome::Ok,
+ (true, "cancelled") => Outcome::Cancelled,
+ (true, _) => Outcome::Error,
+ };
+ let on: Vec<&str> = schema::INSTALL_ITEMS
+ .iter()
+ .copied()
+ .filter(|i| items.iter().any(|x| x.trim() == *i))
+ .collect();
+ Some(
+ Event::new(event::INSTALLER_STEP)
+ .s("step", s)
+ .s("outcome", outcome.as_str())
+ .s(
+ "install_stage",
+ if failed {
+ pick(schema::INSTALL_STAGES, stage, "other")
+ } else {
+ "none"
+ },
+ )
+ .s("error_kind", kind)
+ .s("installer", pick(schema::INSTALLERS, installer, "sh"))
+ .owned(
+ "items",
+ if on.is_empty() {
+ "none".into()
+ } else {
+ on.join("+")
+ },
+ ),
+ )
+}
+
+/// The kind of Spaces tool `tool` is, when calling it means an agent used
+/// a Space (it acts inside one); None for tools that only list, create,
+/// share or administer Spaces.
+pub fn space_tool_group(tool: &str) -> Option<&'static str> {
+ Some(match tool {
+ "space_bash" | "computer_shell" => "shell",
+ "space_write"
+ | "upload"
+ | "send_file"
+ | "download"
+ | "computer_file_read"
+ | "computer_file_write"
+ | "computer_file_list" => "files",
+ "list_tools" | "call_tool" => "services",
+ "agent_start" | "agent_message" => "agents",
+ "list_space_windows" | "teleport_app" => "other",
+ t if t.starts_with("computer_") && !t.starts_with("computer_access_") => "computer",
+ _ => return None,
+ })
+}
+
+/// `cua_agent_used_space`: an agent's tool call reached a Space and worked.
+/// None for a tool that does not act inside a Space.
+pub fn agent_used_space(tool: &str) -> Option<Event> {
+ space_tool_group(tool).map(|g| Event::new(event::AGENT_USED_SPACE).s("tool_group", g))
+}
diff --git a/libs/cua/crates/cua-telemetry/src/schema.rs b/libs/cua/crates/cua-telemetry/src/schema.rs
index 8aab946428..d790d04113 100644
--- a/libs/cua/crates/cua-telemetry/src/schema.rs
+++ b/libs/cua/crates/cua-telemetry/src/schema.rs
@@ -303,9 +303,9 @@ pub const FEATURES: &[&str] = &[
"upsell_plan_card_click",
"upsell_account_menu_click",
];
+/// Install steps live in [`INSTALLER_STEPS`] (`cua_installer_step`): the
+/// installers send them before any Cua program is on the machine.
pub const ONBOARDING_STEPS: &[&str] = &[
- "installer_started",
- "installer_completed",
"app_launched",
"onboarding_shown",
"onboarding_completed",
@@ -331,6 +331,38 @@ pub const ONBOARDING_STEPS: &[&str] = &[
"checklist_dismissed",
"checklist_reopened",
];
+/// What `install.sh` and `install.ps1` report about their own run
+/// (`cua_installer_step`): started, finished, or stopped by an error.
+pub const INSTALLER_STEPS: &[&str] = &[
+ "installer_started",
+ "installer_completed",
+ "installer_failed",
+];
+/// The part of the install that was running when it stopped (`none` on
+/// `installer_started` and `installer_completed`).
+pub const INSTALL_STAGES: &[&str] = &[
+ "none", "platform", "manifest", "cli", "runtime", "app", "driver", "sign_in", "host", "other",
+];
+/// Why an install stopped, as a fixed word the installer picks from its
+/// own error; never the message.
+pub const INSTALL_ERROR_KINDS: &[&str] = &[
+ "none",
+ "network",
+ "checksum",
+ "signature",
+ "unsupported_platform",
+ "permission_denied",
+ "missing_tool",
+ "cancelled",
+ "other",
+];
+/// Which installer sent the event.
+pub const INSTALLERS: &[&str] = &["sh", "ps1"];
+/// What the install was asked to put on the machine (joined with `+`).
+pub const INSTALL_ITEMS: &[&str] = &["cli", "spaces", "cua_driver", "host"];
+/// The kind of Spaces tool an agent's call used (`cua_agent_used_space`).
+pub const SPACE_TOOL_GROUPS: &[&str] =
+ &["shell", "files", "computer", "services", "agents", "other"];
/// Whether an `app_launched` opens the first run (`unknown` elsewhere).
pub const ONBOARDING_ELIGIBLE: &[&str] = &["yes", "no", "unknown"];
/// The Spaces app's first-run pages (`cua-spaces-app-core` `onboarding`).
@@ -566,7 +598,11 @@ pub const COMMON: &[Prop] = &[
Kind::ConstBool(true),
"No location lookup from the IP.",
),
- p("$lib", Kind::Enum(&["cua-telemetry"]), "Sender library."),
+ p(
+ "$lib",
+ Kind::Enum(&["cua-telemetry", "cua-installer"]),
+ "Sender library (cua-installer: install.sh or install.ps1).",
+ ),
p("$lib_version", Kind::Version, "Sender library version."),
];
@@ -666,6 +702,8 @@ pub mod event {
pub const APP_UPDATE: &str = "cua_app_update";
pub const DEVICE_ENROLL: &str = "cua_device_enroll";
pub const EXPERIMENT: &str = "cua_experiment";
+ pub const INSTALLER_STEP: &str = "cua_installer_step";
+ pub const AGENT_USED_SPACE: &str = "cua_agent_used_space";
}
/// Every event.
@@ -908,6 +946,56 @@ pub const EVENTS: &[EventSpec] = &[
],
purpose: "Install and onboarding funnel: where people drop off.",
},
+ EventSpec {
+ name: event::INSTALLER_STEP,
+ version: 1,
+ tier: Tier::Usage,
+ sample_rate: 1.0,
+ // The installers install the CLI, so they report as it.
+ products: &["cli"],
+ props: &[
+ p(
+ "step",
+ Kind::Enum(INSTALLER_STEPS),
+ "installer_started, installer_completed or installer_failed.",
+ ),
+ OUTCOME,
+ p(
+ "install_stage",
+ Kind::Enum(INSTALL_STAGES),
+ "installer_failed only: the part of the install that was running (platform, manifest, cli, runtime, app, driver, sign_in, host, other); none otherwise.",
+ ),
+ p(
+ "error_kind",
+ Kind::Enum(INSTALL_ERROR_KINDS),
+ "installer_failed only: why, as a fixed word (network, checksum, signature, unsupported_platform, permission_denied, missing_tool, cancelled, other); none otherwise. Never the message.",
+ ),
+ p(
+ "installer",
+ Kind::Enum(INSTALLERS),
+ "install.sh (sh) or install.ps1 (ps1).",
+ ),
+ p(
+ "items",
+ Kind::Set(INSTALL_ITEMS),
+ "What the install was asked for: cli, spaces, cua_driver, host, joined with +.",
+ ),
+ ],
+ purpose: "Install funnel: how many installs start, finish, and where and why they fail.",
+ },
+ EventSpec {
+ name: event::AGENT_USED_SPACE,
+ version: 1,
+ tier: Tier::Usage,
+ sample_rate: 1.0,
+ products: ALL,
+ props: &[p(
+ "tool_group",
+ Kind::Enum(SPACE_TOOL_GROUPS),
+ "The kind of tool the call used: shell, files, computer, services, agents or other.",
+ )],
+ purpose: "First success and agent retention: an AI agent's tool call reached a Space and worked. At most one per install per UTC day.",
+ },
EventSpec {
name: event::AGENT_RUN_STARTED,
version: 1,
diff --git a/libs/cua/crates/cua-telemetry/tests/privacy.rs b/libs/cua/crates/cua-telemetry/tests/privacy.rs
index adf211f62c..9cacc49f07 100644
--- a/libs/cua/crates/cua-telemetry/tests/privacy.rs
+++ b/libs/cua/crates/cua-telemetry/tests/privacy.rs
@@ -219,6 +219,15 @@ fn all_builders(s: &str) -> Vec<Event> {
v.push(events::app_update("downloaded", s, s).unwrap());
v.push(events::device_enroll("rekey", Outcome::Ok).unwrap());
v.push(events::experiment("experiment_off", "your_cloud").unwrap());
+ v.push(events::installer_step("installer_started", s, s, s, &[s]).unwrap());
+ v.push(events::installer_step("installer_failed", s, s, "ps1", &[s, "cli"]).unwrap());
+ v.push(
+ events::installer_step("installer_failed", "cli", "cancelled", "sh", &["host"]).unwrap(),
+ );
+ v.push(events::agent_used_space("space_bash").unwrap());
+ v.push(events::agent_used_space("computer_click").unwrap());
+ assert!(events::installer_step(s, "cli", "network", "sh", &[]).is_none());
+ assert!(events::agent_used_space(s).is_none());
v
}
@@ -904,6 +913,7 @@ fn docs_page_names_every_event_and_property() {
let page = concat!(
include_str!("../../../../../docs/content/docs/cua-sdk/concepts/telemetry.mdx"),
include_str!("../../../../../docs/content/docs/cua-sdk/concepts/telemetry-events.mdx"),
+ include_str!("../../../../../docs/content/docs/cua-sdk/concepts/telemetry-counting.mdx"),
);
for spec in schema::EVENTS {
assert!(
@@ -1450,3 +1460,129 @@ fn a_failed_host_space_never_reports_no_error_kind() {
let ok = events::host_space_provided("vm", "macos", Outcome::Ok, None);
assert_eq!(ok.props["error_kind"], "none");
}
+
+/// `cua_installer_step` is sent by install.sh and install.ps1, which build
+/// the payload themselves. The fixture they are tested against
+/// (`scripts/install/tests/test_install_telemetry.py`) must pass the same
+/// schema check as every other event.
+#[test]
+fn installer_payloads_validate_against_the_declared_schema() {
+ let text = include_str!("../../../../../scripts/install/tests/fixtures/installer-step.json");
+ let doc: Value = serde_json::from_str(text).unwrap();
+ let events = doc["events"].as_array().unwrap();
+ assert_eq!(events.len(), 3);
+ for e in events {
+ let name = e["event"].as_str().unwrap();
+ assert_eq!(name, schema::event::INSTALLER_STEP);
+ let props = e["properties"].as_object().unwrap();
+ schema::validate(name, props).unwrap_or_else(|err| panic!("{e}: {err}"));
+ let id = e["distinct_id"].as_str().unwrap();
+ assert!(id.len() == 32 && id.bytes().all(|b| b.is_ascii_hexdigit()));
+ }
+ // The builder agrees with the fixture on the event's own properties.
+ let built =
+ events::installer_step("installer_failed", "cli", "checksum", "sh", &["cli"]).unwrap();
+ for (k, v) in &built.props {
+ assert_eq!(&events[1]["properties"][k], v, "{k}");
+ }
+ let built = events::installer_step(
+ "installer_started",
+ "cli",
+ "network",
+ "sh",
+ &["spaces", "cli"],
+ )
+ .unwrap();
+ assert_eq!(built.props["install_stage"], "none");
+ assert_eq!(built.props["error_kind"], "none");
+ assert_eq!(built.props["items"], "cli+spaces");
+}
+
+/// An agent's tool call that reached a Space: one `cua_agent_used_space`
+/// per install per UTC day, only for tools that act inside a Space, never
+/// while telemetry is off and never from inside an agent run's Space.
+#[test]
+fn agent_used_space_is_sent_once_per_day_for_space_tools_only() {
+ let h = tempfile::tempdir().unwrap();
+ let (t, sink) = ready(h.path(), &[]);
+ assert!(matches!(
+ t.capture_agent_used_space_on(20_000, "list_spaces"),
+ Captured::Invalid(_)
+ ));
+ assert!(matches!(
+ t.capture_agent_used_space_on(20_000, "computer_access_grant"),
+ Captured::Invalid(_)
+ ));
+ assert_eq!(
+ t.capture_agent_used_space_on(20_000, "space_bash"),
+ Captured::Queued
+ );
+ assert_eq!(
+ t.capture_agent_used_space_on(20_000, "call_tool"),
+ Captured::AlreadyRecorded
+ );
+ let (t2, _) = ready(h.path(), &[]);
+ assert_eq!(
+ t2.capture_agent_used_space_on(20_000, "upload"),
+ Captured::AlreadyRecorded
+ );
+ assert_eq!(
+ t.capture_agent_used_space_on(20_001, "call_tool"),
+ Captured::Queued
+ );
+ t.flush(Duration::from_secs(1));
+ let used: Vec<Value> = sink
+ .events()
+ .into_iter()
+ .filter(|e| e["event"] == "cua_agent_used_space")
+ .collect();
+ assert_eq!(used.len(), 2);
+ assert_eq!(used[0]["properties"]["tool_group"], "shell");
+ assert_eq!(used[1]["properties"]["tool_group"], "services");
+
+ for env in [
+ &[("DO_NOT_TRACK", "1")][..],
+ &[("CUA_TELEMETRY", "0")][..],
+ &[("CI", "true")][..],
+ &[("CUA_AGENT_RUN_ID", "run-1")][..],
+ ] {
+ let h = tempfile::tempdir().unwrap();
+ let (t, sink) = ready(h.path(), env);
+ assert_eq!(
+ t.capture_agent_used_space_on(20_000, "space_bash"),
+ Captured::Disabled,
+ "{env:?}"
+ );
+ t.flush(Duration::from_secs(1));
+ assert!(sink.events().is_empty(), "{env:?}");
+ assert!(
+ !h.path()
+ .join("telemetry/funnel/agent_used_space_day")
+ .exists()
+ );
+ }
+}
+
+#[test]
+fn space_tool_groups_cover_the_tools_that_act_inside_a_space() {
+ use events::space_tool_group as g;
+ assert_eq!(g("space_bash"), Some("shell"));
+ assert_eq!(g("computer_shell"), Some("shell"));
+ assert_eq!(g("send_file"), Some("files"));
+ assert_eq!(g("computer_file_write"), Some("files"));
+ assert_eq!(g("call_tool"), Some("services"));
+ assert_eq!(g("agent_start"), Some("agents"));
+ assert_eq!(g("computer_screenshot"), Some("computer"));
+ assert_eq!(g("teleport_app"), Some("other"));
+ for admin in [
+ "list_spaces",
+ "create_space",
+ "delete_space",
+ "share_space",
+ "computer_access_list",
+ "volume_ls",
+ "",
+ ] {
+ assert_eq!(g(admin), None, "{admin}");
+ }
+}
diff --git a/scripts/install/README.md b/scripts/install/README.md
index 465a24dd0c..59b06c29fd 100644
--- a/scripts/install/README.md
+++ b/scripts/install/README.md
@@ -54,6 +54,10 @@ For cua-driver, `--prefix DIR` becomes `--bin-dir DIR/bin`, `--require-signature
Pass install.sh options through the pipe with `sh -s --`, for example `curl -fsSL https://cua.ai/install.sh | sh -s -- --cli-only --yes`. `--only` and `--select` cannot be combined, nor can `--cli-only` with other items. Set `CUA_INSTALL_NONINTERACTIVE=1` to never prompt.
+## Telemetry
+
+Both installers send three anonymous events about their own run (`cua_installer_step`: `installer_started`, then `installer_completed` or `installer_failed` with the stage and a fixed reason word), and print a two-line notice before the first one. They follow the switches every Cua program uses: `DO_NOT_TRACK=1`, `CUA_TELEMETRY=0`, `[telemetry] enabled = "off"` in `$CUA_HOME/config.toml`, and off in CI unless `CUA_TELEMETRY=1`. `--dry-run` sends nothing. They create or reuse `$CUA_HOME/telemetry/install_id`, the anonymous id the CLI and app send with, and never send a name, path, URL or error message. The tests set `CUA_TELEMETRY_FORBID_NETWORK=1`, which allows only a loopback `CUA_TELEMETRY_ENDPOINT`. See [Telemetry and privacy](https://cua.ai/docs/cua-sdk/concepts/telemetry).
+
## What it prints
One plain line per step ("Installing the command-line tools to ...", "Installing the Cua Spaces app to ..."), then what was installed and what to do next: open Cua Spaces, or sign in and run `cua agents setup` to connect your AI agents. `--verbose` adds `cua-install:` lines with the platform, the manifest, each download and the signature checks.
diff --git a/scripts/install/install.ps1 b/scripts/install/install.ps1
index 7a643de202..a548f54919 100644
--- a/scripts/install/install.ps1
+++ b/scripts/install/install.ps1
@@ -138,6 +138,164 @@ function Write-Step([string]$Message) { Write-Host $Message }
function Write-Info([string]$Message) { Write-Verbose "cua-install: $Message" }
function Stop-Install([string]$Message) { throw "cua-install: error: $Message" }
+# ---------------------------------------------------------------- telemetry
+# Three anonymous events about this install (cua_installer_step):
+# installer_started, then installer_completed or installer_failed with the
+# stage that was running and a fixed reason word. The same switches as every
+# Cua program (DO_NOT_TRACK, CUA_TELEMETRY, `[telemetry] enabled = "off"` in
+# $CUA_HOME\config.toml, off in CI) and the same anonymous install id. Never a
+# name, a path, a URL or an error message; nothing with -DryRun. Best effort:
+# a send never fails the install and waits at most 3 seconds.
+$script:TelKeyProd = 'phc_eSkLnbLxsnYFaXksif1ksbrNzYlJShr35miFLDppF14'
+$script:TelKeyStaging = 'phc_xDNLpzypNLVGmzQeBZEgZDZK7Ea6yYAmAnGZ6hMtHNKG'
+$script:TelCiVars = @('CI', 'CONTINUOUS_INTEGRATION', 'GITHUB_ACTIONS', 'GITLAB_CI', 'JENKINS_URL', 'CIRCLECI', 'BUILDKITE',
+ 'TF_BUILD', 'TEAMCITY_VERSION', 'TRAVIS', 'APPVEYOR', 'BITBUCKET_BUILD_NUMBER', 'CODEBUILD_BUILD_ID', 'DRONE', 'HUDSON_URL')
+$script:Tel = $null
+$script:TelStage = 'other'
+$script:TelVersion = ''
+
+# 'on', 'off' or $null for a switch word.
+function ConvertTo-TelSwitch([string]$Value) {
+ if ($null -eq $Value) { return $null }
+ switch -Regex ($Value.Trim().ToLowerInvariant()) {
+ '^(0|false|no|off|disable|disabled)$' { return 'off' }
+ '^(1|true|yes|on|enable|enabled)$' { return 'on' }
+ }
+ return $null
+}
+
+function Get-TelConfigSwitch([string]$CuaHome) {
+ $f = Join-Path $CuaHome 'config.toml'
+ if (-not (Test-Path -LiteralPath $f)) { return $null }
+ $section = ''
+ foreach ($line in (Get-Content -LiteralPath $f -ErrorAction SilentlyContinue)) {
+ if ($line -match '^\s*\[\s*([^\]]+?)\s*\]') { $section = $Matches[1]; continue }
+ if ($section -eq 'telemetry' -and $line -match '^\s*enabled\s*=\s*"?([^"#\s]*)"?') { return (ConvertTo-TelSwitch $Matches[1]) }
+ }
+ return $null
+}
+
+# Whether install events may be sent here (the rules of cua-telemetry's config.rs).
+function Test-TelemetryEnabled([string]$CuaHome) {
+ $dnt = "$env:DO_NOT_TRACK".Trim().ToLowerInvariant()
+ if ($dnt -and $dnt -ne '0' -and $dnt -ne 'false') { return $false }
+ $sw = ConvertTo-TelSwitch $env:CUA_TELEMETRY
+ if ($sw) { return ($sw -eq 'on') }
+ if ((ConvertTo-TelSwitch $env:CUA_TELEMETRY_ENABLED) -eq 'off') { return $false }
+ if ((ConvertTo-TelSwitch $env:CUA_TELEMETRY_DISABLED) -eq 'on') { return $false }
+ $sw = Get-TelConfigSwitch $CuaHome
+ if ($sw) { return ($sw -eq 'on') }
+ return -not (Test-TelCi)
+}
+
+function Test-TelCi {
+ foreach ($v in $script:TelCiVars) {
+ $x = "$([Environment]::GetEnvironmentVariable($v))".Trim().ToLowerInvariant()
+ if ($x -and $x -ne 'false' -and $x -ne '0') { return $true }
+ }
+ return $false
+}
+
+function New-TelHex {
+ $b = New-Object byte[] 16
+ [Security.Cryptography.RandomNumberGenerator]::Create().GetBytes($b)
+ return (($b | ForEach-Object { $_.ToString('x2') }) -join '')
+}
+
+# Decides, shows the notice, and loads or creates the anonymous install id.
+function Initialize-CuaTelemetry {
+ $script:Tel = $null
+ if ($DryRun) { return }
+ $key = if ($env:CUA_POSTHOG_KEY) { $env:CUA_POSTHOG_KEY } elseif ("$env:CUA_ENV".Trim().ToLowerInvariant() -eq 'staging') { $script:TelKeyStaging } else { $script:TelKeyProd }
+ if (-not $key) { return }
+ $cuaHome = if ($env:CUA_HOME) { $env:CUA_HOME } else { Join-Path $HOME '.cua' }
+ if (-not (Test-TelemetryEnabled $cuaHome)) { return }
+ $endpoint = if ($env:CUA_TELEMETRY_ENDPOINT) { $env:CUA_TELEMETRY_ENDPOINT } else { 'https://eu.i.posthog.com/batch/' }
+ if ((ConvertTo-TelSwitch $env:CUA_TELEMETRY_FORBID_NETWORK) -eq 'on' -and $endpoint -notmatch '^http://(127\.0\.0\.1|localhost|\[::1\])[:/]') { return }
+ $dir = Join-Path $cuaHome 'telemetry'
+ $idFile = Join-Path $dir 'install_id'
+ $id = $null
+ try { $id = (Get-Content -Raw -LiteralPath $idFile -ErrorAction Stop).Trim() } catch { }
+ if ($id -notmatch '^[0-9a-f]{32}$') {
+ $id = New-TelHex
+ try {
+ New-Item -ItemType Directory -Force -Path $dir | Out-Null
+ Set-Content -LiteralPath $idFile -Value $id -NoNewline
+ } catch { Write-Verbose "could not store the telemetry id: $_" }
+ }
+ $script:Tel = @{ Key = $key; Endpoint = $endpoint; Id = $id; Session = (New-TelHex); Ci = (Test-TelCi); Ended = $false }
+ Write-Host 'Cua sends anonymous data about this install (whether it worked, and where it stopped if not). Never names, file paths or anything you type.'
+ Write-Host 'To turn it off, set DO_NOT_TRACK=1. Details: https://cua.ai/docs/cua-sdk/concepts/telemetry'
+}
+
+# The fixed reason word for an install error message.
+function Get-CuaInstallErrorKind([string]$Message) {
+ switch -Wildcard ($Message) {
+ '*checksum mismatch*' { return 'checksum' }
+ '*signature*' { return 'signature' }
+ '*release identity*' { return 'signature' }
+ '*unsupported CPU*' { return 'unsupported_platform' }
+ '*cannot create*' { return 'permission_denied' }
+ '*ermission*' { return 'permission_denied' }
+ '*Access*denied*' { return 'permission_denied' }
+ '*cancelled*' { return 'cancelled' }
+ '*download failed*' { return 'network' }
+ '*unsupported release manifest*' { return 'network' }
+ '*Unable to connect*' { return 'network' }
+ '*could not be resolved*' { return 'network' }
+ '*timed out*' { return 'network' }
+ '*No such host*' { return 'network' }
+ '*404*' { return 'network' }
+ }
+ return 'other'
+}
+
+# The cua_installer_step payload (one batch body).
+function New-CuaInstallerStepBody($Sel, [string]$Step, [string]$Stage = 'none', [string]$Kind = 'none') {
+ $failed = $Step -eq 'installer_failed'
+ if (-not $failed) { $Stage = 'none'; $Kind = 'none' }
+ $outcome = if (-not $failed) { 'ok' } elseif ($Kind -eq 'cancelled') { 'cancelled' } else { 'error' }
+ $items = @()
+ if ($Sel) {
+ if ($Sel.cli) { $items += 'cli' }
+ if ($Sel.spaces) { $items += 'spaces' }
+ if ($Sel['cua-driver']) { $items += 'cua_driver' }
+ if ($Sel.host) { $items += 'host' }
+ }
+ $pv = if ($script:TelVersion) { $script:TelVersion } elseif ($Version) { $Version.TrimStart('v') } else { '' }
+ if ($pv -notmatch '^\d+\.\d+\.\d+(-[0-9A-Za-z.-]+)?$') { $pv = '0.0.0' }
+ $isWin = (-not (Test-Path variable:IsWindows)) -or $IsWindows
+ $os = if ($isWin) { 'windows' } elseif ($IsMacOS) { 'macos' } elseif ($IsLinux) { 'linux' } else { 'other' }
+ $major = [int][Environment]::OSVersion.Version.Major
+ if ($major -gt 999 -or $major -lt 0) { $major = 0 }
+ $archRaw = if ($env:PROCESSOR_ARCHITEW6432) { $env:PROCESSOR_ARCHITEW6432 } elseif ($env:PROCESSOR_ARCHITECTURE) { $env:PROCESSOR_ARCHITECTURE } else { [System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture.ToString() }
+ $arch = switch -Regex ($archRaw) { '^(AMD64|x64|X64|x86_64)$' { 'x86_64' } '^(ARM64|Arm64|aarch64)$' { 'arm64' } default { 'other' } }
+ $u = New-TelHex
+ $uuid = '{0}-{1}-4{2}-8{3}-{4}' -f $u.Substring(0, 8), $u.Substring(8, 4), $u.Substring(13, 3), $u.Substring(17, 3), $u.Substring(20, 12)
+ $props = [ordered]@{
+ telemetry_schema_version = 1; event_version = 1; product = 'cli'; product_version = $pv
+ os_family = $os; os_major = $major; arch = $arch; is_ci = [bool]$script:Tel.Ci
+ is_synthetic = ((ConvertTo-TelSwitch $env:CUA_TELEMETRY_SYNTHETIC) -eq 'on'); process_session_id = $script:Tel.Session
+ sample_rate = 1; '$process_person_profile' = $false; '$geoip_disable' = $true
+ '$lib' = 'cua-installer'; '$lib_version' = '1.0.0'
+ step = $Step; outcome = $outcome; install_stage = $Stage; error_kind = $Kind; installer = 'ps1'
+ items = $(if ($items.Count) { $items -join '+' } else { 'none' })
+ }
+ $event = [ordered]@{ event = 'cua_installer_step'; distinct_id = $script:Tel.Id; uuid = $uuid
+ timestamp = [DateTime]::UtcNow.ToString('yyyy-MM-ddTHH:mm:ssZ'); properties = $props }
+ return (ConvertTo-Json -Depth 6 -Compress ([ordered]@{ api_key = $script:Tel.Key; batch = @($event) }))
+}
+
+function Send-CuaInstallerStep($Sel, [string]$Step, [string]$Stage = 'none', [string]$Kind = 'none') {
+ if (-not $script:Tel -or $script:Tel.Ended) { return }
+ if ($Step -ne 'installer_started') { $script:Tel.Ended = $true }
+ try {
+ $body = New-CuaInstallerStepBody $Sel $Step $Stage $Kind
+ if ((ConvertTo-TelSwitch $env:CUA_TELEMETRY_DEBUG) -eq 'on') { Write-Host "[cua telemetry] $body" }
+ Invoke-RestMethod -Method Post -Uri $script:Tel.Endpoint -ContentType 'application/json' -Body $body -TimeoutSec 3 | Out-Null
+ } catch { Write-Verbose "install telemetry not sent: $_" }
+}
+
function Get-CuaPlatform {
param([string]$Arch = $env:CUA_INSTALL_ARCH)
if (-not $Arch) {
@@ -603,13 +761,33 @@ function Invoke-CuaInstall {
$ProgressPreference = 'SilentlyContinue'
Test-PairArgs
$sel = Get-CuaSelection
+ Initialize-CuaTelemetry
+ $script:TelStage = 'platform'
+ try {
+ Invoke-CuaInstallSteps $sel
+ $script:TelStage = 'other'
+ Send-CuaInstallerStep $sel 'installer_completed'
+ } catch {
+ Send-CuaInstallerStep $sel 'installer_failed' $script:TelStage (Get-CuaInstallErrorKind "$_")
+ throw
+ } finally {
+ # Stopped without an error (Ctrl+C): cancelled.
+ Send-CuaInstallerStep $sel 'installer_failed' $script:TelStage 'cancelled'
+ }
+}
+
+function Invoke-CuaInstallSteps($sel) {
if ($sel.spaces -and -not $script:SpacesSupported) {
Write-Step 'Cua Spaces is not available on this platform; skipping the app'
$sel.spaces = $false
}
if ($Version -and $Version.TrimStart('v') -notmatch '^\d+\.\d+\.\d+') { Stop-Install "-Version must look like 1.2.3, not '$Version'" }
$platform = Get-CuaPlatform
- if (Test-WantsChecklist) { $sel = Read-CuaSelection $sel }
+ Send-CuaInstallerStep $sel 'installer_started'
+ if (Test-WantsChecklist) {
+ $picked = Read-CuaSelection $sel
+ foreach ($k in @($picked.Keys)) { $sel[$k] = $picked[$k] }
+ }
if ($DryRun) { Write-Step 'Dry run: nothing will be changed' }
$tmp = Join-Path ([IO.Path]::GetTempPath()) ("cua-install-" + [Guid]::NewGuid().ToString('N'))
New-Item -ItemType Directory -Force -Path $tmp | Out-Null
@@ -624,15 +802,19 @@ function Invoke-CuaInstall {
$manifestUrl = $null
# Nothing to download (-Pair on a computer that has Cua): no manifest to read.
if ($sel.cli -or $sel.spaces -or $sel['cua-driver']) {
+ $script:TelStage = 'manifest'
$manifestUrl = Get-ManifestUrl
$manifestFile = Join-Path $tmp 'release-artifacts.json'
Get-Remote $manifestUrl $manifestFile
$manifest = Get-Content -Raw -LiteralPath $manifestFile | ConvertFrom-Json
if ($manifest.schema -ne 1) { Stop-Install "unsupported release manifest at $manifestUrl" }
Write-Info "platform $platform, manifest $manifestUrl"
+ $script:TelVersion = "$(@($manifest.artifacts | Where-Object { $_.component -eq 'cli' } | Select-Object -First 1).version)"
}
+ $script:TelStage = 'cli'
if ($sel.cli) { $cua = Install-Cli $manifest $manifestUrl $platform $tmp }
if ($sel.spaces) {
+ $script:TelStage = 'app'
if ($Mode) {
$cuaHome = if ($env:CUA_HOME) { $env:CUA_HOME } else { Join-Path $HOME '.cua' }
$modeFile = Join-Path $cuaHome 'spaces-install-mode'
@@ -656,13 +838,16 @@ function Invoke-CuaInstall {
} catch { Write-Verbose "could not record the install channel: $_" }
}
$env:CUA_INSTALL_CHANNEL = 'install_script'
+ $script:TelStage = 'driver'
if ($sel['cua-driver']) { Install-CuaDriver $cua $tmp $manifestUrl }
+ $script:TelStage = 'sign_in'
if ($cua -and -not $Pair -and -not $CliOnly -and -not $NoOnboarding -and -not $DryRun -and (Test-Interactive)) {
Write-Host ''
Write-Step 'Signing in to Cua'
& $cua auth login
if ($LASTEXITCODE -ne 0) { Write-Warning 'Sign-in did not finish. Run "cua auth login" later (add --remote if this computer has no web browser).' } else { $script:SignedIn = $true }
}
+ $script:TelStage = 'host'
if ($sel.host) { Invoke-HostSetup $cua }
Write-CuaSummary $sel
if ($script:PathHint) {
diff --git a/scripts/install/install.sh b/scripts/install/install.sh
index 4bda6cdada..987240b0c9 100755
--- a/scripts/install/install.sh
+++ b/scripts/install/install.sh
@@ -55,6 +55,14 @@
# CUA_INSTALL_OS, CUA_INSTALL_ARCH override platform detection
# CUA_INSTALL_MINISIGN_PUBKEY minisign public key for artifacts
# CUA_HOME cua state dir (default ~/.cua)
+# DO_NOT_TRACK=1 or CUA_TELEMETRY=0 send no install events (see below)
+#
+# Telemetry: the installer sends three anonymous events about its own run
+# (installer started, finished, or failed with the step and a fixed reason
+# word), under the same switches as every Cua program: DO_NOT_TRACK,
+# CUA_TELEMETRY, `[telemetry] enabled = "off"` in $CUA_HOME/config.toml, and
+# off in CI. Never names, paths or messages. Nothing on --dry-run.
+# https://cua.ai/docs/cua-sdk/concepts/telemetry
set -eu
REPO="${CUA_INSTALL_REPO:-trycua/cua}"
@@ -114,6 +122,7 @@ info() { [ "$verbose" = 0 ] || printf 'cua-install: %s\n' "$*"; }
warn() { printf 'cua-install: warning: %s\n' "$*" >&2; }
die() {
printf 'cua-install: error: %s\n' "$*" >&2
+ tel_kind="$(tel_error_kind "$*")"
exit 1
}
@@ -122,6 +131,11 @@ usage() {
}
cleanup() {
+ rc=$?
+ if [ "$rc" != 0 ] && [ "$tel_on" = 1 ] && [ "$tel_ended" = 0 ]; then
+ [ "$rc" != 130 ] || tel_kind=cancelled
+ tel_send installer_failed "$tel_stage" "${tel_kind:-other}"
+ fi
if [ -n "$tty_saved" ]; then
stty "$tty_saved" </dev/tty 2>/dev/null || true
tty_saved=""
@@ -136,6 +150,202 @@ cleanup() {
trap cleanup EXIT
trap 'exit 130' INT TERM
+# ---------------------------------------------------------------- telemetry
+
+# Three anonymous events about this install (cua_installer_step):
+# installer_started, then installer_completed or installer_failed with the
+# stage that was running and a fixed reason word. The same switches as
+# every Cua program (libs/cua/crates/cua-telemetry/src/config.rs); the same
+# anonymous install id, so the app's later events continue this install.
+# Never a name, a path, a URL or an error message. Best effort: a send never
+# fails or slows the install by more than a few seconds.
+TEL_ENDPOINT_DEFAULT="https://eu.i.posthog.com/batch/"
+TEL_KEY_PROD="phc_eSkLnbLxsnYFaXksif1ksbrNzYlJShr35miFLDppF14"
+TEL_KEY_STAGING="phc_xDNLpzypNLVGmzQeBZEgZDZK7Ea6yYAmAnGZ6hMtHNKG"
+tel_on=0
+tel_ended=0
+tel_stage=other
+tel_kind=""
+tel_version=""
+tel_id=""
+tel_session=""
+tel_ci=false
+
+tel_lower() { printf '%s' "$1" | tr '[:upper:]' '[:lower:]' | tr -d '[:space:]'; }
+
+# tel_bool VALUE: prints on, off, or nothing (unset or not a switch word).
+tel_bool() {
+ case "$(tel_lower "$1")" in
+ 0 | false | no | off | disable | disabled) echo off ;;
+ 1 | true | yes | on | enable | enabled) echo on ;;
+ esac
+}
+
+# The [telemetry] enabled value in $CUA_HOME/config.toml (on, off, or nothing).
+tel_config() {
+ f="${CUA_HOME:-$HOME/.cua}/config.toml"
+ [ -r "$f" ] || return 0
+ v="$(awk '
+ /^[[:space:]]*\[/ { sect = $0; gsub(/[[:space:]\[\]]/, "", sect); next }
+ sect == "telemetry" && /^[[:space:]]*enabled[[:space:]]*=/ {
+ sub(/^[^=]*=[[:space:]]*/, ""); sub(/[[:space:]]*(#.*)?$/, ""); gsub(/"/, ""); print; exit
+ }' "$f" 2>/dev/null)"
+ tel_bool "$v"
+}
+
+tel_is_ci() {
+ for v in CI CONTINUOUS_INTEGRATION GITHUB_ACTIONS GITLAB_CI JENKINS_URL CIRCLECI BUILDKITE \
+ TF_BUILD TEAMCITY_VERSION TRAVIS APPVEYOR BITBUCKET_BUILD_NUMBER CODEBUILD_BUILD_ID DRONE HUDSON_URL; do
+ eval "x=\${$v:-}"
+ x="$(tel_lower "$x")"
+ case "$x" in "" | false | 0) ;; *) return 0 ;; esac
+ done
+ return 1
+}
+
+tel_hex() { # BYTES: random lowercase hex
+ od -An -N"$1" -tx1 /dev/urandom 2>/dev/null | tr -d ' \n'
+}
+
+# telemetry_init: decide, show the notice, load or create the install id.
+telemetry_init() {
+ tel_on=0
+ [ "$dry_run" = 0 ] || return 0
+ if [ -n "${CUA_POSTHOG_KEY:-}" ]; then
+ tel_key="$CUA_POSTHOG_KEY"
+ elif [ "$(tel_lower "${CUA_ENV:-}")" = staging ]; then
+ tel_key="$TEL_KEY_STAGING"
+ else
+ tel_key="$TEL_KEY_PROD"
+ fi
+ [ -n "$tel_key" ] || return 0
+ case "$(tel_lower "${DO_NOT_TRACK:-}")" in "" | 0 | false) ;; *) return 0 ;; esac
+ tel_is_ci && tel_ci=true
+ sw="$(tel_bool "${CUA_TELEMETRY:-}")"
+ if [ -z "$sw" ]; then
+ [ "$(tel_bool "${CUA_TELEMETRY_ENABLED:-}")" != off ] || return 0
+ [ "$(tel_bool "${CUA_TELEMETRY_DISABLED:-}")" != on ] || return 0
+ sw="$(tel_config)"
+ [ -n "$sw" ] || { [ "$tel_ci" = true ] && sw=off; }
+ fi
+ [ "$sw" != off ] || return 0
+ tel_endpoint="${CUA_TELEMETRY_ENDPOINT:-$TEL_ENDPOINT_DEFAULT}"
+ if [ "$(tel_bool "${CUA_TELEMETRY_FORBID_NETWORK:-}")" = on ]; then
+ case "$tel_endpoint" in
+ http://127.0.0.1[:/]* | http://localhost[:/]* | http://\[::1\][:/]*) ;;
+ *) return 0 ;;
+ esac
+ fi
+ has curl || has wget || return 0
+ dir="${CUA_HOME:-$HOME/.cua}/telemetry"
+ tel_id="$(cat "$dir/install_id" 2>/dev/null | tr -d '[:space:]')"
+ case "$tel_id" in
+ *[!0-9a-f]* | "") tel_id="" ;;
+ esac
+ [ "${#tel_id}" = 32 ] || tel_id=""
+ if [ -z "$tel_id" ]; then
+ tel_id="$(tel_hex 16)"
+ [ "${#tel_id}" = 32 ] || return 0
+ (umask 077 && mkdir -p "$dir" && printf '%s' "$tel_id" >"$dir/install_id") 2>/dev/null || true
+ fi
+ tel_session="$(tel_hex 16)"
+ [ "${#tel_session}" = 32 ] || return 0
+ tel_on=1
+ say "Cua sends anonymous data about this install (whether it worked, and where it stopped if not). Never names, file paths or anything you type."
+ say "To turn it off, set DO_NOT_TRACK=1. Details: https://cua.ai/docs/cua-sdk/concepts/telemetry"
+}
+
+# tel_error_kind MESSAGE: the fixed reason word for a die message.
+tel_error_kind() {
+ case "$1" in
+ *"checksum mismatch"*) echo checksum ;;
+ *signature* | *"release identity"*) echo signature ;;
+ *"unsupported OS"* | *"unsupported CPU"* | *"on Windows, run"*) echo unsupported_platform ;;
+ *"cannot create"* | *ermission*) echo permission_denied ;;
+ *"need curl"* | *"need sha256sum"* | *"not found; cannot"* | *"needs bash"* | *"needs QEMU"*) echo missing_tool ;;
+ *cancelled*) echo cancelled ;;
+ *"download failed"* | *"could not read"* | *"unsupported release manifest"*) echo network ;;
+ *) echo other ;;
+ esac
+}
+
+tel_os() {
+ case "$(uname -s 2>/dev/null)" in
+ Darwin) echo macos ;;
+ Linux) echo linux ;;
+ MINGW* | MSYS* | CYGWIN*) echo windows ;;
+ *) echo other ;;
+ esac
+}
+
+tel_os_major() {
+ case "$(uname -s 2>/dev/null)" in
+ Darwin) v="$(sw_vers -productVersion 2>/dev/null)" ;;
+ Linux) v="$(sed -n 's/^VERSION_ID=//p' /etc/os-release 2>/dev/null | tr -d '"')" ;;
+ *) v="" ;;
+ esac
+ v="${v%%[!0-9]*}"
+ v="$(printf '%s' "$v" | sed 's/^0*//')"
+ case "$v" in "" | ????*) echo 0 ;; *) echo "$v" ;; esac
+}
+
+tel_arch() {
+ case "$(uname -m 2>/dev/null)" in
+ arm64 | aarch64) echo arm64 ;;
+ x86_64 | amd64) echo x86_64 ;;
+ *) echo other ;;
+ esac
+}
+
+tel_items() {
+ items=""
+ [ "$want_cli" = 0 ] || items="cli"
+ [ "$sel_spaces" = 0 ] || items="${items:+$items+}spaces"
+ [ "$sel_driver" = 0 ] || items="${items:+$items+}cua_driver"
+ [ "$sel_host" = 0 ] || items="${items:+$items+}host"
+ echo "${items:-none}"
+}
+
+# tel_send STEP [STAGE KIND]: one cua_installer_step, at most 3 seconds.
+tel_send() {
+ [ "$tel_on" = 1 ] || return 0
+ step_name="$1"
+ stage=none
+ kind=none
+ outcome=ok
+ case "$step_name" in
+ installer_completed) tel_ended=1 ;;
+ installer_failed)
+ tel_ended=1
+ stage="${2:-other}"
+ kind="${3:-other}"
+ outcome=error
+ [ "$kind" != cancelled ] || outcome=cancelled
+ ;;
+ esac
+ pv="${tel_version:-${version:-}}"
+ case "$pv" in
+ [0-9]*.[0-9]*.[0-9]*) ;;
+ *) pv=0.0.0 ;;
+ esac
+ pv="${pv%%[!0-9A-Za-z.-]*}"
+ synthetic=false
+ [ "$(tel_bool "${CUA_TELEMETRY_SYNTHETIC:-}")" != on ] || synthetic=true
+ u="$(tel_hex 16)"
+ uuid="$(printf '%s' "$u" | sed 's/^\(.\{8\}\)\(.\{4\}\).\(.\{3\}\).\(.\{3\}\)\(.\{12\}\)$/\1-\2-4\3-8\4-\5/')"
+ # The $-prefixed names are PostHog property names, not shell expansions.
+ # shellcheck disable=SC2016
+ body="$(printf '{"api_key":"%s","batch":[{"event":"cua_installer_step","distinct_id":"%s","uuid":"%s","timestamp":"%s","properties":{"telemetry_schema_version":1,"event_version":1,"product":"cli","product_version":"%s","os_family":"%s","os_major":%s,"arch":"%s","is_ci":%s,"is_synthetic":%s,"process_session_id":"%s","sample_rate":1,"$process_person_profile":false,"$geoip_disable":true,"$lib":"cua-installer","$lib_version":"1.0.0","step":"%s","outcome":"%s","install_stage":"%s","error_kind":"%s","installer":"sh","items":"%s"}}]}' \
+ "$tel_key" "$tel_id" "$uuid" "$(date -u +%Y-%m-%dT%H:%M:%SZ)" "$pv" "$(tel_os)" "$(tel_os_major)" "$(tel_arch)" \
+ "$tel_ci" "$synthetic" "$tel_session" "$step_name" "$outcome" "$stage" "$kind" "$(tel_items)")"
+ [ "$(tel_bool "${CUA_TELEMETRY_DEBUG:-}")" != on ] || printf '[cua telemetry] %s\n' "$body" >&2
+ if has curl; then
+ curl -s -o /dev/null --max-time 3 -H 'Content-Type: application/json' --data "$body" "$tel_endpoint" >/dev/null 2>&1 || true
+ else
+ wget -q -O /dev/null --timeout=3 --tries=1 --header='Content-Type: application/json' --post-data="$body" "$tel_endpoint" >/dev/null 2>&1 || true
+ fi
+}
+
# ---------------------------------------------------------------- arguments
parse_args() {
@@ -641,6 +851,7 @@ load_manifest() {
fetch "$manifest_url" "$manifest"
grep -q '"schema" *: *1' "$manifest" || die "unsupported release manifest at $manifest_url"
manifest_dir="${manifest_url%/*}"
+ tel_version="$(sed -n 's/.*"version" *: *"\([0-9][0-9A-Za-z.-]*\)".*/\1/p' "$manifest" | head -n 1)"
}
# resolve_url VALUE: absolute URLs pass through; names are relative to the manifest.
@@ -1176,9 +1387,12 @@ run_onboarding() {
main() {
parse_args "$@"
+ telemetry_init
+ tel_stage=platform
detect_platform
init_selection
reuse_installed
+ tel_send installer_started
tmp="$(mktemp -d 2>/dev/null || mktemp -d -t cua-install)"
if wants_checklist; then
checklist
@@ -1188,6 +1402,7 @@ main() {
fi
# Nothing to download (--pair on a computer that has Cua): no manifest to read.
if [ "$want_cli" = 1 ] || [ "$sel_spaces" = 1 ] || [ "$sel_driver" = 1 ]; then
+ tel_stage=manifest
load_manifest
info "platform $platform, manifest $manifest_url"
fi
@@ -1200,12 +1415,15 @@ main() {
cua_bin="${existing_cua:-cua}"
app_path=""
if [ "$want_cli" = 1 ]; then
+ tel_stage=cli
install_cli
fi
+ tel_stage=runtime
if [ "$os" = linux ] && [ "$sel_spaces" = 1 ] && { [ "$dry_run" = 1 ] || has "$cua_bin"; }; then
linux_runtime
fi
if [ "$sel_spaces" = 1 ]; then
+ tel_stage=app
case "$os" in
darwin) install_app_darwin ;;
linux) install_app_linux ;;
@@ -1217,12 +1435,17 @@ main() {
CUA_INSTALL_CHANNEL=install_script
export CUA_INSTALL_CHANNEL
if [ "$sel_driver" = 1 ]; then
+ tel_stage=driver
install_driver
fi
+ tel_stage=sign_in
run_onboarding
if [ "$sel_host" = 1 ]; then
+ tel_stage=host
host_setup
fi
+ tel_stage=other
+ tel_send installer_completed
summary
if [ -n "$path_hint" ]; then
say ""
diff --git a/scripts/install/tests/fixtures/installer-step.json b/scripts/install/tests/fixtures/installer-step.json
new file mode 100644
index 0000000000..0ac76678ca
--- /dev/null
+++ b/scripts/install/tests/fixtures/installer-step.json
@@ -0,0 +1,86 @@
+{
+ "_comment": "What install.sh and install.ps1 send (cua_installer_step). libs/cua/crates/cua-telemetry/tests/privacy.rs validates these against the schema; scripts/install/tests/test_install_telemetry.py checks install.sh sends exactly these properties. Ids, times and host values vary per run.",
+ "events": [
+ {
+ "event": "cua_installer_step",
+ "distinct_id": "0123456789abcdef0123456789abcdef",
+ "properties": {
+ "telemetry_schema_version": 1,
+ "event_version": 1,
+ "product": "cli",
+ "product_version": "0.0.0",
+ "os_family": "macos",
+ "os_major": 26,
+ "arch": "arm64",
+ "is_ci": false,
+ "is_synthetic": false,
+ "process_session_id": "fedcba9876543210fedcba9876543210",
+ "sample_rate": 1,
+ "$process_person_profile": false,
+ "$geoip_disable": true,
+ "$lib": "cua-installer",
+ "$lib_version": "1.0.0",
+ "step": "installer_started",
+ "outcome": "ok",
+ "install_stage": "none",
+ "error_kind": "none",
+ "installer": "sh",
+ "items": "cli+spaces"
+ }
+ },
+ {
+ "event": "cua_installer_step",
+ "distinct_id": "0123456789abcdef0123456789abcdef",
+ "properties": {
+ "telemetry_schema_version": 1,
+ "event_version": 1,
+ "product": "cli",
+ "product_version": "1.2.3",
+ "os_family": "linux",
+ "os_major": 24,
+ "arch": "x86_64",
+ "is_ci": false,
+ "is_synthetic": false,
+ "process_session_id": "fedcba9876543210fedcba9876543210",
+ "sample_rate": 1,
+ "$process_person_profile": false,
+ "$geoip_disable": true,
+ "$lib": "cua-installer",
+ "$lib_version": "1.0.0",
+ "step": "installer_failed",
+ "outcome": "error",
+ "install_stage": "cli",
+ "error_kind": "checksum",
+ "installer": "sh",
+ "items": "cli"
+ }
+ },
+ {
+ "event": "cua_installer_step",
+ "distinct_id": "0123456789abcdef0123456789abcdef",
+ "properties": {
+ "telemetry_schema_version": 1,
+ "event_version": 1,
+ "product": "cli",
+ "product_version": "1.2.3",
+ "os_family": "windows",
+ "os_major": 10,
+ "arch": "x86_64",
+ "is_ci": false,
+ "is_synthetic": false,
+ "process_session_id": "fedcba9876543210fedcba9876543210",
+ "sample_rate": 1,
+ "$process_person_profile": false,
+ "$geoip_disable": true,
+ "$lib": "cua-installer",
+ "$lib_version": "1.0.0",
+ "step": "installer_completed",
+ "outcome": "ok",
+ "install_stage": "none",
+ "error_kind": "none",
+ "installer": "ps1",
+ "items": "cli+spaces+cua_driver"
+ }
+ }
+ ]
+}
diff --git a/scripts/install/tests/install.Tests.ps1 b/scripts/install/tests/install.Tests.ps1
index 0cb93fc47e..0566b4c3c4 100644
--- a/scripts/install/tests/install.Tests.ps1
+++ b/scripts/install/tests/install.Tests.ps1
@@ -30,7 +30,8 @@ BeforeAll {
function script:Invoke-Installer([hashtable]$Params, [hashtable]$Vars = @{}) {
$saved = @{}
- $all = @{ CUA_INSTALL_MANIFEST_URL = $env:CUA_TEST_MANIFEST_URL; CUA_INSTALL_ARCH = 'AMD64'; CUA_HOME = (Join-Path $env:CUA_TEST_ROOT 'cuahome') }
+ # Install telemetry may only reach a loopback endpoint here.
+ $all = @{ CUA_INSTALL_MANIFEST_URL = $env:CUA_TEST_MANIFEST_URL; CUA_INSTALL_ARCH = 'AMD64'; CUA_HOME = (Join-Path $env:CUA_TEST_ROOT 'cuahome'); CUA_TELEMETRY_FORBID_NETWORK = '1' }
foreach ($k in $Vars.Keys) { $all[$k] = $Vars[$k] }
foreach ($k in $all.Keys) { $saved[$k] = [Environment]::GetEnvironmentVariable($k); [Environment]::SetEnvironmentVariable($k, $all[$k]) }
try {
@@ -421,3 +422,98 @@ Describe 'install.ps1' {
$out | Should -Match 'spaces-install-mode'
}
}
+
+Describe 'install telemetry' {
+ BeforeAll {
+ $env:CUA_INSTALL_NO_RUN = '1'
+ . $script:Installer
+ Remove-Item Env:CUA_INSTALL_NO_RUN
+ $script:Fixture = Get-Content -Raw (Join-Path $PSScriptRoot 'fixtures/installer-step.json') | ConvertFrom-Json
+ $script:Switches = @('DO_NOT_TRACK', 'CUA_TELEMETRY', 'CUA_TELEMETRY_ENABLED', 'CUA_TELEMETRY_DISABLED') + $script:TelCiVars
+ function script:With-Env([hashtable]$Vars, [scriptblock]$Body) {
+ $saved = @{}
+ foreach ($k in $script:Switches + @($Vars.Keys)) { $saved[$k] = [Environment]::GetEnvironmentVariable($k); [Environment]::SetEnvironmentVariable($k, $null) }
+ foreach ($k in $Vars.Keys) { [Environment]::SetEnvironmentVariable($k, $Vars[$k]) }
+ try { return (& $Body) } finally { foreach ($k in $saved.Keys) { [Environment]::SetEnvironmentVariable($k, $saved[$k]) } }
+ }
+ function script:Get-Events([string]$Out) {
+ return @(($Out -split "`r?`n") | Where-Object { $_ -like '`[cua telemetry`] *' } | ForEach-Object { ($_.Substring(16) | ConvertFrom-Json).batch[0] })
+ }
+ }
+
+ It 'follows the same switches as every Cua program' {
+ $h = Join-Path $env:CUA_TEST_ROOT 'tel-home'
+ New-Item -ItemType Directory -Force -Path $h | Out-Null
+ With-Env @{} { Test-TelemetryEnabled $h } | Should -BeTrue
+ With-Env @{ DO_NOT_TRACK = '1'; CUA_TELEMETRY = '1' } { Test-TelemetryEnabled $h } | Should -BeFalse
+ With-Env @{ CUA_TELEMETRY = '0' } { Test-TelemetryEnabled $h } | Should -BeFalse
+ With-Env @{ CUA_TELEMETRY_ENABLED = 'false' } { Test-TelemetryEnabled $h } | Should -BeFalse
+ With-Env @{ CUA_TELEMETRY_DISABLED = '1' } { Test-TelemetryEnabled $h } | Should -BeFalse
+ With-Env @{ GITHUB_ACTIONS = 'true' } { Test-TelemetryEnabled $h } | Should -BeFalse
+ With-Env @{ CI = 'true'; CUA_TELEMETRY = '1' } { Test-TelemetryEnabled $h } | Should -BeTrue
+ Set-Content -LiteralPath (Join-Path $h 'config.toml') -Value "[other]`nenabled = `"on`"`n[telemetry]`nenabled = `"off`""
+ With-Env @{} { Test-TelemetryEnabled $h } | Should -BeFalse
+ }
+
+ It 'names a failure with a fixed word, never the message' {
+ Get-CuaInstallErrorKind 'cua-install: error: checksum mismatch for cua.zip' | Should -Be 'checksum'
+ Get-CuaInstallErrorKind 'cua-install: error: cosign signature check failed' | Should -Be 'signature'
+ Get-CuaInstallErrorKind 'cua-install: error: unsupported CPU architecture: x86' | Should -Be 'unsupported_platform'
+ Get-CuaInstallErrorKind 'Unable to connect to the remote server' | Should -Be 'network'
+ Get-CuaInstallErrorKind 'C:\Users\alice\secret' | Should -Be 'other'
+ }
+
+ It 'sends nothing on a dry run, with DO_NOT_TRACK, or to a real endpoint in tests' {
+ foreach ($vars in @(@{ DO_NOT_TRACK = '1' }, @{ CUA_TELEMETRY_ENDPOINT = 'https://eu.i.posthog.com/batch/' })) {
+ $vars.CUA_TELEMETRY_DEBUG = '1'
+ $vars.CUA_HOME = Join-Path $env:CUA_TEST_ROOT ('tel-off-' + [Guid]::NewGuid().ToString('N'))
+ $out = Invoke-Installer @{ CliOnly = $true; Yes = $true; NoOnboarding = $true; NoModifyPath = $true; Prefix = (Join-Path $env:CUA_TEST_ROOT 'p-tel-off') } $vars
+ $out | Should -Not -Match 'cua telemetry'
+ $out | Should -Not -Match 'anonymous data about this install'
+ Test-Path (Join-Path $vars.CUA_HOME 'telemetry/install_id') | Should -BeFalse
+ }
+ $out = Invoke-Installer @{ DryRun = $true; CliOnly = $true } @{ CUA_TELEMETRY_DEBUG = '1'; CUA_TELEMETRY_ENDPOINT = 'http://127.0.0.1:9/batch/' }
+ $out | Should -Not -Match 'cua telemetry'
+ }
+
+ It 'sends started then completed with the fixture properties and the stored install id' {
+ $cuaHome = Join-Path $env:CUA_TEST_ROOT 'tel-on'
+ $vars = @{ CUA_TELEMETRY_DEBUG = '1'; CUA_TELEMETRY_ENDPOINT = 'http://127.0.0.1:9/batch/'; CUA_HOME = $cuaHome; CUA_POSTHOG_KEY = 'phc_test' }
+ $out = Invoke-Installer @{ CliOnly = $true; Yes = $true; NoOnboarding = $true; NoModifyPath = $true; Prefix = (Join-Path $env:CUA_TEST_ROOT 'p-tel') } $vars
+ $out | Should -Match 'anonymous data about this install'
+ $events = Get-Events $out
+ @($events | ForEach-Object { $_.properties.step }) | Should -Be @('installer_started', 'installer_completed')
+ $id = (Get-Content -Raw (Join-Path $cuaHome 'telemetry/install_id')).Trim()
+ $ref = $script:Fixture.events[2].properties
+ foreach ($e in $events) {
+ $e.distinct_id | Should -Be $id
+ $e.event | Should -Be 'cua_installer_step'
+ @($e.properties.PSObject.Properties.Name | Sort-Object) | Should -Be @($ref.PSObject.Properties.Name | Sort-Object)
+ $e.properties.installer | Should -Be 'ps1'
+ $e.properties.items | Should -Be 'cli'
+ $e.properties.'$lib' | Should -Be 'cua-installer'
+ }
+ $events[1].properties.product_version | Should -Be '1.2.3'
+ $out | Should -Not -Match ([regex]::Escape('"api_key":"phc_eSk'))
+ }
+
+ It 'sends installer_failed with the stage and kind when the install stops' {
+ $vars = @{ CUA_INSTALL_MANIFEST_URL = (ConvertTo-FileUrl (Join-Path $env:CUA_TEST_ROOT 'missing/release-artifacts.json'))
+ CUA_INSTALL_ARCH = 'AMD64'; CUA_HOME = (Join-Path $env:CUA_TEST_ROOT 'tel-fail'); CUA_TELEMETRY_FORBID_NETWORK = '1'
+ CUA_TELEMETRY_DEBUG = '1'; CUA_TELEMETRY_ENDPOINT = 'http://127.0.0.1:9/batch/'
+ }
+ $out = With-Env $vars {
+ $params = @{ CliOnly = $true; Yes = $true; NoOnboarding = $true; NoModifyPath = $true; Prefix = (Join-Path $env:CUA_TEST_ROOT 'p-tel-fail') }
+ $lines = & { try { & $env:CUA_TEST_INSTALLER @params } catch { "threw: $_" } } *>&1
+ ($lines | ForEach-Object { "$_" }) -join "`n"
+ }
+ $out | Should -Match 'threw: '
+ $events = Get-Events $out
+ @($events | ForEach-Object { $_.properties.step }) | Should -Be @('installer_started', 'installer_failed')
+ $failed = $events[1].properties
+ $failed.install_stage | Should -Be 'manifest'
+ $failed.outcome | Should -Be 'error'
+ $failed.error_kind | Should -BeIn @('network', 'other')
+ ($events | ConvertTo-Json -Depth 6) | Should -Not -Match 'missing'
+ }
+}
diff --git a/scripts/install/tests/run.sh b/scripts/install/tests/run.sh
index 4062230fdf..524fc08664 100755
--- a/scripts/install/tests/run.sh
+++ b/scripts/install/tests/run.sh
@@ -158,7 +158,8 @@ inst() {
done
[ "${1:-}" = -- ] && shift
# shellcheck disable=SC2086
- env -i PATH="$PATH" HOME="$home" SHELL=/bin/sh CUA_HOME="$home/.cua" \
+ # Install telemetry may only reach a loopback endpoint here (none is set).
+ env -i PATH="$PATH" HOME="$home" SHELL=/bin/sh CUA_HOME="$home/.cua" CUA_TELEMETRY_FORBID_NETWORK=1 \
CUA_INSTALL_BASE_URL="$base" CUA_INSTALL_OS=linux CUA_INSTALL_ARCH=x86_64 \
CUA_FAKE_LOG="$home/cua.log" $envs \
"$sh_bin" "$installer" "$@" </dev/null >"$work/out" 2>&1
diff --git a/scripts/install/tests/test_install_telemetry.py b/scripts/install/tests/test_install_telemetry.py
new file mode 100644
index 0000000000..867e147b0e
--- /dev/null
+++ b/scripts/install/tests/test_install_telemetry.py
@@ -0,0 +1,217 @@
+"""Install telemetry in install.sh (python3 -m unittest discover scripts/install/tests).
+
+Hermetic: a throwaway HOME and a loopback server that serves a fake release
+and records every telemetry POST. CUA_TELEMETRY_FORBID_NETWORK=1 is set on
+every run, so nothing can reach a real endpoint. The payload must carry
+exactly the properties of fixtures/installer-step.json, which the Rust
+privacy tests validate against the telemetry schema.
+"""
+
+import http.server
+import json
+import os
+import subprocess
+import tarfile
+import tempfile
+import threading
+import unittest
+from pathlib import Path
+
+HERE = Path(__file__).resolve().parent
+SCRIPT = HERE.parent / "install.sh"
+MANIFEST_TOOL = HERE.parent / "release_manifest.py"
+FIXTURE = json.loads((HERE / "fixtures" / "installer-step.json").read_text())
+VERSION = "1.2.3"
+# Values that vary per run or per host; everything else must match the fixture.
+VARYING = {"product_version", "os_family", "os_major", "arch", "process_session_id",
+ "step", "outcome", "install_stage", "error_kind", "items", "is_ci"}
+
+
+class Recorder(http.server.SimpleHTTPRequestHandler):
+ posts = []
+
+ def log_message(self, *args):
+ pass
+
+ def do_POST(self):
+ n = int(self.headers.get("Content-Length", "0"))
+ Recorder.posts.append(json.loads(self.rfile.read(n)))
+ self.send_response(200)
+ self.end_headers()
+ self.wfile.write(b"{}")
+
+
+class InstallTelemetry(unittest.TestCase):
+ @classmethod
+ def setUpClass(cls):
+ cls.tmp = tempfile.TemporaryDirectory()
+ root = Path(cls.tmp.name)
+ cls.release = root / "release"
+ tag = cls.release / f"cua-sdk-v{VERSION}"
+ tag.mkdir(parents=True)
+ stage = root / "stage"
+ stage.mkdir()
+ cua = stage / "cua"
+ cua.write_text(f'#!/bin/sh\n[ "$1" = --version ] && echo "cua {VERSION}"\nexit 0\n')
+ cua.chmod(0o755)
+ with tarfile.open(tag / f"cua-cli-{VERSION}-linux-x64.tar.gz", "w:gz") as t:
+ t.add(cua, arcname="cua")
+ handler = lambda *a, **k: Recorder(*a, directory=str(cls.release), **k) # noqa: E731
+ cls.server = http.server.ThreadingHTTPServer(("127.0.0.1", 0), handler)
+ threading.Thread(target=cls.server.serve_forever, daemon=True).start()
+ cls.base = f"http://127.0.0.1:{cls.server.server_address[1]}"
+ subprocess.run(
+ ["python3", str(MANIFEST_TOOL), "--component", "cli", "--version", VERSION,
+ "--dir", str(tag), "--base-url", f"{cls.base}/cua-sdk-v{VERSION}",
+ "--out", str(tag / "release-artifacts.json")],
+ check=True,
+ )
+ cls.manifest = f"{cls.base}/cua-sdk-v{VERSION}/release-artifacts.json"
+
+ @classmethod
+ def tearDownClass(cls):
+ cls.server.shutdown()
+ cls.tmp.cleanup()
+
+ def setUp(self):
+ Recorder.posts.clear()
+ self.home = Path(tempfile.mkdtemp(dir=self.tmp.name))
+
+ def run_installer(self, *args, env=None, manifest=None):
+ e = {
+ "PATH": os.environ["PATH"],
+ "HOME": str(self.home),
+ "CUA_HOME": str(self.home / ".cua"),
+ "CUA_INSTALL_OS": "linux",
+ "CUA_INSTALL_ARCH": "x86_64",
+ "CUA_INSTALL_MANIFEST_URL": manifest or self.manifest,
+ "CUA_TELEMETRY_FORBID_NETWORK": "1",
+ "CUA_TELEMETRY_ENDPOINT": f"{self.base}/batch/",
+ "CUA_POSTHOG_KEY": "phc_test",
+ }
+ e.update(env or {})
+ p = subprocess.run(
+ ["sh", str(SCRIPT), "--yes", "--no-onboarding", "--prefix", str(self.home / "p"), *args],
+ env=e, stdin=subprocess.DEVNULL, capture_output=True, text=True, timeout=120,
+ )
+ return p
+
+ def events(self):
+ out = []
+ for body in Recorder.posts:
+ self.assertEqual(body["api_key"], "phc_test")
+ out.extend(body["batch"])
+ return out
+
+ def assert_matches_fixture(self, ev):
+ ref = FIXTURE["events"][0]
+ self.assertEqual(ev["event"], ref["event"])
+ self.assertEqual(set(ev["properties"]), set(ref["properties"]))
+ for k, v in ref["properties"].items():
+ if k not in VARYING:
+ self.assertEqual(ev["properties"][k], v, k)
+ self.assertRegex(ev["distinct_id"], r"^[0-9a-f]{32}$")
+ self.assertRegex(ev["properties"]["process_session_id"], r"^[0-9a-f]{32}$")
+ self.assertRegex(ev["uuid"], r"^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$")
+ self.assertRegex(ev["timestamp"], r"^\d{4}-\d\d-\d\dT\d\d:\d\d:\d\dZ$")
+
+ def test_a_finished_install_sends_started_then_completed(self):
+ p = self.run_installer("--cli-only")
+ self.assertEqual(p.returncode, 0, p.stdout + p.stderr)
+ evs = self.events()
+ self.assertEqual([e["properties"]["step"] for e in evs], ["installer_started", "installer_completed"])
+ for e in evs:
+ self.assert_matches_fixture(e)
+ self.assertEqual(e["properties"]["outcome"], "ok")
+ self.assertEqual(e["properties"]["install_stage"], "none")
+ self.assertEqual(e["properties"]["error_kind"], "none")
+ self.assertEqual(e["properties"]["items"], "cli")
+ self.assertEqual(evs[1]["properties"]["product_version"], VERSION)
+ # The same anonymous id the Cua programs use afterwards, owner-only.
+ id_file = self.home / ".cua" / "telemetry" / "install_id"
+ self.assertEqual(id_file.read_text().strip(), evs[0]["distinct_id"])
+ self.assertEqual(id_file.stat().st_mode & 0o077, 0)
+ self.assertEqual(evs[0]["distinct_id"], evs[1]["distinct_id"])
+ self.assertIn("anonymous data about this install", p.stdout)
+ self.assertIn("DO_NOT_TRACK=1", p.stdout)
+ # Nothing from the machine: no home path, no user name.
+ raw = json.dumps(Recorder.posts)
+ self.assertNotIn(str(self.home), raw)
+ self.assertNotIn(os.environ.get("USER", "nobody-here"), raw)
+
+ def test_an_existing_install_id_is_kept(self):
+ d = self.home / ".cua" / "telemetry"
+ d.mkdir(parents=True)
+ (d / "install_id").write_text("0123456789abcdef0123456789abcdef")
+ self.run_installer("--cli-only")
+ self.assertTrue(self.events())
+ for e in self.events():
+ self.assertEqual(e["distinct_id"], "0123456789abcdef0123456789abcdef")
+
+ def test_a_failed_install_says_where_and_why_never_the_message(self):
+ p = self.run_installer("--cli-only", manifest=f"{self.base}/no-such-release/release-artifacts.json")
+ self.assertNotEqual(p.returncode, 0)
+ evs = self.events()
+ self.assertEqual([e["properties"]["step"] for e in evs], ["installer_started", "installer_failed"])
+ failed = evs[1]["properties"]
+ self.assertEqual((failed["outcome"], failed["install_stage"], failed["error_kind"]),
+ ("error", "manifest", "network"))
+ self.assert_matches_fixture(evs[1])
+ self.assertNotIn("no-such-release", json.dumps(Recorder.posts))
+
+ def test_every_switch_turns_it_off(self):
+ cfg = self.home / ".cua"
+ cases = [
+ {"DO_NOT_TRACK": "1"},
+ {"DO_NOT_TRACK": "1", "CUA_TELEMETRY": "1"},
+ {"CUA_TELEMETRY": "0"},
+ {"CUA_TELEMETRY_ENABLED": "false"},
+ {"CUA_TELEMETRY_DISABLED": "1"},
+ {"CI": "true"},
+ {"GITHUB_ACTIONS": "true"},
+ "config",
+ "dry-run",
+ # The test guard: a real endpoint is refused before anything happens.
+ {"CUA_TELEMETRY_ENDPOINT": "https://eu.i.posthog.com/batch/"},
+ ]
+ for case in cases:
+ with self.subTest(case=case):
+ Recorder.posts.clear()
+ self.home = Path(tempfile.mkdtemp(dir=self.tmp.name))
+ cfg = self.home / ".cua"
+ args, env = ["--cli-only"], {}
+ if case == "config":
+ cfg.mkdir(parents=True)
+ (cfg / "config.toml").write_text('[other]\nenabled = "on"\n\n[telemetry]\nenabled = "off"\n')
+ elif case == "dry-run":
+ args.append("--dry-run")
+ else:
+ env = case
+ p = self.run_installer(*args, env=env)
+ self.assertEqual(Recorder.posts, [], p.stdout + p.stderr)
+ self.assertNotIn("anonymous data about this install", p.stdout)
+ self.assertFalse((cfg / "telemetry" / "install_id").exists())
+
+ def test_ci_with_cua_telemetry_on_sends_and_is_marked(self):
+ p = self.run_installer("--cli-only", env={"CI": "true", "CUA_TELEMETRY": "1"})
+ self.assertEqual(p.returncode, 0, p.stdout + p.stderr)
+ evs = self.events()
+ self.assertEqual(len(evs), 2)
+ self.assertTrue(all(e["properties"]["is_ci"] is True for e in evs))
+
+ def test_staging_sends_to_the_staging_project(self):
+ self.run_installer("--cli-only", env={"CUA_ENV": "staging", "CUA_POSTHOG_KEY": ""})
+ self.assertEqual(len(Recorder.posts), 2)
+ for body in Recorder.posts:
+ self.assertTrue(body["api_key"].startswith("phc_xDNL"), "staging project key")
+
+ def test_a_config_file_on_wins_over_ci(self):
+ cfg = self.home / ".cua"
+ cfg.mkdir(parents=True)
+ (cfg / "config.toml").write_text('[telemetry]\nenabled = true\n')
+ self.run_installer("--cli-only", env={"CI": "1"})
+ self.assertEqual(len(self.events()), 2)
+
+
+if __name__ == "__main__":
+ unittest.main()