The ultimate harness for coding agents and beyond. All your agents. All your machines. One command center. Start with code, then follow your curiosity and build across disciplines: CAD, circuits, robots, games and music.
TypeScript 3d-modelingagent-frameworkagent-harnessai-agentscadclaude-codecodexcoding-agentsdeveloper-toolsesp32game-developmentmcp-servermulti-agentopen-hardwareopencodeopenharnesspcbroboticsterminaltmux
perf: measure core UI workflows under output and process resource use
dee2fe8e6fc49af
diff --git a/desktop/tool/native_benchmark/NativeBenchmark.swift b/desktop/tool/native_benchmark/NativeBenchmark.swiftindex 965c2cfcf..f067d8e32 100644--- a/desktop/tool/native_benchmark/NativeBenchmark.swift+++ b/desktop/tool/native_benchmark/NativeBenchmark.swift@@ -3,6 +3,8 @@// Runner sources and the user's V2 bundle never contain this channel.private enum NativeBenchmark {private static var channel: FlutterMethodChannel?+ private static var focusLosses = 0+ private static var focusObserver: NSObjectProtocol?static func install(window: NSWindow, messenger: FlutterBinaryMessenger) {guard Bundle.main.bundleIdentifier == "ai.autonomous.harness.benchmark",@@ -46,6 +48,18 @@ private enum NativeBenchmark {// Once, after the synthetic UI is mounted. Never reset native or Dart// focus between observations: navigation must perform its own handoff.result(String(describing: type(of: window.firstResponder!)))+ case "captureState":+ if focusObserver == nil {+ focusObserver = NotificationCenter.default.addObserver(+ forName: NSWindow.didResignKeyNotification, object: window, queue: .main+ ) { _ in focusLosses += 1 }+ }+ result([+ "key": window.isKeyWindow,+ "active": NSApp.isActive,+ "visible": window.isVisible,+ "focusLosses": focusLosses,+ ])case "key":guard window.isKeyWindow, NSApp.isActive,let args = call.arguments as? [String: Any],diff --git a/desktop/tool/native_benchmark/README.md b/desktop/tool/native_benchmark/README.mdindex 6f1f5a8f0..f98e554a8 100644--- a/desktop/tool/native_benchmark/README.md+++ b/desktop/tool/native_benchmark/README.md@@ -3,6 +3,67 @@Start with the [September 22 wrap-up and resume notes](../../../docs/performance/2026-09-22-wrap-up.md)for the ready PRs, measured improvements, rejected experiments and remaining priorities.+## Core experiences and process resources++```sh+python3 tool/native_benchmark/prepare.py --flutter /path/to/flutter \+ --core-workflows --terminals 16 --samples 120 --hold+```++Open the printed isolated app through normal application controls. Use three+samples for calibration first, then 120 for reporting. `run-config.json` in the+printed temporary root selects `terminals`, `samples`, `hold`, and a fresh JSON+`output` filename. After quitting the fixture, edit that file and reopen the same+bundle to repeat without changing its compiled production code. Existing results+are never overwritten. `active-run.json` identifies the latest invocation. The core mode+supports 1, 16, or 48 retained terminals, with up to four visible and 1,000+initial scrollback rows each. It visits every retained tab before measuring,+then repeats at idle and while every terminal receives an eight-row ANSI redraw+at 20 Hz. Actual bytes, duration, and skipped output bursts are retained.++It validates eleven common actions: text commit with a synthetic terminal echo;+Cmd+N; Cmd+O; picker query; picker selection followed by immediate input; Cmd+T;+tab change followed by input; pane focus followed by input; zoom; Find through a+completed index scan; and a page-sized scroll. Inapplicable navigation actions+are skipped with one terminal. Five warmups precede each series. Every input+commit must arrive exactly once in the focused terminal. Picker queries and Find+must produce the expected real results before their frame is accepted.++These are **framework dispatch to completed Flutter raster** measurements.+Text uses the attached `TextInputClient`; shortcuts use the normal framework+keyboard/focus dispatcher. Text echo passes through the production terminal+binary decoder and parser with zero simulated network delay. Navigation-plus-+input measurements include the navigation frame and the following verified echo+frame; they are not interchangeable with a shortcut's first-frame time. Scrolling+uses the terminal's scroll position controller and excludes native wheel-event+delivery. AppKit keyboard delivery, actual networking, model inference, GPU+presentation, and native titlebar paint completion remain outside this boundary.++The native host records loss of foreground focus; those runs fail rather than+mixing foreground and background timings. `.progress` identifies the current+invocation and last completed operation. Preserve failed runs for diagnosis and+use the result's fresh timestamps rather than accepting an old result file.+`--hold` leaves the idle fixture open after the workload for resource sampling:++```sh+xcrun swiftc tool/native_benchmark/process_usage.swift -o /private/tmp/harness-process-usage+/private/tmp/harness-process-usage PID 30 foreground-idle /private/tmp/usage-foreground.json+```++Use the PID in the completed result's metadata. Hide the fixture through the app's+normal controls and repeat with a new label/file for background idle. This sampler+reads `proc_pid_rusage` once per second: CPU deltas (100% means one core), physical+memory footprint, resident memory, interrupt/package-idle wakeups, and disk I/O.+It does not control windows. Its totals cover that process, excluding daemon,+agent processes, and GPU energy. The fixture has no real network connections, so+its background cost cannot establish that the connected product has zero timers+or zero wakeups. Record other app activity and host contention; these are real+workstation measurements, not results from an otherwise isolated machine.++For actual loopback/direct/TURN/relay terminal round trips, use the separate+[real terminal probe](../../../cli/scripts/benchmark-terminal-latency.md). Never+add independently measured p95 values and label the sum end-to-end latency.+## Framework-dispatch comparison and manual feature checksThe fixture supports three separate modes:diff --git a/desktop/tool/native_benchmark/core_driver.dart b/desktop/tool/native_benchmark/core_driver.dartnew file mode 100644index 000000000..966674005--- /dev/null+++ b/desktop/tool/native_benchmark/core_driver.dart@@ -0,0 +1,485 @@+// Framework text/key dispatch through the production parser and Release renderer.+// No OS input, network, model, or display-presentation latency is claimed here.+import 'dart:async';+import 'dart:convert';+import 'dart:io';+import 'dart:ui' as ui;++import 'package:flutter/material.dart';+import 'package:flutter/services.dart';+import 'package:harness/state/app_state.dart';+import 'package:harness/state/new_harness.dart';+import 'package:harness/widgets/new_harness_box.dart';+import 'package:harness/widgets/swarm_search_input.dart';+import 'package:harness/widgets/terminal_find_bar.dart';+import 'package:harness/widgets/workspace_welcome.dart';+import 'package:xterm/src/ui/custom_text_edit.dart';+import 'package:xterm/xterm.dart';++import 'flutter_driver.dart';++const _escape = (LogicalKeyboardKey.escape, PhysicalKeyboardKey.escape);+const _enter = (LogicalKeyboardKey.enter, PhysicalKeyboardKey.enter);++void _check(bool ok, String message) {+ if (!ok) throw StateError(message);+}++TextEditingValue _editingValue() {+ final terminal = benchmarkFind(+ (w) => w is CustomTextEdit && w.focusNode.hasFocus,+ );+ if (terminal != null) {+ final state = (terminal as StatefulElement).state as CustomTextEditState;+ _check(+ state.hasInputConnection,+ 'Focused terminal has no text input connection',+ );+ return state.currentTextEditingValue!;+ }+ final field = benchmarkFind((w) => w is EditableText && w.focusNode.hasFocus);+ if (field != null) return (field.widget as EditableText).controller.value;+ throw StateError(+ 'No focused text input: ${FocusManager.instance.primaryFocus}',+ );+}++void _insert(String text, {bool replace = false}) {+ final value = _editingValue();+ final selection = replace+ ? TextSelection(baseOffset: 0, extentOffset: value.text.length)+ : value.selection;+ _check(selection.isValid, 'Text input has no valid selection');+ final updated = value.text.replaceRange(selection.start, selection.end, text);+ TextInput.updateEditingValue(+ TextEditingValue(+ text: updated,+ selection: TextSelection.collapsed(offset: selection.start + text.length),+ ),+ );+}++Future<void> runCoreBenchmark(+ AppNotifier app,+ String output,+ Map<String, Object?> metadata,+ Map<int, ui.FrameTiming> timings, {+ required int samples,+ required List<(String, String)> inputs,+ required Future<void> Function(bool) setOutput,+ required Map<String, int> Function() outputCounters,+}) async {+ final rows = <Map<String, Object?>>[];+ final phases = <Map<String, Object?>>[];+ final fixtureCount = app.allPanes.length;+ final firstTab = app.activeSwarmId;+ final startedAt = DateTime.now().toUtc().toIso8601String();+ const host = MethodChannel('harness/isolated_benchmark');+ var initialWindow = await host.invokeMapMethod<String, Object?>(+ 'captureState',+ );+ // Normal app launch can initially be backgrounded. Let the operator raise+ // this fixture once; never reclaim focus during the measured series.+ for (+ var attempt = 0;+ attempt < 120 &&+ (initialWindow?['key'] != true || initialWindow?['active'] != true);+ attempt+++ ) {+ await Future<void>.delayed(const Duration(milliseconds: 250));+ initialWindow = await host.invokeMapMethod<String, Object?>('captureState');+ }+ // Warm every retained renderer once, not just the first workspace.+ for (final tab in app.swarms.toList()) {+ app.selectSwarm(tab.id, attachPending: false);+ await benchmarkFrame();+ }+ app.selectSwarm(firstTab, attachPending: false);+ await benchmarkFrame();++ Future<void> terminalInput() async {+ final before = inputs.length;+ final expected = app.focusedPane?.session?.agentId;+ _check(expected != null, 'No destination terminal');+ _insert('x');+ await benchmarkFrame();+ _check(+ inputs.length == before + 1 && inputs.last == (expected, 'x'),+ 'Input was lost, duplicated, or sent to the wrong terminal',+ );+ }++ Future<void> measure(+ String operation,+ String load,+ int sample,+ Future<void> Function() action,+ ) async {+ final previousFrame = await benchmarkFrame();+ final delay = (sample + 5) * 11003 % 20000;+ await Future<void>.delayed(Duration(microseconds: delay));+ final began = DateTime.now().microsecondsSinceEpoch;+ await action();+ // action has awaited the frame that validates its state. Capture that exact+ // frame, not a later unrelated redraw, and join the renderer's timing below.+ rows.add({+ 'operation': operation,+ 'load': load,+ 'phase': sample < 0 ? 'warmup' : 'measured',+ 'startedWallMicros': began,+ 'previousFrame': previousFrame,+ 'readyFrame': ui.PlatformDispatcher.instance.frameData.frameNumber,+ 'phaseDelayMicros': delay,+ });+ }++ Future<void> dismissNew() async {+ final element = benchmarkFind((w) => w is NewHarnessBox);+ if (element == null) return;+ final controller = (element.widget as NewHarnessBox).controller;+ for (+ var step = 0;+ step < 8 && controller.field != NewHarnessField.launch;+ step+++ ) {+ benchmarkKey(_escape, command: false);+ await benchmarkFrame();+ }+ benchmarkKey(_escape, command: false);+ await benchmarkFrame();+ _check(+ benchmarkFind((w) => w is NewHarnessBox) == null,+ 'New harness did not close',+ );+ }++ try {+ _check(+ initialWindow?['key'] == true && initialWindow?['active'] == true,+ 'Core measurement requires the foreground benchmark window: $initialWindow',+ );+ for (final load in ['idle', 'all_terminals_20hz']) {+ await setOutput(load != 'idle');+ final phaseBegan = DateTime.now().microsecondsSinceEpoch;+ final countersBefore = outputCounters();+ for (final operation in [+ 'typing_echo',+ 'cmd_n',+ 'cmd_o',+ 'picker_query',+ 'picker_accept_input',+ 'cmd_t',+ if (fixtureCount > 4) 'tab_input',+ if (fixtureCount > 1) ...['focus_input', 'zoom'],+ 'find_query',+ 'scroll_page',+ ]) {+ for (var sample = -5; sample < samples; sample++) {+ final originalTab = app.activeSwarmId;+ final tabsBefore = app.swarms.length;+ SwarmSearchInput? picker;+ if (operation == 'picker_query' ||+ operation == 'picker_accept_input') {+ benchmarkKey((LogicalKeyboardKey.keyO, PhysicalKeyboardKey.keyO));+ await benchmarkFrame();+ picker =+ benchmarkFind((w) => w is SwarmSearchInput && w.search != null)+ ?.widget+ as SwarmSearchInput?;+ _check(+ picker != null && picker.focusNode.hasFocus,+ 'Picker did not take input focus',+ );+ if (operation == 'picker_accept_input') {+ // Picking an agent from a different workspace can add a tile.+ // Focus another retained tile here so the declared fixture size+ // and visible count remain constant through the entire series.+ final target = app.panes+ .firstWhere(+ (pane) => pane.id != app.focusedPaneId,+ orElse: () => app.panes.first,+ )+ .agentId!;+ _insert('Fixture agent ${target.split('-').last}', replace: true);+ await benchmarkFrame();+ _check(+ picker!.search!.selected?.agentId == target,+ 'Picker selected an unexpected terminal',+ );+ }+ } else if (operation == 'find_query') {+ benchmarkKey((LogicalKeyboardKey.keyF, PhysicalKeyboardKey.keyF));+ await benchmarkFrame();+ _insert('', replace: true);+ await benchmarkFrame();+ } else if (operation == 'focus_input') {+ app.focusPaneByIndex(0);+ await benchmarkFrame();+ }++ await measure(operation, load, sample, () async {+ switch (operation) {+ case 'typing_echo':+ await terminalInput();+ case 'cmd_n':+ benchmarkKey((+ LogicalKeyboardKey.keyN,+ PhysicalKeyboardKey.keyN,+ ));+ await benchmarkFrame();+ _check(+ benchmarkFind((w) => w is NewHarnessBox) != null,+ 'Cmd+N did not open',+ );+ _editingValue(); // The visible surface must also own editable focus.+ case 'cmd_o':+ benchmarkKey((+ LogicalKeyboardKey.keyO,+ PhysicalKeyboardKey.keyO,+ ));+ await benchmarkFrame();+ final input =+ benchmarkFind(+ (w) => w is SwarmSearchInput && w.search != null,+ )?.widget+ as SwarmSearchInput?;+ _check(+ input != null && input.focusNode.hasFocus,+ 'Cmd+O is not input-ready',+ );+ case 'picker_query':+ _insert('Fixture agent 0', replace: true);+ await benchmarkFrame();+ _check(+ picker!.search!.query == 'Fixture agent 0' &&+ picker.search!.selected?.agentId == 'fixture-0',+ 'Query did not update results',+ );+ case 'picker_accept_input':+ final expected = picker!.search!.selected!.agentId;+ benchmarkKey(_enter, command: false);+ await benchmarkFrame();+ _check(+ app.focusedPane?.agentId == expected,+ 'Picker did not focus the selected terminal',+ );+ await terminalInput();+ case 'cmd_t':+ benchmarkKey((+ LogicalKeyboardKey.keyT,+ PhysicalKeyboardKey.keyT,+ ));+ await benchmarkFrame();+ _check(+ app.activeSwarmId != originalTab &&+ app.swarms.length == tabsBefore + 1 &&+ app.panes.isEmpty &&+ benchmarkFind((w) => w is WorkspaceWelcome) != null,+ 'Cmd+T did not create its welcome tab',+ );+ case 'tab_input':+ benchmarkKey((+ LogicalKeyboardKey.bracketRight,+ PhysicalKeyboardKey.bracketRight,+ ), shift: true);+ await benchmarkFrame();+ _check(app.activeSwarmId != originalTab, 'Tab did not change');+ await terminalInput();+ case 'focus_input':+ benchmarkKey((+ LogicalKeyboardKey.arrowRight,+ PhysicalKeyboardKey.arrowRight,+ ));+ await benchmarkFrame();+ _check(+ app.focusedPaneId == app.panes[1].id,+ 'Pane focus did not move',+ );+ await terminalInput();+ case 'zoom':+ benchmarkKey(_enter);+ await benchmarkFrame();+ _check(app.zoomedPaneId != null, 'Pane did not zoom');+ case 'find_query':+ _insert('project context', replace: true);+ // The text callback owns query changes; await the real index's+ // completion before claiming that the result frame is ready.+ final find =+ benchmarkFind(+ (w) => w is TerminalFindBar && w.search != null,+ )?.widget+ as TerminalFindBar?;+ _check(+ find?.search?.query == 'project context',+ 'Find query did not reach its index',+ );+ await find!.search!.settled.timeout(const Duration(seconds: 5));+ await benchmarkFrame();+ _check(+ find.search!.count > 0,+ 'Find did not return a real match',+ );+ case 'scroll_page':+ final view =+ benchmarkFind(+ (w) =>+ w is TerminalView &&+ w.focusNode?.hasFocus == true,+ )?.widget+ as TerminalView?;+ final controller = view?.scrollController;+ _check(+ controller != null && controller.hasClients,+ 'Focused terminal has no scroll position',+ );+ final position = controller!.position;+ final target = (position.pixels - position.viewportDimension)+ .clamp(position.minScrollExtent, position.maxScrollExtent);+ _check(target != position.pixels, 'Scroll did not move');+ controller.jumpTo(target);+ await benchmarkFrame();+ _check(+ (controller.offset - target).abs() < 1,+ 'Scroll destination is wrong',+ );+ }+ });++ if (operation == 'cmd_n') {+ await dismissNew();+ } else if ([+ 'cmd_o',+ 'picker_query',+ 'find_query',+ ].contains(operation)) {+ benchmarkKey(_escape, command: false);+ await benchmarkFrame();+ } else if (operation == 'cmd_t') {+ benchmarkKey((LogicalKeyboardKey.keyW, PhysicalKeyboardKey.keyW));+ await benchmarkFrame();+ _check(+ app.swarms.length == tabsBefore,+ 'Benchmark tab was not closed',+ );+ app.selectSwarm(originalTab, attachPending: false);+ await benchmarkFrame();+ } else if (operation == 'zoom') {+ benchmarkKey(_enter);+ await benchmarkFrame();+ _check(app.zoomedPaneId == null, 'Zoom did not restore');+ } else if (operation == 'scroll_page') {+ final view =+ benchmarkFind(+ (w) => w is TerminalView && w.focusNode?.hasFocus == true,+ )!.widget+ as TerminalView;+ view.scrollController!.jumpTo(+ view.scrollController!.position.maxScrollExtent,+ );+ await benchmarkFrame();+ }+ _check(+ app.allPanes.length == fixtureCount &&+ app.panes.length == (fixtureCount < 4 ? fixtureCount : 4),+ 'Operation changed the declared terminal or visible-pane count',+ );+ }+ // A progress artifact distinguishes this invocation from any stale file.+ await File('$output.progress').writeAsString(+ jsonEncode({+ 'startedAt': startedAt,+ 'load': load,+ 'completed': operation,+ 'observations': rows.length,+ }),+ );+ }+ await setOutput(false);+ final after = outputCounters();+ phases.add({+ 'load': load,+ 'durationMicros': DateTime.now().microsecondsSinceEpoch - phaseBegan,+ for (final key in after.keys) key: after[key]! - countersBefore[key]!,+ });+ }+ for (+ var attempt = 0;+ attempt < 30 && rows.any((r) => !timings.containsKey(r['readyFrame']));+ attempt+++ ) {+ await Future<void>.delayed(const Duration(milliseconds: 100));+ await benchmarkFrame();+ }+ for (final row in rows) {+ final timing = timings[row['readyFrame']];+ _check(timing != null, 'Missing exact renderer frame');+ row['readyRasterMicros'] =+ timing!.timestampInMicroseconds(ui.FramePhase.rasterFinishWallTime) -+ (row['startedWallMicros'] as int);+ row['buildMicros'] = timing.buildDuration.inMicroseconds;+ row['rasterMicros'] = timing.rasterDuration.inMicroseconds;+ _check((row['readyRasterMicros'] as int) >= 0, 'Invalid frame timestamp');+ }+ final finalWindow = await host.invokeMapMethod<String, Object?>(+ 'captureState',+ );+ _check(+ finalWindow?['focusLosses'] == initialWindow?['focusLosses'] &&+ finalWindow?['key'] == true &&+ finalWindow?['active'] == true,+ 'Benchmark lost foreground focus; preserve this run but do not accept timings',+ );+ metadata['initialWindow'] = initialWindow;+ metadata['finalWindow'] = finalWindow;+ await File(output).writeAsString(+ const JsonEncoder.withIndent(' ').convert({+ 'success': true,+ 'kind': 'release_core_framework_dispatch_to_raster',+ 'startedAt': startedAt,+ 'finishedAt': DateTime.now().toUtc().toIso8601String(),+ 'metadata': metadata,+ 'terminals': fixtureCount,+ 'visible': fixtureCount < 4 ? fixtureCount : 4,+ 'initialScrollbackRows': 1000,+ 'samplesPerOperationPerLoad': samples,+ 'warmups': 5,+ 'inputCommitsVerified': inputs.length,+ 'phases': phases,+ 'summaries': [+ for (final load in ['idle', 'all_terminals_20hz'])+ for (final operation in rows.map((r) => r['operation']).toSet())+ {+ 'load': load,+ 'operation': operation,+ for (final metric in [+ 'readyRasterMicros',+ 'buildMicros',+ 'rasterMicros',+ ])+ metric: benchmarkDistribution([+ for (final row in rows)+ if (row['load'] == load &&+ row['operation'] == operation &&+ row['phase'] == 'measured')+ row[metric] as int,+ ]),+ },+ ],+ 'observations': rows,+ }),+ );+ } catch (error, stack) {+ await File(output).writeAsString(+ jsonEncode({+ 'success': false,+ 'startedAt': startedAt,+ 'error': '$error',+ 'stack': '$stack',+ 'observations': rows,+ }),+ );+ rethrow;+ } finally {+ await setOutput(false);+ }+}diff --git a/desktop/tool/native_benchmark/flutter_driver.dart b/desktop/tool/native_benchmark/flutter_driver.dartindex 085be8592..4a117525e 100644--- a/desktop/tool/native_benchmark/flutter_driver.dart+++ b/desktop/tool/native_benchmark/flutter_driver.dart@@ -92,6 +92,18 @@ Map<String, num> _distribution(List<int> values) {};}+// Shared by the core-experience fixture; the original benchmark cadence and+// operations remain unchanged for comparisons with its published results.+void benchmarkKey(+ (LogicalKeyboardKey, PhysicalKeyboardKey) key, {+ bool command = true,+ bool shift = false,+}) => _key(key, command: command, shift: shift);+Future<int> benchmarkFrame() => _frame();+Element? benchmarkFind(bool Function(Widget) matches) => _find(matches);+Map<String, num> benchmarkDistribution(List<int> values) =>+ _distribution(values);+Future<void> runFlutterDispatchBenchmark(AppNotifier app,String output,diff --git a/desktop/tool/native_benchmark/main.dart b/desktop/tool/native_benchmark/main.dartindex f0d9a1b30..6ab905020 100644--- a/desktop/tool/native_benchmark/main.dart+++ b/desktop/tool/native_benchmark/main.dart@@ -24,6 +24,7 @@ import 'package:harness/terminal/terminal_binary.dart';import 'package:harness/terminal/terminal_session.dart';import 'flutter_driver.dart';+import 'core_driver.dart';const _host = MethodChannel('harness/isolated_benchmark');final _timings = <int, ui.FrameTiming>{};@@ -37,6 +38,8 @@ int _skippedBursts = 0;int _inputBytes = 0;final _interactive = Platform.environment['HARNESS_BENCH_MANUAL'] == '1';final _flutterDriven = Platform.environment['HARNESS_BENCH_FLUTTER'] == '1';+final _coreDriven = Platform.environment['HARNESS_BENCH_CORE'] == '1';+final _coreInputs = <(String, String)>[];class _Sample {_Sample(@@ -146,8 +149,14 @@ Future<AppNotifier> _fixture(int count, Directory stateDirectory) async {send: (_, _) async => true,sendBinary: (packet) async {if (packet.kind != TerminalBinaryKind.input) return true;- if (_interactive || _flutterDriven) {+ if (_interactive || _flutterDriven || _coreDriven) {+ if (_coreDriven && !identical(app.focusedPane?.session, session)) {+ throw StateError('Input reached an unfocused terminal');+ }await _output(session, packet.bytes);+ if (_coreDriven) {+ _coreInputs.add((session.agentId, utf8.decode(packet.bytes)));+ }return true;}final sample = _pending;@@ -277,13 +286,42 @@ Future<void> _run() async {!env.containsKey('FLUTTER_TEST')) {throw StateError('Use the isolated native benchmark runner');}- final output = env['HARNESS_BENCH_OUTPUT'];+ var output = env['HARNESS_BENCH_OUTPUT'];if (output == null || !output.startsWith('/private/tmp/')) {throw StateError('A private temporary output path is required');}- final count = int.parse(env['HARNESS_BENCH_TERMINALS'] ?? '16');- final observations = int.parse(env['HARNESS_BENCH_SAMPLES'] ?? '120');- if (![16, 48].contains(count) || observations < 1 || observations > 500) {+ var count = int.parse(env['HARNESS_BENCH_TERMINALS'] ?? '16');+ var observations = int.parse(env['HARNESS_BENCH_SAMPLES'] ?? '120');+ var hold = env['HARNESS_BENCH_HOLD'] == '1';+ if (_coreDriven) {+ final root = File(output).parent;+ final config = File('${root.path}/run-config.json');+ if (config.existsSync()) {+ final values =+ jsonDecode(await config.readAsString()) as Map<String, dynamic>;+ count = values['terminals'] as int;+ observations = values['samples'] as int;+ hold = values['hold'] as bool;+ final name = values['output'] as String;+ if (!RegExp(r'^[a-zA-Z0-9_-]+\.json$').hasMatch(name)) {+ throw StateError('Core output must be a simple JSON filename');+ }+ output = '${root.path}/$name';+ }+ if (File(output).existsSync()) {+ throw StateError('Use a fresh core output filename');+ }+ await File('${root.path}/active-run.json').writeAsString(+ jsonEncode({+ 'output': output,+ 'terminals': count,+ 'samples': observations,+ 'pid': pid,+ 'startedAt': DateTime.now().toUtc().toIso8601String(),+ }),+ );+ }+ if (![1, 16, 48].contains(count) || observations < 1 || observations > 500) {throw StateError('Invalid fixture size');}final binding = WidgetsFlutterBinding.ensureInitialized();@@ -298,6 +336,8 @@ Future<void> _run() async {if (metadata?['bundle'] != 'ai.autonomous.harness.benchmark') {throw StateError('Wrong native host');}+ metadata!['pid'] = pid;+ metadata['sourceRevision'] = env['HARNESS_BENCH_REVISION'] ?? 'unspecified';final app = await _fixture(count,Directory(@@ -324,9 +364,75 @@ Future<void> _run() async {await _frame();if (_interactive) return;await Future<void>.delayed(const Duration(seconds: 1));+ if (_coreDriven) {+ final packet = utf8.encode(+ '\x1b7\x1b[1;1H${List.filled(8, '${List.filled(110, 'o').join()}\x1b[K\r\n').join()}\x1b8',+ );+ Future<void> setOutput(bool active) async {+ _outputTimer?.cancel();+ _outputTimer = null;+ while (_burstPending) {+ await Future<void>.delayed(Duration.zero);+ }+ if (!active) return;+ _outputTimer = Timer.periodic(const Duration(milliseconds: 50), (+ _,+ ) async {+ if (_burstPending) {+ _skippedBursts++;+ return;+ }+ _burstPending = true;+ try {+ final sessions = app.allPanes+ .map((p) => p.session)+ .whereType<TerminalSession>()+ .toList();+ await Future.wait([+ for (final session in sessions) _output(session, packet),+ ]);+ _outputBytes += packet.length * sessions.length;+ } finally {+ _burstPending = false;+ }+ });+ }++ try {+ await runCoreBenchmark(+ app,+ output,+ metadata,+ _timings,+ samples: observations,+ inputs: _coreInputs,+ setOutput: setOutput,+ outputCounters: () => {+ 'outputBytes': _outputBytes,+ 'skippedBursts': _skippedBursts,+ },+ );+ } catch (error) {+ stderr.writeln('Core benchmark failed: $error');+ await File('$output.failure').writeAsString(+ jsonEncode({+ 'success': false,+ 'at': DateTime.now().toUtc().toIso8601String(),+ 'error': '$error',+ 'metadata': metadata,+ }),+ );+ } finally {+ await setOutput(false);+ if (!hold) {+ await _host.invokeMethod<void>('finish');+ }+ }+ return;+ }if (_flutterDriven) {try {- await runFlutterDispatchBenchmark(app, output, metadata!, _timings);+ await runFlutterDispatchBenchmark(app, output, metadata, _timings);} catch (error, stack) {await File(output).writeAsString(jsonEncode({'success': false, 'error': '$error', 'stack': '$stack'}),@@ -339,7 +445,7 @@ Future<void> _run() async {final random = Random(77);final phases = <Map<String, Object?>>[];try {- metadata!['initialResponder'] = await _host.invokeMethod<String>('begin');+ metadata['initialResponder'] = await _host.invokeMethod<String>('begin');for (final op in ['typing', 'focus', 'tab']) {await _sample(app, op, 'cold_interaction', 'idle');}diff --git a/desktop/tool/native_benchmark/prepare.py b/desktop/tool/native_benchmark/prepare.pyindex 3ea788b9c..0834c83c6 100644--- a/desktop/tool/native_benchmark/prepare.py+++ b/desktop/tool/native_benchmark/prepare.py@@ -1,6 +1,7 @@#!/usr/bin/env python3"""Build an isolated Release benchmark; never rewrite the user's V2 bundle."""import argparse+import jsonimport osfrom pathlib import Pathimport shutil@@ -18,7 +19,16 @@ mode.add_argument('--flutter-dispatch', action='store_true',help='Benchmark Flutter key dispatch and release raster, excluding OS input delivery')mode.add_argument('--primary-workflows', action='store_true',help='Benchmark Cmd+N/O/T and tab switching with varied input phase')+mode.add_argument('--core-workflows', action='store_true',+ help='Validate and measure typing, navigation, search, and scrolling at idle and under output')+parser.add_argument('--terminals', type=int, choices=[1, 16, 48], default=16)+parser.add_argument('--samples', type=int, default=120)+parser.add_argument('--hold', action='store_true', help='Keep core fixture open after measurements for process resource sampling')args = parser.parse_args()+if not 1 <= args.samples <= 500:+ parser.error('--samples must be 1–500')+if args.hold and not args.core_workflows:+ parser.error('--hold requires --core-workflows')source = Path(__file__).resolve().parents[2]root = Path(tempfile.mkdtemp(prefix='harness-native-benchmark-', dir='/private/tmp'))desktop = root / 'desktop'@@ -29,15 +39,20 @@ marker = ' super.awakeFromNib()'if code.count(marker) != 1:raise RuntimeError('Native benchmark insertion point changed')code = code.replace(marker, ' NativeBenchmark.install(window: self, messenger: flutterViewController.engine.binaryMessenger)\n' + marker)-if args.interactive or args.flutter_dispatch or args.primary_workflows:+if args.interactive or args.flutter_dispatch or args.primary_workflows or args.core_workflows:# Only this copied host gets fixture state. Launching through normal app# controls needs no shell environment and never opens a real transport.setup = '\n'.join(f' setenv("{key}", "{value}", 1)' for key, value in {'FLUTTER_TEST': '1','HARNESS_NATIVE_BENCHMARK': '1','HARNESS_BENCH_MANUAL': '1' if args.interactive else '0',- 'HARNESS_BENCH_FLUTTER': '0' if args.interactive else '1',+ 'HARNESS_BENCH_FLUTTER': '1' if args.flutter_dispatch or args.primary_workflows else '0','HARNESS_BENCH_PRIMARY': '1' if args.primary_workflows else '0',+ 'HARNESS_BENCH_CORE': '1' if args.core_workflows else '0',+ 'HARNESS_BENCH_TERMINALS': str(args.terminals),+ 'HARNESS_BENCH_SAMPLES': str(args.samples),+ 'HARNESS_BENCH_HOLD': '1' if args.hold else '0',+ 'HARNESS_BENCH_REVISION': subprocess.check_output(['git', 'rev-parse', 'HEAD'], cwd=source, text=True).strip(),'HARNESS_BENCH_OUTPUT': str(root / 'interactive.json'),}.items())engine = ' let flutterViewController = FlutterViewController()'@@ -46,6 +61,11 @@ if args.interactive or args.flutter_dispatch or args.primary_workflows:code = code.replace(engine, setup + '\n' + engine)code += '\n' + (source / 'tool/native_benchmark/NativeBenchmark.swift').read_text()window.write_text(code)+if args.core_workflows:+ (root / 'run-config.json').write_text(json.dumps({+ 'terminals': args.terminals, 'samples': args.samples,+ 'hold': args.hold, 'output': 'core-first.json',+ }, indent=2) + '\n')info = desktop / 'macos/Runner/Configs/AppInfo.xcconfig'info.write_text(benchmark_configuration(info.read_text()))env = dict(os.environ, XDG_CONFIG_HOME=str(root / 'tool-config'))diff --git a/desktop/tool/native_benchmark/process_usage.swift b/desktop/tool/native_benchmark/process_usage.swiftnew file mode 100644index 000000000..857bdad0f--- /dev/null+++ b/desktop/tool/native_benchmark/process_usage.swift@@ -0,0 +1,94 @@+// Standalone macOS process accounting. Does not launch, focus, or control apps.+// Build: xcrun swiftc process_usage.swift -o /private/tmp/harness-process-usage+// Run: /private/tmp/harness-process-usage PID SECONDS LABEL OUTPUT.json+import Foundation+import Darwin++func fail(_ message: String) -> Never {+ FileHandle.standardError.write(Data((message + "\n").utf8))+ exit(1)+}++let args = CommandLine.arguments+guard args.count == 5, let target = Int32(args[1]), target > 1,+ let seconds = Int(args[2]), (5...1800).contains(seconds) else {+ fail("Usage: process_usage PID SECONDS(5–1800) LABEL OUTPUT.json")+}+guard !FileManager.default.fileExists(atPath: args[4]) else {+ fail("Output already exists; preserve previous observations")+}++struct Sample: Codable {+ let elapsedSeconds: Double+ let userNanoseconds: UInt64+ let systemNanoseconds: UInt64+ let physicalFootprintBytes: UInt64+ let residentBytes: UInt64+ let interruptWakeups: UInt64+ let packageIdleWakeups: UInt64+ let diskReadBytes: UInt64+ let diskWriteBytes: UInt64+}++let began = ProcessInfo.processInfo.systemUptime+let startedAt = ISO8601DateFormatter().string(from: Date())+func sample() -> Sample {+ var usage = rusage_info_v4()+ let status = withUnsafeMutablePointer(to: &usage) { pointer in+ pointer.withMemoryRebound(to: rusage_info_t?.self, capacity: 1) {+ proc_pid_rusage(target, RUSAGE_INFO_V4, $0)+ }+ }+ guard status == 0 else { fail("proc_pid_rusage failed (errno \(errno)); process may have exited") }+ return Sample(+ elapsedSeconds: ProcessInfo.processInfo.systemUptime - began,+ userNanoseconds: usage.ri_user_time,+ systemNanoseconds: usage.ri_system_time,+ physicalFootprintBytes: usage.ri_phys_footprint,+ residentBytes: usage.ri_resident_size,+ interruptWakeups: usage.ri_interrupt_wkups,+ packageIdleWakeups: usage.ri_pkg_idle_wkups,+ diskReadBytes: usage.ri_diskio_bytesread,+ diskWriteBytes: usage.ri_diskio_byteswritten)+}++var samples = [sample()]+for tick in 1...seconds {+ let remaining = began + Double(tick) - ProcessInfo.processInfo.systemUptime+ if remaining > 0 { Thread.sleep(forTimeInterval: remaining) }+ samples.append(sample())+}+let first = samples.first!+let last = samples.last!+let elapsed = last.elapsedSeconds - first.elapsedSeconds+let cpuSeconds = Double(last.userNanoseconds - first.userNanoseconds+ + last.systemNanoseconds - first.systemNanoseconds) / 1_000_000_000+let footprints = samples.map { Double($0.physicalFootprintBytes) / 1_048_576 }.sorted()+let summary: [String: Any] = [+ "elapsedSeconds": elapsed,+ "cpuPercentOneCore": cpuSeconds / elapsed * 100,+ "cpuSeconds": cpuSeconds,+ "physicalFootprintMiBMedian": footprints[footprints.count / 2],+ "physicalFootprintMiBPeak": footprints.last!,+ "physicalFootprintMiBStart": Double(first.physicalFootprintBytes) / 1_048_576,+ "physicalFootprintMiBEnd": Double(last.physicalFootprintBytes) / 1_048_576,+ "interruptWakeupsPerSecond": Double(last.interruptWakeups - first.interruptWakeups) / elapsed,+ "packageIdleWakeupsPerSecond": Double(last.packageIdleWakeups - first.packageIdleWakeups) / elapsed,+ "diskReadBytes": last.diskReadBytes - first.diskReadBytes,+ "diskWriteBytes": last.diskWriteBytes - first.diskWriteBytes,+]+let encodedSamples = try JSONEncoder().encode(samples)+let output: [String: Any] = [+ "schema": 1, "success": true, "label": args[3], "pid": target,+ "startedAt": startedAt, "os": ProcessInfo.processInfo.operatingSystemVersionString,+ "boundary": "proc_pid_rusage for this process only; CPU 100% = one core; excludes daemon, agents, and GPU energy",+ "sampleIntervalSeconds": 1, "summary": summary,+ "samples": try JSONSerialization.jsonObject(with: encodedSamples),+]+let data = try JSONSerialization.data(withJSONObject: output, options: [.prettyPrinted, .sortedKeys])+guard FileManager.default.createFile(atPath: args[4], contents: data,+ attributes: [.posixPermissions: 0o600]) else {+ fail("Could not write result")+}+let summaryData = try JSONSerialization.data(withJSONObject: summary, options: [.sortedKeys])+print(String(data: summaryData, encoding: .utf8)!)