Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
Add the macOS helper that reads Messages for gbrain (#9620)
Garry Tanf224c0ec1e45
diff --git a/.github/workflows/apple-helper.yml b/.github/workflows/apple-helper.ymlnew file mode 100644index 0000000000..09bc4fb0a0--- /dev/null+++ b/.github/workflows/apple-helper.yml@@ -0,0 +1,53 @@+name: Apple helper++# Builds and tests the macOS helper under apple/helper/ (Swift), then runs the+# headless signing probe (apple/helper/scripts/signing-probe.sh), whose answer+# lands in the job summary. Not a required check, and kept out of test.yml so+# full-suite reuse is unchanged for every other change. Uses no secrets.++on:+ pull_request:+ branches: [master]+ # Event parity (docs/ci-event-parity.md): path-gated named exception; covered by the master push run on the same paths.+ paths:+ - 'apple/**'+ - '.github/workflows/apple-helper.yml'+ push:+ branches: [master]+ paths:+ - 'apple/**'+ - '.github/workflows/apple-helper.yml'+ workflow_dispatch:++permissions:+ contents: read++concurrency:+ group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}+ cancel-in-progress: true++jobs:+ helper:+ name: Apple helper (macOS 26)+ runs-on: macos-26+ timeout-minutes: 30+ defaults:+ run:+ working-directory: apple/helper+ steps:+ - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4+ - name: Toolchain+ run: |+ sw_vers+ xcode-select -p+ swift --version+ - name: Bundle property lists parse+ run: plutil -lint Bundle/Info.plist Bundle/io.gbrain.apple-helper.plist Bundle/gbrain-apple-helper.entitlements+ - name: Build+ run: swift build+ - name: Test+ run: swift test+ - name: Signing probe+ timeout-minutes: 15+ working-directory: .+ run: bash apple/helper/scripts/signing-probe.shdiff --git a/apple/helper/.gitignore b/apple/helper/.gitignorenew file mode 100644index 0000000000..2d9f16e2d2--- /dev/null+++ b/apple/helper/.gitignore@@ -0,0 +1,2 @@+.build/+.swiftpm/diff --git a/apple/helper/Bundle/Info.plist b/apple/helper/Bundle/Info.plistnew file mode 100644index 0000000000..6abe4bdc6f--- /dev/null+++ b/apple/helper/Bundle/Info.plist@@ -0,0 +1,30 @@+<?xml version="1.0" encoding="UTF-8"?>+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">+<plist version="1.0">+<dict>+ <key>CFBundleDevelopmentRegion</key>+ <string>en</string>+ <key>CFBundleDisplayName</key>+ <string>GBrain Helper</string>+ <key>CFBundleExecutable</key>+ <string>gbrain-apple-helper</string>+ <key>CFBundleIdentifier</key>+ <string>io.gbrain.apple-helper</string>+ <key>CFBundleInfoDictionaryVersion</key>+ <string>6.0</string>+ <key>CFBundleName</key>+ <string>GBrain Helper</string>+ <key>CFBundlePackageType</key>+ <string>APPL</string>+ <key>CFBundleShortVersionString</key>+ <string>0.1.0</string>+ <key>CFBundleVersion</key>+ <string>1</string>+ <key>LSMinimumSystemVersion</key>+ <string>13.0</string>+ <key>LSUIElement</key>+ <true/>+ <key>NSAppleEventsUsageDescription</key>+ <string>GBrain Helper asks to control Messages only when you turn on sending from gbrain.</string>+</dict>+</plist>diff --git a/apple/helper/Bundle/gbrain-apple-helper.entitlements b/apple/helper/Bundle/gbrain-apple-helper.entitlementsnew file mode 100644index 0000000000..cffd6379ce--- /dev/null+++ b/apple/helper/Bundle/gbrain-apple-helper.entitlements@@ -0,0 +1,8 @@+<?xml version="1.0" encoding="UTF-8"?>+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">+<plist version="1.0">+<dict>+ <key>com.apple.security.automation.apple-events</key>+ <true/>+</dict>+</plist>diff --git a/apple/helper/Bundle/io.gbrain.apple-helper.plist b/apple/helper/Bundle/io.gbrain.apple-helper.plistnew file mode 100644index 0000000000..5202e14323--- /dev/null+++ b/apple/helper/Bundle/io.gbrain.apple-helper.plist@@ -0,0 +1,23 @@+<?xml version="1.0" encoding="UTF-8"?>+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">+<plist version="1.0">+<dict>+ <key>Label</key>+ <string>io.gbrain.apple-helper</string>+ <key>BundleProgram</key>+ <string>Contents/MacOS/gbrain-apple-helper</string>+ <key>ProgramArguments</key>+ <array>+ <string>gbrain-apple-helper</string>+ <string>serve</string>+ </array>+ <key>RunAtLoad</key>+ <true/>+ <key>KeepAlive</key>+ <true/>+ <key>ThrottleInterval</key>+ <integer>10</integer>+ <key>ProcessType</key>+ <string>Background</string>+</dict>+</plist>diff --git a/apple/helper/Package.swift b/apple/helper/Package.swiftnew file mode 100644index 0000000000..2f2deda309--- /dev/null+++ b/apple/helper/Package.swift@@ -0,0 +1,13 @@+// swift-tools-version:5.10+import PackageDescription++let package = Package(+ name: "gbrain-apple-helper",+ platforms: [.macOS(.v13)],+ products: [.executable(name: "gbrain-apple-helper", targets: ["gbrain-apple-helper"])],+ targets: [+ .target(name: "HelperCore", linkerSettings: [.linkedLibrary("sqlite3")]),+ .executableTarget(name: "gbrain-apple-helper", dependencies: ["HelperCore"]),+ .testTarget(name: "HelperCoreTests", dependencies: ["HelperCore"]),+ ]+)diff --git a/apple/helper/README.md b/apple/helper/README.mdnew file mode 100644index 0000000000..1c6161fd4a--- /dev/null+++ b/apple/helper/README.md@@ -0,0 +1,140 @@+# GBrain Helper (macOS)++A small macOS program that reads the Messages database on your own Mac and answers gbrain over `127.0.0.1`. It never writes to the database, never sends a message and never shows a macOS dialog while it answers a request.++This folder is Swift source only. On Linux and Windows nothing in gbrain's install, upgrade, doctor or test suite builds or loads it. A later `gbrain apple-setup` builds it on your Mac; until then, `.github/workflows/apple-helper.yml` builds and tests it on a macOS runner.++## Build and test on a Mac++```bash+cd apple/helper+swift build+swift test+```++The helper runs on macOS 13 or newer. Building it needs the Xcode Command Line Tools (`xcode-select --install`), and `swift test` needs Swift 6 (Xcode 16 or its Command Line Tools, macOS 14.5 or newer) because the tests use Swift Testing.++## Layout++- `Sources/HelperCore/`: the Messages reader, the HTTP server, tokens, config and source modules.+- `Sources/gbrain-apple-helper/`: the command-line program.+- `Bundle/`: `Info.plist`, the background agent's property list and the entitlements for `GBrain Helper.app`.+- `scripts/signing-probe.sh`: CI probe that asks whether a self-signed certificate made on the machine signs the app with no dialog, or first needs a one-time Always Trust step. Its answer goes to the job summary.+- `Tests/HelperCoreTests/`: tests against a synthetic Messages database.++## Routes++| Route | Token scope | Returns |+| --- | --- | --- |+| `GET /health` | none | `ok` and `version`, nothing else |+| `GET /v1/sources` | any token | each source: on or off, the permission it needs, its state, scope and routes; no messages |+| `GET /v1/imessage/threads` | `read:imessage` | recent threads; `limit` (1 to 200, default 30), `include_last_message` (default true) |+| `GET /v1/imessage/thread` | `read:imessage` | one thread's messages; one of `thread_id`, `chat_rowid` or `chat_guid`, plus `limit` (1 to 500, default 50), `since_rowid`, `since_iso`, `include_reactions` (default false) |++Without a cursor a thread page holds the newest messages, oldest first. With `since_rowid` it holds the messages after that row, in row order; `meta.next_since_rowid` is the cursor for the next page.++## Safety++- Listens on `127.0.0.1` only. It refuses a peer that isn't loopback, a `Host` header other than `127.0.0.1:<port>` and any request with an `Origin` header, and it sends no CORS headers, so a web page can't reach it through DNS rebinding.+- Opens the database read-only (`mode=ro` and `query_only`).+- The config stores a SHA-256 verifier and the scopes for each token, never the token itself. A presented token is checked against every verifier in constant time.+- Requests are limited to 8 KB, 4 at a time, with a 10-second read timeout.+- The log, `~/Library/Logs/gbrain-apple-helper.log`, records route, status, duration, row count and request id only.+- A source module checks its permission without prompting. A missing grant returns a problem response, never a dialog. iMessage needs Full Disk Access, which macOS never asks for with a dialog.+- A listener for allowlisted peers on the Mac's tailnet address (100.64.0.0/10 on a `utun` interface, never loopback or the Mac's own addresses) is built and tested, but no config field turns it on yet.++## Config++`~/Library/Application Support/GBrain Helper/config.json`, in a 0700 folder, mode 0600:++```json+{+ "port": 8792,+ "sources": { "imessage": { "enabled": true } },+ "tokens": [{ "name": "gbrain", "scopes": ["read:imessage"], "verifier": "<sha256 of the token, hex>" }]+}+```++`db_path` overrides the Messages database path (default `~/Library/Messages/chat.db`). Sources are off unless turned on.++## Commands++```text+gbrain-apple-helper init --port <n> [--enable imessage] [--db <path>]+gbrain-apple-helper token add --name <name> --verifier <sha256 hex> [--scope read:imessage]+gbrain-apple-helper token remove --name <name>+gbrain-apple-helper validate-config+gbrain-apple-helper check [--db <path>]+gbrain-apple-helper serve+gbrain-apple-helper --version+```++Every command takes `--config <path>`. `token add` takes the token's SHA-256, never the token. A running `serve` picks up `token add` and `token remove` on its next request, and refuses every token while the config can't be read. `check` prints counts only.++## Errors++Errors are RFC 9457 problem responses (`application/problem+json`) with `code`, `detail`, `retryable`, `fix` when there is one, and `request_id`. Each `type` links to its entry here.++### bad_param++A query parameter is missing or out of range. The detail names it. A repeated parameter keeps its first value. A request that isn't valid HTTP gets this code too.++### unauthorized++No bearer token, or one the helper doesn't know.++### insufficient_scope++The token is valid but lacks the route's scope, such as `read:imessage`.++### source_disabled++The source is off in the helper's config. Set `sources.<source>.enabled` to `true` in the config, then restart `serve`. The response's `fix` names `gbrain apple enable <source>`, a gbrain command that a later release adds.++### forbidden_source++The caller's address isn't allowed on this listener. The loopback listener accepts only this Mac.++### forbidden_host++The `Host` header isn't `127.0.0.1:<port>`. Names such as `localhost` are refused on purpose.++### forbidden_origin++The request carried an `Origin` header, so it came from a web page.++### not_found++No route, or no chat matching the selector. The handle you asked for is never echoed back.++### method_not_allowed++Only `GET` is supported.++### too_large++The request was over 8 KB.++### overloaded++Four requests were already running. Retry after the `Retry-After` seconds.++### no_access++The helper has no Full Disk Access. Turn on GBrain Helper in System Settings > Privacy & Security > Full Disk Access.++### db_missing++There is no Messages database at the configured path.++### db_busy++The database stayed locked for 2 seconds. Retry after the `Retry-After` seconds.++### schema_mismatch++macOS changed the Messages database layout. Rebuild the helper from a newer gbrain.++### internal++An unexpected error. The request id is in the helper's log.diff --git a/apple/helper/Sources/HelperCore/AppleTime.swift b/apple/helper/Sources/HelperCore/AppleTime.swiftnew file mode 100644index 0000000000..86a78485a7--- /dev/null+++ b/apple/helper/Sources/HelperCore/AppleTime.swift@@ -0,0 +1,38 @@+import Foundation++public enum AppleTime {+ public static let referenceOffset: Int64 = 978_307_200+ static let secondsThreshold: Int64 = 100_000_000_000++ public static func nanoseconds(fromRaw raw: Int64) -> Int64 {+ guard raw > -secondsThreshold, raw < secondsThreshold else { return raw }+ let (nanoseconds, overflow) = raw.multipliedReportingOverflow(by: 1_000_000_000)+ return overflow ? (raw < 0 ? .min : .max) : nanoseconds+ }++ public static func date(fromRaw raw: Int64) -> Date {+ Date(timeIntervalSince1970: Double(referenceOffset) + Double(nanoseconds(fromRaw: raw)) / 1e9)+ }++ public static func raw(from date: Date) -> Int64 {+ let nanoseconds = ((date.timeIntervalSince1970 - Double(referenceOffset)) * 1e9).rounded()+ return Int64(exactly: nanoseconds) ?? (nanoseconds < 0 ? .min : .max)+ }++ public static func iso(fromRaw raw: Int64) -> String {+ iso(date(fromRaw: raw))+ }++ public static func iso(_ date: Date) -> String {+ date.formatted(Date.ISO8601FormatStyle(includingFractionalSeconds: true))+ }++ public static func parseISO(_ string: String) -> Date? {+ if let date = try? Date.ISO8601FormatStyle(includingFractionalSeconds: true).parse(string) { return date }+ return try? Date.ISO8601FormatStyle().parse(string)+ }++ static func normalizedSQL(_ column: String) -> String {+ "(CASE WHEN ABS(\(column)) < \(secondsThreshold) THEN \(column) * 1000000000 ELSE \(column) END)"+ }+}diff --git a/apple/helper/Sources/HelperCore/ChatDB.swift b/apple/helper/Sources/HelperCore/ChatDB.swiftnew file mode 100644index 0000000000..1864538a2f--- /dev/null+++ b/apple/helper/Sources/HelperCore/ChatDB.swift@@ -0,0 +1,148 @@+import Darwin+import Foundation+import SQLite3++public enum DBError: Error, Equatable {+ case missing+ case noAccess+ case busy+ case schemaMismatch(String)+ case sqlite(Int32, String)+}++public enum SQLValue {+ case int(Int64)+ case text(String)+}++public struct Row {+ let statement: OpaquePointer++ public func int(_ index: Int32) -> Int64 { sqlite3_column_int64(statement, index) }++ public func text(_ index: Int32) -> String? {+ guard sqlite3_column_type(statement, index) != SQLITE_NULL, let pointer = sqlite3_column_text(statement, index) else { return nil }+ let count = Int(sqlite3_column_bytes(statement, index))+ return String(decoding: UnsafeBufferPointer(start: pointer, count: count), as: UTF8.self)+ }++ public func blob(_ index: Int32) -> [UInt8]? {+ guard sqlite3_column_type(statement, index) != SQLITE_NULL else { return nil }+ let count = Int(sqlite3_column_bytes(statement, index))+ guard count > 0, let pointer = sqlite3_column_blob(statement, index) else { return [] }+ return Array(UnsafeRawBufferPointer(start: pointer, count: count))+ }+}++public final class ChatDB {+ static let required: [String: [String]] = [+ "message": ["ROWID", "guid", "text", "date", "is_from_me", "handle_id"],+ "chat": ["ROWID", "guid", "chat_identifier"],+ "handle": ["ROWID", "id"],+ "chat_message_join": ["chat_id", "message_id"],+ "chat_handle_join": ["chat_id", "handle_id"],+ ]+ static let transient = unsafeBitCast(-1, to: sqlite3_destructor_type.self)++ let handle: OpaquePointer+ public private(set) var columns: [String: Set<String>] = [:]++ public static func open(path: String) throws -> ChatDB {+ let fd = Darwin.open(path, O_RDONLY)+ if fd < 0 {+ switch errno {+ case ENOENT, ENOTDIR: throw DBError.missing+ case EPERM, EACCES: throw DBError.noAccess+ default: throw DBError.sqlite(SQLITE_CANTOPEN, String(cString: strerror(errno)))+ }+ }+ close(fd)+ let encoded = path.addingPercentEncoding(withAllowedCharacters: .urlPathAllowed) ?? path+ var pointer: OpaquePointer?+ let rc = sqlite3_open_v2("file:\(encoded)?mode=ro", &pointer, SQLITE_OPEN_READONLY | SQLITE_OPEN_URI | SQLITE_OPEN_NOMUTEX, nil)+ guard rc == SQLITE_OK, let db = pointer else {+ if let pointer { sqlite3_close_v2(pointer) }+ throw rc == SQLITE_CANTOPEN || rc == SQLITE_PERM || rc == SQLITE_AUTH ? DBError.noAccess : DBError.sqlite(rc, "open failed")+ }+ let chat = ChatDB(handle: db)+ sqlite3_busy_timeout(db, 2000)+ try chat.query("PRAGMA query_only = 1", []) { _ in }+ try chat.loadColumns()+ return chat+ }++ init(handle: OpaquePointer) {+ self.handle = handle+ }++ deinit {+ sqlite3_close_v2(handle)+ }++ func loadColumns() throws {+ for table in Self.required.keys {+ var names = Set<String>()+ try query("PRAGMA table_info(\(table))", []) { row in+ if let name = row.text(1) { names.insert(name) }+ }+ columns[table] = names+ }+ }++ public func validateSchema() throws {+ for (table, needed) in Self.required.sorted(by: { $0.key < $1.key }) {+ let have = columns[table] ?? []+ if have.isEmpty { throw DBError.schemaMismatch("table \(table) is missing") }+ for column in needed where column != "ROWID" && !have.contains(column) {+ throw DBError.schemaMismatch("column \(table).\(column) is missing")+ }+ }+ }++ public func has(_ table: String, _ column: String) -> Bool {+ columns[table]?.contains(column) ?? false+ }++ func prepare(_ sql: String) throws -> OpaquePointer {+ var statement: OpaquePointer?+ let rc = sqlite3_prepare_v2(handle, sql, -1, &statement, nil)+ guard rc == SQLITE_OK, let statement else {+ if let statement { sqlite3_finalize(statement) }+ throw error(rc)+ }+ return statement+ }++ func error(_ rc: Int32) -> DBError {+ let message = String(cString: sqlite3_errmsg(handle))+ if rc == SQLITE_BUSY || rc == SQLITE_LOCKED { return .busy }+ if message.contains("no such") { return .schemaMismatch(message) }+ if rc == SQLITE_CANTOPEN || rc == SQLITE_AUTH || rc == SQLITE_PERM { return .noAccess }+ return .sqlite(rc, message)+ }++ public func check(_ sql: String) throws {+ sqlite3_finalize(try prepare(sql))+ }++ public func query(_ sql: String, _ binds: [SQLValue], _ each: (Row) throws -> Void) throws {+ let statement = try prepare(sql)+ defer { sqlite3_finalize(statement) }+ for (offset, value) in binds.enumerated() {+ let index = Int32(offset + 1)+ switch value {+ case .int(let number): sqlite3_bind_int64(statement, index, number)+ case .text(let string): sqlite3_bind_text(statement, index, string, -1, Self.transient)+ }+ }+ while true {+ let rc = sqlite3_step(statement)+ if rc == SQLITE_ROW {+ try each(Row(statement: statement))+ continue+ }+ if rc == SQLITE_DONE { return }+ throw error(rc)+ }+ }+}diff --git a/apple/helper/Sources/HelperCore/Config.swift b/apple/helper/Sources/HelperCore/Config.swiftnew file mode 100644index 0000000000..cf8d7ad775--- /dev/null+++ b/apple/helper/Sources/HelperCore/Config.swift@@ -0,0 +1,162 @@+import Darwin+import Foundation++public enum HelperIdentity {+ public static let appName = "GBrain Helper"+ public static let bundleId = "io.gbrain.apple-helper"+ public static let executable = "gbrain-apple-helper"+}++public struct ConfigError: Error, CustomStringConvertible {+ public let description: String+}++public struct TokenEntry: Codable, Equatable {+ public var name: String+ public var verifier: String+ public var scopes: [String]++ public init(name: String, verifier: String, scopes: [String]) {+ self.name = name+ self.verifier = verifier+ self.scopes = scopes+ }+}++public struct SourceSetting: Codable, Equatable {+ public var enabled: Bool++ public init(enabled: Bool) {+ self.enabled = enabled+ }+}++public struct Config: Codable, Equatable {+ public var port: Int+ public var tokens: [TokenEntry]+ public var sources: [String: SourceSetting]+ public var dbPath: String?++ enum CodingKeys: String, CodingKey {+ case port, tokens, sources+ case dbPath = "db_path"+ }++ public init(port: Int, tokens: [TokenEntry] = [], sources: [String: SourceSetting] = [:], dbPath: String? = nil) {+ self.port = port+ self.tokens = tokens+ self.sources = sources+ self.dbPath = dbPath+ }++ public static var defaultDirectory: String { NSHomeDirectory() + "/Library/Application Support/\(HelperIdentity.appName)" }+ public static var defaultPath: String { defaultDirectory + "/config.json" }+ public static var defaultDBPath: String { NSHomeDirectory() + "/Library/Messages/chat.db" }++ public static func load(path: String) throws -> Config {+ guard let data = FileManager.default.contents(atPath: path) else {+ throw ConfigError(description: "no config at \(path)")+ }+ do {+ return try JSONDecoder().decode(Config.self, from: data)+ } catch {+ throw ConfigError(description: "config at \(path) is not valid JSON for \(HelperIdentity.executable)")+ }+ }++ public func validated() throws -> ValidatedConfig {+ guard (1024...65535).contains(port) else { throw ConfigError(description: "port must be 1024-65535") }+ var grants: [TokenGrant] = []+ var names = Set<String>()+ for entry in tokens {+ guard !entry.name.isEmpty, names.insert(entry.name).inserted else {+ throw ConfigError(description: "every token needs a unique, non-empty name")+ }+ guard let verifier = Token.verifier(fromHex: entry.verifier) else {+ throw ConfigError(description: "token \(entry.name): verifier must be the token's sha256 as 64 hex characters")+ }+ for scope in entry.scopes where !SourceCatalog.scopes.contains(scope) {+ throw ConfigError(description: "token \(entry.name): unknown scope \(scope)")+ }+ grants.append(TokenGrant(name: entry.name, verifier: verifier, scopes: Set(entry.scopes)))+ }+ for id in sources.keys where !SourceCatalog.ids.contains(id) {+ throw ConfigError(description: "unknown source \(id)")+ }+ let enabled = Set(sources.filter { $0.value.enabled }.map(\.key))+ return ValidatedConfig(port: UInt16(port), grants: grants, enabled: enabled, dbPath: dbPath ?? Config.defaultDBPath)+ }++ public func write(path: String) throws {+ let directory = (path as NSString).deletingLastPathComponent+ try FileManager.default.createDirectory(atPath: directory, withIntermediateDirectories: true, attributes: [.posixPermissions: 0o700])+ try FileManager.default.setAttributes([.posixPermissions: 0o700], ofItemAtPath: directory)+ let encoder = JSONEncoder()+ encoder.outputFormatting = [.prettyPrinted, .sortedKeys, .withoutEscapingSlashes]+ let data = try encoder.encode(self)+ let temp = path + ".tmp-\(getpid())-\(UUID().uuidString)"+ let fd = Darwin.open(temp, O_WRONLY | O_CREAT | O_EXCL, 0o600)+ guard fd >= 0 else { throw ConfigError(description: "could not write \(path)") }+ let written = data.withUnsafeBytes { Darwin.write(fd, $0.baseAddress, $0.count) }+ let synced = fsync(fd) == 0+ close(fd)+ guard written == data.count, synced, rename(temp, path) == 0 else {+ unlink(temp)+ throw ConfigError(description: "could not write \(path)")+ }+ }+}++public final class LiveGrants {+ let path: String+ let lock = NSLock()+ var loaded = false+ var stamp: FileStamp?+ var grants: [TokenGrant] = []++ public init(path: String) {+ self.path = path+ }++ public func current() -> [TokenGrant] {+ lock.lock()+ defer { lock.unlock() }+ let now = FileStamp(path: path)+ if loaded && now == stamp { return grants }+ loaded = true+ stamp = now+ do {+ grants = try Config.load(path: path).validated().grants+ Log.line("config loaded: \(grants.count) token(s)")+ } catch {+ grants = []+ Log.line("config unreadable, refusing every token until it is fixed: \(error)")+ }+ return grants+ }+}++struct FileStamp: Equatable {+ let device: Int64+ let inode: UInt64+ let size: Int64+ let seconds: Int+ let nanoseconds: Int++ init?(path: String) {+ var info = stat()+ guard stat(path, &info) == 0 else { return nil }+ device = Int64(info.st_dev)+ inode = UInt64(info.st_ino)+ size = Int64(info.st_size)+ seconds = Int(info.st_mtimespec.tv_sec)+ nanoseconds = Int(info.st_mtimespec.tv_nsec)+ }+}++public struct ValidatedConfig {+ public let port: UInt16+ public let grants: [TokenGrant]+ public let enabled: Set<String>+ public let dbPath: String+}diff --git a/apple/helper/Sources/HelperCore/HTTP.swift b/apple/helper/Sources/HelperCore/HTTP.swiftnew file mode 100644index 0000000000..6c820964ac--- /dev/null+++ b/apple/helper/Sources/HelperCore/HTTP.swift@@ -0,0 +1,102 @@+import Foundation++public struct HTTPRequest {+ public let method: String+ public let path: String+ public let query: [String: String]+ public let headers: [String: String]+}++public enum HTTPParse {+ case incomplete+ case tooLarge+ case bad+ case request(HTTPRequest)+}++public struct HTTPResponse {+ public var status: Int+ public var contentType: String+ public var extraHeaders: [(String, String)]+ public var body: [UInt8]++ public static func json<T: Encodable>(_ status: Int, _ value: T) -> HTTPResponse {+ HTTPResponse(status: status, contentType: "application/json", extraHeaders: [], body: encode(value))+ }++ public static func problem(_ problem: Problem, requestId: String) -> HTTPResponse {+ let body = ProblemBody(type: Problem.docBase + problem.code, title: problem.title, status: problem.status, code: problem.code,+ detail: problem.detail, retryable: problem.retryable, fix: problem.fix, requestId: requestId)+ var headers: [(String, String)] = []+ if let retryAfter = problem.retryAfter { headers.append(("Retry-After", String(retryAfter))) }+ return HTTPResponse(status: problem.status, contentType: "application/problem+json", extraHeaders: headers, body: encode(body))+ }++ static func encode<T: Encodable>(_ value: T) -> [UInt8] {+ let encoder = JSONEncoder()+ encoder.keyEncodingStrategy = .convertToSnakeCase+ encoder.outputFormatting = [.sortedKeys, .withoutEscapingSlashes]+ return (try? Array(encoder.encode(value))) ?? Array("{}".utf8)+ }++ public func serialize(requestId: String) -> [UInt8] {+ var head = "HTTP/1.1 \(status) \(Self.reason(status))\r\n"+ head += "Content-Type: \(contentType)\r\n"+ head += "Content-Length: \(body.count)\r\n"+ head += "Cache-Control: no-store\r\n"+ head += "Connection: close\r\n"+ head += "X-Request-Id: \(requestId)\r\n"+ for (name, value) in extraHeaders { head += "\(name): \(value)\r\n" }+ head += "\r\n"+ return Array(head.utf8) + body+ }++ static func reason(_ status: Int) -> String {+ switch status {+ case 200: return "OK"+ case 400: return "Bad Request"+ case 401: return "Unauthorized"+ case 403: return "Forbidden"+ case 404: return "Not Found"+ case 405: return "Method Not Allowed"+ case 408: return "Request Timeout"+ case 413: return "Content Too Large"+ case 500: return "Internal Server Error"+ case 503: return "Service Unavailable"+ default: return "Status"+ }+ }+}++public enum HTTPParser {+ static let terminator: [UInt8] = [13, 10, 13, 10]++ public static func parse(_ buffer: [UInt8], maxBytes: Int) -> HTTPParse {+ guard let end = TypedStream.find(terminator, in: buffer, from: 0) else {+ return buffer.count >= maxBytes ? .tooLarge : .incomplete+ }+ guard end + terminator.count <= maxBytes else { return .tooLarge }+ guard let head = String(bytes: buffer[0..<end], encoding: .utf8) else { return .bad }+ let lines = head.components(separatedBy: "\r\n")+ let requestLine = lines[0].split(separator: " ")+ guard requestLine.count == 3, requestLine[2].hasPrefix("HTTP/1.") else { return .bad }+ var headers: [String: String] = [:]+ for line in lines.dropFirst() {+ guard let colon = line.firstIndex(of: ":") else { return .bad }+ headers[line[..<colon].lowercased()] = line[line.index(after: colon)...].trimmingCharacters(in: .whitespaces)+ }+ let target = String(requestLine[1])+ let parts = target.split(separator: "?", maxSplits: 1, omittingEmptySubsequences: false)+ guard let path = parts.first.map(String.init), path.hasPrefix("/") else { return .bad }+ var query: [String: String] = [:]+ if parts.count == 2 {+ for pair in parts[1].split(separator: "&") {+ let kv = pair.split(separator: "=", maxSplits: 1, omittingEmptySubsequences: false)+ guard let key = String(kv[0]).removingPercentEncoding,+ let value = (kv.count == 2 ? String(kv[1]) : "").removingPercentEncoding else { return .bad }+ if query[key] == nil { query[key] = value }+ }+ }+ return .request(HTTPRequest(method: String(requestLine[0]), path: path, query: query, headers: headers))+ }+}diff --git a/apple/helper/Sources/HelperCore/Log.swift b/apple/helper/Sources/HelperCore/Log.swiftnew file mode 100644index 0000000000..ebd0a82862--- /dev/null+++ b/apple/helper/Sources/HelperCore/Log.swift@@ -0,0 +1,28 @@+import Foundation++public enum Log {+ static let lock = NSLock()+ static var sink: FileHandle?++ public static var defaultPath: String { NSHomeDirectory() + "/Library/Logs/\(HelperIdentity.executable).log" }++ public static func open(path: String) throws {+ let directory = (path as NSString).deletingLastPathComponent+ try FileManager.default.createDirectory(atPath: directory, withIntermediateDirectories: true)+ if !FileManager.default.fileExists(atPath: path) {+ FileManager.default.createFile(atPath: path, contents: nil, attributes: [.posixPermissions: 0o600])+ }+ guard let handle = FileHandle(forWritingAtPath: path) else { throw ConfigError(description: "could not open the log at \(path)") }+ handle.seekToEndOfFile()+ lock.lock()+ sink = handle+ lock.unlock()+ }++ public static func line(_ message: String) {+ let stamp = Date().formatted(Date.ISO8601FormatStyle(includingFractionalSeconds: true))+ lock.lock()+ defer { lock.unlock() }+ (sink ?? FileHandle.standardError).write(Data("\(stamp) \(message)\n".utf8))+ }+}diff --git a/apple/helper/Sources/HelperCore/Net.swift b/apple/helper/Sources/HelperCore/Net.swiftnew file mode 100644index 0000000000..032275d100--- /dev/null+++ b/apple/helper/Sources/HelperCore/Net.swift@@ -0,0 +1,49 @@+import Darwin+import Foundation++public enum IPv4 {+ public static func parse(_ string: String) -> UInt32? {+ var address = in_addr()+ guard string.withCString({ inet_pton(AF_INET, $0, &address) }) == 1 else { return nil }+ return UInt32(bigEndian: address.s_addr)+ }++ public static func string(_ ip: UInt32) -> String {+ "\(ip >> 24 & 0xFF).\(ip >> 16 & 0xFF).\(ip >> 8 & 0xFF).\(ip & 0xFF)"+ }++ public static func isTailnet(_ ip: UInt32) -> Bool {+ ip & 0xFFC0_0000 == 0x6440_0000+ }++ public static func isLoopback(_ ip: UInt32) -> Bool {+ ip >> 24 == 127+ }+}++public enum Interfaces {+ public static func ipv4() -> [UInt32: String] {+ var result: [UInt32: String] = [:]+ var head: UnsafeMutablePointer<ifaddrs>?+ guard getifaddrs(&head) == 0 else { return result }+ defer { freeifaddrs(head) }+ var cursor = head+ while let entry = cursor {+ if let address = entry.pointee.ifa_addr, address.pointee.sa_family == sa_family_t(AF_INET) {+ let ip = address.withMemoryRebound(to: sockaddr_in.self, capacity: 1) { UInt32(bigEndian: $0.pointee.sin_addr.s_addr) }+ result[ip] = String(cString: entry.pointee.ifa_name)+ }+ cursor = entry.pointee.ifa_next+ }+ return result+ }+}++public enum BindPolicy {+ public static func problem(host: UInt32, interfaces: [UInt32: String]) -> String? {+ guard IPv4.isTailnet(host) else { return "host \(IPv4.string(host)) is not a Tailscale address (100.64.0.0/10)" }+ guard let name = interfaces[host] else { return "host \(IPv4.string(host)) is not on any interface yet" }+ guard name.hasPrefix("utun") else { return "host \(IPv4.string(host)) is on \(name), not a Tailscale utun interface" }+ return nil+ }+}diff --git a/apple/helper/Sources/HelperCore/Problem.swift b/apple/helper/Sources/HelperCore/Problem.swiftnew file mode 100644index 0000000000..e22c6474cb--- /dev/null+++ b/apple/helper/Sources/HelperCore/Problem.swift@@ -0,0 +1,98 @@+import Foundation++public struct Problem: Error, Equatable {+ public static let docBase = "https://github.com/garrytan/gbrain/blob/master/apple/helper/README.md#"++ public let code: String+ public let status: Int+ public let title: String+ public let detail: String+ public let retryable: Bool+ public let retryAfter: Int?+ public let fix: String?++ public init(code: String, status: Int, title: String, detail: String, retryable: Bool = false, retryAfter: Int? = nil, fix: String? = nil) {+ self.code = code+ self.status = status+ self.title = title+ self.detail = detail+ self.retryable = retryable+ self.retryAfter = retryAfter+ self.fix = fix+ }++ public static var noAccess: Problem {+ Problem(code: "no_access", status: 503, title: "No access to the Messages database",+ detail: "macOS refused to open the Messages database because \(HelperIdentity.appName) has no Full Disk Access.",+ fix: "On the Mac, open System Settings > Privacy & Security > Full Disk Access, turn on \(HelperIdentity.appName), then run: launchctl kickstart -k gui/\(getuid())/\(HelperIdentity.bundleId)")+ }++ public static func dbMissing(path: String) -> Problem {+ Problem(code: "db_missing", status: 503, title: "Messages database not found",+ detail: "There is no Messages database at the configured path.",+ fix: "Check db_path in the helper's config (default \(path)) and that Messages is signed in on this Mac.")+ }++ public static let dbBusy = Problem(code: "db_busy", status: 503, title: "Messages database is busy",+ detail: "The database stayed locked for 2 seconds.", retryable: true, retryAfter: 2)++ public static func schemaMismatch(_ detail: String) -> Problem {+ Problem(code: "schema_mismatch", status: 503, title: "Messages database layout changed",+ detail: "A query the helper needs no longer fits the database: \(detail).",+ fix: "macOS changed the Messages database; rebuild the helper from a newer gbrain.")+ }++ public static func badParam(_ detail: String) -> Problem {+ Problem(code: "bad_param", status: 400, title: "Bad parameter", detail: detail)+ }++ public static func sourceDisabled(_ id: String) -> Problem {+ Problem(code: "source_disabled", status: 403, title: "Source turned off",+ detail: "The \(id) source is turned off in the helper's config.", fix: "Run: gbrain apple enable \(id)")+ }++ public static func insufficientScope(_ scope: String) -> Problem {+ Problem(code: "insufficient_scope", status: 403, title: "Token lacks the scope",+ detail: "This route needs a token with the \(scope) scope.")+ }++ public static let unauthorized = Problem(code: "unauthorized", status: 401, title: "Missing or wrong token",+ detail: "This route needs Authorization: Bearer <token>.",+ fix: "Send a token this helper knows as a bearer token.")+ public static let forbiddenSource = Problem(code: "forbidden_source", status: 403, title: "Caller not allowed",+ detail: "This listener does not accept requests from that address.")+ public static let forbiddenHost = Problem(code: "forbidden_host", status: 403, title: "Wrong Host header",+ detail: "The helper answers only requests addressed to 127.0.0.1 and its own port; other names, localhost included, are refused.",+ fix: "Call http://127.0.0.1:<port>/ directly.")+ public static let forbiddenOrigin = Problem(code: "forbidden_origin", status: 403, title: "Browser requests refused",+ detail: "A request with an Origin header comes from a web page, and the helper refuses it.")+ public static let overloaded = Problem(code: "overloaded", status: 503, title: "Too many requests at once",+ detail: "The helper handles a few requests at a time.", retryable: true, retryAfter: 1)+ public static let notFound = Problem(code: "not_found", status: 404, title: "Not found", detail: "No route or chat matches the request.")+ public static let tooLarge = Problem(code: "too_large", status: 413, title: "Request too large", detail: "Requests are limited to 8 KB.")+ public static let methodNotAllowed = Problem(code: "method_not_allowed", status: 405, title: "Method not allowed", detail: "Only GET is supported.")+ public static let internalError = Problem(code: "internal", status: 500, title: "Internal error",+ detail: "The helper hit an unexpected error; the request id is in the helper's log.", retryable: true, retryAfter: 5)++ public static func from(_ error: Error, dbPath: String) -> Problem {+ if let problem = error as? Problem { return problem }+ switch error as? DBError {+ case .missing: return .dbMissing(path: dbPath)+ case .noAccess: return .noAccess+ case .busy: return .dbBusy+ case .schemaMismatch(let detail): return .schemaMismatch(detail)+ default: return .internalError+ }+ }+}++struct ProblemBody: Encodable {+ let type: String+ let title: String+ let status: Int+ let code: String+ let detail: String+ let retryable: Bool+ let fix: String?+ let requestId: String+}diff --git a/apple/helper/Sources/HelperCore/Reader.swift b/apple/helper/Sources/HelperCore/Reader.swiftnew file mode 100644index 0000000000..8dc4d02732--- /dev/null+++ b/apple/helper/Sources/HelperCore/Reader.swift@@ -0,0 +1,360 @@+import Foundation++public struct Message: Encodable, Equatable {+ public let id: String+ public let rowid: Int64+ public let timestamp: String+ public let isFromMe: Bool+ public let text: String+ public let textStatus: String+ public let from: String?+ public let fromIdentifier: String?+ public let hasAttachments: Bool+ public let editedAt: String?+ public let retracted: Bool+}++public struct ThreadSummary: Encodable {+ public let threadId: String+ public let chatRowids: [Int64]+ public let chatGuids: [String]+ public let displayName: String+ public let participantCount: Int+ public let participants: [String]+ public let isGroup: Bool+ public let service: String?+ public let lastMessageAt: String?+ public let lastMessage: Message?+}++public struct ThreadPage: Encodable {+ public let contact: String+ public let threadId: String+ public let isGroup: Bool+ public let messages: [Message]+}++public struct PageMeta: Encodable {+ public let count: Int+ public let undecodable: Int+ public let hasMore: Bool+ public let nextSinceRowid: Int64?+}++public struct ListMeta: Encodable {+ public let count: Int+ public let undecodable: Int+}++public struct Envelope<Data: Encodable, Meta: Encodable>: Encodable {+ public let data: Data+ public let meta: Meta+}++public struct Stats: Encodable {+ public let db: String+ public let threads: Int+ public let threadIdShapes: [String: Int]+ public let recentMessagesScanned: Int+ public let decodedFromBody: Int+ public let undecodable: Int+ public let edited: Int+ public let editedTextDiffers: Int+ public let retracted: Int+}++public enum ThreadSelector: Equatable {+ case identifier(String)+ case chatRowid(Int64)+ case chatGuid(String)+}++struct ChatInfo {+ let rowid: Int64+ let guid: String+ let identifier: String+ let displayName: String?+ let service: String?+ let style: Int64+}++public final class Reader {+ let db: ChatDB++ public init(db: ChatDB) {+ self.db = db+ }++ public func listThreads(limit: Int, includeLastMessage: Bool) throws -> Envelope<[ThreadSummary], ListMeta> {+ var rows: [(String, Int64)] = []+ try db.query(listSQL(), [.int(Int64(limit))]) { row in+ if let identifier = row.text(0) { rows.append((identifier, row.int(1))) }+ }+ var threads: [ThreadSummary] = []+ var undecodable = 0+ for (identifier, lastRaw) in rows {+ let chats = try chats(for: .identifier(identifier))+ guard !chats.isEmpty else { continue }+ let participants = try participants(of: chats)+ var last: Message?+ if includeLastMessage {+ last = try messages(in: chats, sinceRowid: nil, since: nil, limit: 1, includeReactions: false).messages.last+ if last?.textStatus == "undecodable" { undecodable += 1 }+ }+ threads.append(summary(identifier: identifier, chats: chats, participants: participants, last: last, lastRaw: lastRaw))+ }+ return Envelope(data: threads, meta: ListMeta(count: threads.count, undecodable: undecodable))+ }++ public func getThread(_ selector: ThreadSelector, sinceRowid: Int64?, since: Date?, limit: Int, includeReactions: Bool) throws -> Envelope<ThreadPage, PageMeta>? {+ let chats = try chats(for: selector)+ guard let first = chats.first else { return nil }+ let participants = try participants(of: chats)+ let page = try messages(in: chats, sinceRowid: sinceRowid, since: since, limit: limit, includeReactions: includeReactions)+ let named = chats.compactMap { $0.displayName }.first { !$0.isEmpty }+ let data = ThreadPage(contact: named ?? first.identifier, threadId: first.identifier, isGroup: isGroup(chats, participants), messages: page.messages)+ let meta = PageMeta(+ count: page.messages.count,+ undecodable: page.messages.filter { $0.textStatus == "undecodable" }.count,+ hasMore: page.hasMore,+ nextSinceRowid: page.messages.map(\.rowid).max() ?? sinceRowid+ )+ return Envelope(data: data, meta: meta)+ }++ public func stats(recent: Int = 500) throws -> Stats {+ var shapes: [String: Int] = [:]+ try db.query("SELECT MIN(COALESCE(chat_identifier, guid)) FROM chat GROUP BY lower(COALESCE(chat_identifier, guid))", []) { row in+ shapes[Self.shape(of: row.text(0) ?? ""), default: 0] += 1+ }+ let threads = shapes.values.reduce(0, +)+ var scanned = 0, decoded = 0, undecodable = 0, edited = 0, differs = 0, retracted = 0+ try db.query(statsSQL(), [.int(Int64(recent))]) { row in+ scanned += 1+ let text = row.text(0)+ let body = row.blob(1)+ let isEdited = row.int(2) != 0+ let isRetracted = row.int(3) != 0+ if isRetracted { retracted += 1 }+ if isEdited {+ edited += 1+ if let body, case .text(let fromBody) = TypedStream.decodeText(body), Self.clean(fromBody) != Self.clean(text ?? "") { differs += 1 }+ }+ let resolved = Self.resolveText(text: text, body: body, edited: isEdited, retracted: isRetracted)+ if resolved.status == "undecodable" { undecodable += 1 }+ if resolved.status == "ok", Self.clean(text ?? "").trimmingCharacters(in: .whitespacesAndNewlines).isEmpty { decoded += 1 }+ }+ return Stats(db: "ok", threads: threads, threadIdShapes: shapes, recentMessagesScanned: scanned, decodedFromBody: decoded, undecodable: undecodable, edited: edited, editedTextDiffers: differs, retracted: retracted)+ }++ public func healthCheck() throws {+ try db.validateSchema()+ try db.check(listSQL())+ try db.check(chatSQL(.identifier("")))+ try db.check(chatSQL(.chatRowid(0)))+ try db.check(chatSQL(.chatGuid("")))+ try db.check(participantsSQL(count: 1))+ try db.check(statsSQL())+ for (rowid, date) in [(false, false), (true, false), (false, true), (true, true)] {+ try db.check(messageSQL(chatCount: 1, sinceRowid: rowid, sinceDate: date, includeReactions: false))+ try db.check(messageSQL(chatCount: 1, sinceRowid: rowid, sinceDate: date, includeReactions: true))+ }+ try db.query("SELECT 1 FROM message LIMIT 1", []) { _ in }+ }++ static func shape(of identifier: String) -> String {+ let digits = CharacterSet.decimalDigits+ func allDigits(_ text: Substring) -> Bool { !text.isEmpty && text.unicodeScalars.allSatisfy { digits.contains($0) } }+ if identifier.hasPrefix("+"), allDigits(identifier.dropFirst()) { return "plus_digits" }+ if identifier.contains("@") { return "email" }+ if identifier.hasPrefix("chat"), allDigits(identifier.dropFirst(4)) { return "chat_digits" }+ if allDigits(Substring(identifier)) { return identifier.count <= 6 ? "short_code" : "long_digits" }+ if identifier.hasPrefix("urn:biz:") { return "business" }+ return identifier.count >= 30 ? "other_30_plus" : "other_under_30"+ }++ static func resolveText(text: String?, body: [UInt8]?, edited: Bool, retracted: Bool) -> (text: String, status: String) {+ if retracted { return ("", "empty") }+ if edited, let body, !body.isEmpty {+ switch TypedStream.decodeText(body) {+ case .text(let fromBody) where !isBlank(clean(fromBody)): return (clean(fromBody), "ok")+ case .undecodable: return ("", "undecodable")+ default: break+ }+ }+ if let text, !isBlank(clean(text)) { return (clean(text), "ok") }+ guard let body, !body.isEmpty else { return ("", "empty") }+ switch TypedStream.decodeText(body) {+ case .text(let fromBody):+ let cleaned = clean(fromBody)+ return isBlank(cleaned) ? ("", "empty") : (cleaned, "ok")+ case .undecodable:+ return ("", "undecodable")+ }+ }++ static func clean(_ text: String) -> String {+ text.replacingOccurrences(of: "\u{FFFC}", with: "")+ }++ static func isBlank(_ text: String) -> Bool {+ text.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty+ }++ func summary(identifier: String, chats: [ChatInfo], participants: [String], last: Message?, lastRaw: Int64) -> ThreadSummary {+ let named = chats.compactMap { $0.displayName }.first { !$0.isEmpty }+ let services = chats.compactMap { $0.service }.reduce(into: [String]()) { if !$0.contains($1) { $0.append($1) } }+ return ThreadSummary(+ threadId: identifier,+ chatRowids: chats.map(\.rowid),+ chatGuids: chats.map(\.guid),+ displayName: named ?? identifier,+ participantCount: participants.count + 1,+ participants: participants,+ isGroup: isGroup(chats, participants),+ service: services.isEmpty ? nil : services.joined(separator: ","),+ lastMessageAt: last?.timestamp ?? (lastRaw != 0 ? AppleTime.iso(fromRaw: lastRaw) : nil),+ lastMessage: last+ )+ }++ func isGroup(_ chats: [ChatInfo], _ participants: [String]) -> Bool {+ chats.contains { $0.style == 43 } || participants.count > 1+ }++ func chats(for selector: ThreadSelector) throws -> [ChatInfo] {+ var result: [ChatInfo] = []+ let bind: SQLValue+ switch selector {+ case .identifier(let value): bind = .text(value)+ case .chatRowid(let value): bind = .int(value)+ case .chatGuid(let value): bind = .text(value)+ }+ try db.query(chatSQL(selector), [bind]) { row in+ result.append(ChatInfo(rowid: row.int(0), guid: row.text(1) ?? "", identifier: row.text(2) ?? "", displayName: row.text(3), service: row.text(4), style: row.int(5)))+ }+ return result+ }++ func participants(of chats: [ChatInfo]) throws -> [String] {+ var handles: [String] = []+ try db.query(participantsSQL(count: chats.count), chats.map { .int($0.rowid) }) { row in+ if let id = row.text(0) { handles.append(id) }+ }+ return handles+ }++ func messages(in chats: [ChatInfo], sinceRowid: Int64?, since: Date?, limit: Int, includeReactions: Bool) throws -> (messages: [Message], hasMore: Bool) {+ let cursor = sinceRowid != nil || since != nil+ var binds: [SQLValue] = chats.map { .int($0.rowid) }+ if let sinceRowid { binds.append(.int(sinceRowid)) }+ if let since { binds.append(.int(AppleTime.raw(from: since))) }+ binds.append(.int(Int64(limit + 1)))+ var messages: [Message] = []+ let sql = messageSQL(chatCount: chats.count, sinceRowid: sinceRowid != nil, sinceDate: since != nil, includeReactions: includeReactions)+ try db.query(sql, binds) { row in+ messages.append(Self.message(from: row))+ }+ let hasMore = messages.count > limit+ if hasMore { messages.removeLast(messages.count - limit) }+ if !cursor { messages.reverse() }+ return (messages, hasMore)+ }++ static func message(from row: Row) -> Message {+ let fromMe = row.int(5) != 0+ let editedRaw = row.int(8)+ let resolved = resolveText(text: row.text(2), body: row.blob(3), edited: editedRaw != 0, retracted: row.int(9) != 0)+ return Message(+ id: row.text(1) ?? "",+ rowid: row.int(0),+ timestamp: AppleTime.iso(fromRaw: row.int(4)),+ isFromMe: fromMe,+ text: resolved.text,+ textStatus: resolved.status,+ from: fromMe ? nil : row.text(6),+ fromIdentifier: fromMe ? nil : row.text(6),+ hasAttachments: row.int(7) != 0,+ editedAt: editedRaw != 0 ? AppleTime.iso(fromRaw: editedRaw) : nil,+ retracted: row.int(9) != 0+ )+ }++ func column(_ table: String, _ name: String, _ alias: String, fallback: String) -> String {+ db.has(table, name) ? "\(alias).\(name)" : fallback+ }++ func listSQL() -> String {+ let joinHasDate = db.has("chat_message_join", "message_date")+ let date = AppleTime.normalizedSQL(joinHasDate ? "cmj.message_date" : "m.date")+ let join = joinHasDate ? "" : " JOIN message m ON m.ROWID = cmj.message_id"+ return """+ SELECT MIN(COALESCE(c.chat_identifier, c.guid)) AS ident, MAX(\(date)) AS last+ FROM chat c JOIN chat_message_join cmj ON cmj.chat_id = c.ROWID\(join)+ GROUP BY lower(COALESCE(c.chat_identifier, c.guid))+ ORDER BY last DESC+ LIMIT ?+ """+ }++ func chatSQL(_ selector: ThreadSelector) -> String {+ let condition: String+ switch selector {+ case .identifier: condition = "lower(COALESCE(c.chat_identifier, c.guid)) = lower(?)"+ case .chatRowid: condition = "c.ROWID = ?"+ case .chatGuid: condition = "c.guid = ?"+ }+ return """+ SELECT c.ROWID, c.guid, COALESCE(c.chat_identifier, c.guid), \(column("chat", "display_name", "c", fallback: "NULL")),+ \(column("chat", "service_name", "c", fallback: "NULL")), \(column("chat", "style", "c", fallback: "0"))+ FROM chat c WHERE \(condition) ORDER BY c.ROWID+ """+ }++ func participantsSQL(count: Int) -> String {+ """+ SELECT DISTINCT h.id FROM chat_handle_join chj JOIN handle h ON h.ROWID = chj.handle_id+ WHERE chj.chat_id IN (\(placeholders(count))) ORDER BY h.id+ """+ }++ func messageSQL(chatCount: Int, sinceRowid: Bool, sinceDate: Bool, includeReactions: Bool) -> String {+ let date = AppleTime.normalizedSQL("m.date")+ var sql = """+ SELECT m.ROWID, m.guid, m.text, \(column("message", "attributedBody", "m", fallback: "NULL")), m.date, m.is_from_me, h.id,+ \(column("message", "cache_has_attachments", "m", fallback: "0")), \(column("message", "date_edited", "m", fallback: "0")),+ \(column("message", "date_retracted", "m", fallback: "0"))+ FROM message m LEFT JOIN handle h ON h.ROWID = m.handle_id+ WHERE m.ROWID IN (SELECT cmj.message_id FROM chat_message_join cmj WHERE cmj.chat_id IN (\(placeholders(chatCount))))+ """+ if !includeReactions { sql += reactionFilter() }+ if sinceRowid { sql += " AND m.ROWID > ?" }+ if sinceDate { sql += " AND \(date) > ?" }+ if sinceRowid || sinceDate {+ sql += " ORDER BY m.ROWID ASC"+ } else {+ sql += " ORDER BY \(date) DESC, m.ROWID DESC"+ }+ return sql + " LIMIT ?"+ }++ func statsSQL() -> String {+ """+ SELECT m.text, \(column("message", "attributedBody", "m", fallback: "NULL")), \(column("message", "date_edited", "m", fallback: "0")),+ \(column("message", "date_retracted", "m", fallback: "0"))+ FROM message m WHERE 1 = 1\(reactionFilter())+ ORDER BY m.ROWID DESC LIMIT ?+ """+ }++ func reactionFilter() -> String {+ var sql = ""+ if db.has("message", "associated_message_type") { sql += " AND COALESCE(m.associated_message_type, 0) NOT BETWEEN 2000 AND 3999" }+ if db.has("message", "item_type") { sql += " AND COALESCE(m.item_type, 0) = 0" }+ return sql+ }++ func placeholders(_ count: Int) -> String {+ Array(repeating: "?", count: max(count, 1)).joined(separator: ", ")+ }+}diff --git a/apple/helper/Sources/HelperCore/Router.swift b/apple/helper/Sources/HelperCore/Router.swiftnew file mode 100644index 0000000000..d97bf54613--- /dev/null+++ b/apple/helper/Sources/HelperCore/Router.swift@@ -0,0 +1,106 @@+import Foundation++public enum Listener: Equatable {+ case loopback+ case tailnet(allowed: Set<UInt32>)+}++public struct RouterSettings {+ public let listener: Listener+ public let grants: () -> [TokenGrant]+ public let sources: [SourceModule]+ public let enabled: Set<String>+ public let version: String++ public init(listener: Listener, grants: [TokenGrant], sources: [SourceModule], enabled: Set<String>, version: String) {+ self.init(listener: listener, liveGrants: { grants }, sources: sources, enabled: enabled, version: version)+ }++ public init(listener: Listener, liveGrants: @escaping () -> [TokenGrant], sources: [SourceModule], enabled: Set<String>, version: String) {+ self.listener = listener+ self.grants = liveGrants+ self.sources = sources+ self.enabled = enabled+ self.version = version+ }+}++struct Health: Encodable {+ let ok: Bool+ let version: String+}++struct SourcesMeta: Encodable {+ let count: Int+}++public struct Routed {+ public let response: HTTPResponse+ public let route: String+ public let rows: Int+}++public final class Router {+ public let settings: RouterSettings+ let localAddresses: () -> Set<UInt32>++ public init(settings: RouterSettings, localAddresses: @escaping () -> Set<UInt32> = { Set(Interfaces.ipv4().keys) }) {+ self.settings = settings+ self.localAddresses = localAddresses+ }++ public func admits(_ peer: UInt32) -> Bool {+ switch settings.listener {+ case .loopback:+ return IPv4.isLoopback(peer)+ case .tailnet(let allowed):+ return allowed.contains(peer) && !IPv4.isLoopback(peer) && !localAddresses().contains(peer)+ }+ }++ public func handle(_ request: HTTPRequest, peer: UInt32, port: UInt16, requestId: String) -> Routed {+ let source = settings.sources.first { $0.routes.contains(request.path) }+ let known = request.path == "/health" || request.path == "/v1/sources" || source != nil+ let route = known ? request.path : "unknown"+ func fail(_ problem: Problem) -> Routed {+ Routed(response: .problem(problem, requestId: requestId), route: route, rows: 0)+ }+ guard admits(peer) else { return fail(.forbiddenSource) }+ if settings.listener == .loopback {+ guard request.headers["origin"] == nil else { return fail(.forbiddenOrigin) }+ guard request.headers["host"] == "127.0.0.1:\(port)" else { return fail(.forbiddenHost) }+ }+ guard request.method == "GET" else { return fail(.methodNotAllowed) }+ if request.path == "/health" {+ return Routed(response: .json(200, Health(ok: true, version: settings.version)), route: route, rows: 0)+ }+ guard let grant = Token.grant(authorization: request.headers["authorization"], grants: settings.grants()) else { return fail(.unauthorized) }+ if request.path == "/v1/sources" {+ let statuses = settings.sources.map { status($0) }+ return Routed(response: .json(200, Envelope(data: statuses, meta: SourcesMeta(count: statuses.count))), route: route, rows: 0)+ }+ guard let source else { return fail(.notFound) }+ guard settings.enabled.contains(source.id) else { return fail(.sourceDisabled(source.id)) }+ let scope = SourceCatalog.readScope(source.id)+ guard grant.scopes.contains(scope) else { return fail(.insufficientScope(scope)) }+ do {+ let result = try source.handle(path: request.path, query: request.query)+ return Routed(response: result.response, route: route, rows: result.rows)+ } catch {+ let problem = (error as? Problem) ?? source.problem(for: error)+ if problem.code == Problem.internalError.code { Log.line("route=\(route) error=\(Self.kind(error)) rid=\(requestId)") }+ return fail(problem)+ }+ }++ func status(_ source: SourceModule) -> SourceStatus {+ let enabled = settings.enabled.contains(source.id)+ return SourceStatus(id: source.id, enabled: enabled, permission: source.permission, state: enabled ? source.state() : "disabled",+ scopes: [SourceCatalog.readScope(source.id)], routes: source.routes)+ }++ static func kind(_ error: Error) -> String {+ if case DBError.sqlite(let code, _) = error { return "sqlite_\(code)" }+ return String(describing: type(of: error))+ }+}diff --git a/apple/helper/Sources/HelperCore/Server.swift b/apple/helper/Sources/HelperCore/Server.swiftnew file mode 100644index 0000000000..241602bb86--- /dev/null+++ b/apple/helper/Sources/HelperCore/Server.swift@@ -0,0 +1,158 @@+import Darwin+import Foundation++public struct ServerError: Error, CustomStringConvertible {+ public let description: String+}++public final class Server {+ let host: UInt32+ let port: UInt16+ let router: Router+ let maxConcurrent: Int+ let ioTimeout: TimeInterval+ let maxRequestBytes: Int+ let lock = NSLock()+ var listener: Int32 = -1+ var inflight = 0+ var stopped = false+ public private(set) var boundPort: UInt16 = 0++ public init(host: UInt32, port: UInt16, router: Router, maxConcurrent: Int = 4, ioTimeout: TimeInterval = 10, maxRequestBytes: Int = 8192) {+ self.host = host+ self.port = port+ self.router = router+ self.maxConcurrent = maxConcurrent+ self.ioTimeout = ioTimeout+ self.maxRequestBytes = maxRequestBytes+ }++ public func start() throws {+ let fd = socket(AF_INET, SOCK_STREAM, 0)+ guard fd >= 0 else { throw ServerError(description: "socket: \(String(cString: strerror(errno)))") }+ var yes: Int32 = 1+ setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, &yes, socklen_t(MemoryLayout<Int32>.size))+ var address = sockaddr_in()+ address.sin_len = UInt8(MemoryLayout<sockaddr_in>.size)+ address.sin_family = sa_family_t(AF_INET)+ address.sin_port = port.bigEndian+ address.sin_addr = in_addr(s_addr: host.bigEndian)+ let bound = withUnsafePointer(to: &address) {+ $0.withMemoryRebound(to: sockaddr.self, capacity: 1) { bind(fd, $0, socklen_t(MemoryLayout<sockaddr_in>.size)) }+ }+ guard bound == 0, listen(fd, 32) == 0 else {+ let reason = String(cString: strerror(errno))+ close(fd)+ throw ServerError(description: "bind \(IPv4.string(host)):\(port): \(reason)")+ }+ var actual = sockaddr_in()+ var length = socklen_t(MemoryLayout<sockaddr_in>.size)+ _ = withUnsafeMutablePointer(to: &actual) {+ $0.withMemoryRebound(to: sockaddr.self, capacity: 1) { getsockname(fd, $0, &length) }+ }+ boundPort = UInt16(bigEndian: actual.sin_port)+ listener = fd+ }++ public func stop() {+ lock.lock()+ stopped = true+ lock.unlock()+ }++ var isStopped: Bool {+ lock.lock()+ defer { lock.unlock() }+ return stopped+ }++ public func serveForever() {+ defer { close(listener) }+ while !isStopped {+ var poller = pollfd(fd: listener, events: Int16(POLLIN), revents: 0)+ guard poll(&poller, 1, 200) > 0 else { continue }+ var peer = sockaddr_in()+ var length = socklen_t(MemoryLayout<sockaddr_in>.size)+ let client = withUnsafeMutablePointer(to: &peer) {+ $0.withMemoryRebound(to: sockaddr.self, capacity: 1) { accept(listener, $0, &length) }+ }+ guard client >= 0 else { continue }+ var noSigPipe: Int32 = 1+ setsockopt(client, SOL_SOCKET, SO_NOSIGPIPE, &noSigPipe, socklen_t(MemoryLayout<Int32>.size))+ let peerIP = UInt32(bigEndian: peer.sin_addr.s_addr)+ guard router.admits(peerIP) else {+ let rid = Self.requestId()+ finish(client, .problem(.forbiddenSource, requestId: rid), route: "-", rows: 0, requestId: rid, started: Date())+ continue+ }+ lock.lock()+ let busy = inflight >= maxConcurrent+ if !busy { inflight += 1 }+ lock.unlock()+ if busy {+ let rid = Self.requestId()+ finish(client, .problem(.overloaded, requestId: rid), route: "-", rows: 0, requestId: rid, started: Date())+ continue+ }+ DispatchQueue.global(qos: .userInitiated).async {+ self.serve(client, peer: peerIP)+ self.lock.lock()+ self.inflight -= 1+ self.lock.unlock()+ }+ }+ }++ func serve(_ client: Int32, peer: UInt32) {+ let started = Date()+ let rid = Self.requestId()+ var buffer: [UInt8] = []+ var chunk = [UInt8](repeating: 0, count: 4096)+ var parsed = HTTPParse.incomplete+ while case .incomplete = parsed {+ let remaining = ioTimeout - Date().timeIntervalSince(started)+ guard remaining > 0 else { return abandon(client, requestId: rid, started: started) }+ var timeout = timeval(tv_sec: Int(remaining), tv_usec: Int32((remaining - Double(Int(remaining))) * 1_000_000))+ setsockopt(client, SOL_SOCKET, SO_RCVTIMEO, &timeout, socklen_t(MemoryLayout<timeval>.size))+ setsockopt(client, SOL_SOCKET, SO_SNDTIMEO, &timeout, socklen_t(MemoryLayout<timeval>.size))+ let count = recv(client, &chunk, chunk.count, 0)+ guard count > 0 else { return abandon(client, requestId: rid, started: started) }+ buffer.append(contentsOf: chunk[0..<count])+ parsed = HTTPParser.parse(buffer, maxBytes: maxRequestBytes)+ }+ switch parsed {+ case .request(let request):+ let routed = router.handle(request, peer: peer, port: boundPort, requestId: rid)+ finish(client, routed.response, route: routed.route, rows: routed.rows, requestId: rid, started: started)+ case .tooLarge:+ finish(client, .problem(.tooLarge, requestId: rid), route: "-", rows: 0, requestId: rid, started: started)+ default:+ finish(client, .problem(.badParam("malformed HTTP request"), requestId: rid), route: "-", rows: 0, requestId: rid, started: started)+ }+ }++ func abandon(_ client: Int32, requestId: String, started: Date) {+ close(client)+ Log.line("route=- status=408 ms=\(Self.millis(since: started)) rows=0 rid=\(requestId)")+ }++ func finish(_ client: Int32, _ response: HTTPResponse, route: String, rows: Int, requestId: String, started: Date) {+ let bytes = response.serialize(requestId: requestId)+ var sent = 0+ while sent < bytes.count {+ let count = bytes[sent...].withUnsafeBytes { send(client, $0.baseAddress, $0.count, 0) }+ if count <= 0 { break }+ sent += count+ }+ close(client)+ Log.line("route=\(route) status=\(response.status) ms=\(Self.millis(since: started)) rows=\(rows) rid=\(requestId)")+ }++ static func millis(since start: Date) -> Int {+ Int(Date().timeIntervalSince(start) * 1000)+ }++ static func requestId() -> String {+ String(format: "%016llx", UInt64.random(in: 0...UInt64.max))+ }+}diff --git a/apple/helper/Sources/HelperCore/SourceModule.swift b/apple/helper/Sources/HelperCore/SourceModule.swiftnew file mode 100644index 0000000000..faf61a1896--- /dev/null+++ b/apple/helper/Sources/HelperCore/SourceModule.swift@@ -0,0 +1,117 @@+import Foundation++public enum SourceCatalog {+ public static let ids = ["imessage"]+ public static var scopes: Set<String> { Set(ids.map(readScope)) }++ public static func readScope(_ id: String) -> String {+ "read:\(id)"+ }+}++public struct SourceStatus: Encodable, Equatable {+ public let id: String+ public let enabled: Bool+ public let permission: String+ public let state: String+ public let scopes: [String]+ public let routes: [String]+}++public protocol SourceModule {+ var id: String { get }+ var permission: String { get }+ var routes: [String] { get }+ func state() -> String+ func handle(path: String, query: [String: String]) throws -> (response: HTTPResponse, rows: Int)+ func problem(for error: Error) -> Problem+}++public final class IMessageSource: SourceModule {+ public let id = "imessage"+ public let permission = "full_disk_access"+ public let routes = ["/v1/imessage/threads", "/v1/imessage/thread"]+ let dbPath: String++ public init(dbPath: String) {+ self.dbPath = dbPath+ }++ public func state() -> String {+ do {+ try Reader(db: try ChatDB.open(path: dbPath)).healthCheck()+ return "ok"+ } catch DBError.missing {+ return "missing"+ } catch DBError.noAccess {+ return "no_access"+ } catch DBError.schemaMismatch {+ return "schema_mismatch"+ } catch DBError.busy {+ return "busy"+ } catch {+ return "error"+ }+ }++ public func handle(path: String, query: [String: String]) throws -> (response: HTTPResponse, rows: Int) {+ let db = try ChatDB.open(path: dbPath)+ try db.validateSchema()+ let reader = Reader(db: db)+ if path == "/v1/imessage/threads" {+ let limit = try Self.intParam(query, "limit", 1...200) ?? 30+ let includeLast = try Self.boolParam(query, "include_last_message", default: true)+ let result = try reader.listThreads(limit: Int(limit), includeLastMessage: includeLast)+ return (response: .json(200, result), rows: result.data.count)+ }+ let selector = try Self.selector(query)+ let limit = try Self.intParam(query, "limit", 1...500) ?? 50+ let sinceRowid = try Self.intParam(query, "since_rowid", 0...Int64.max)+ var since: Date?+ if let raw = query["since_iso"] {+ guard let parsed = AppleTime.parseISO(raw) else { throw Problem.badParam("since_iso must be an ISO-8601 time such as 2026-01-01T00:00:00Z") }+ since = parsed+ }+ let includeReactions = try Self.boolParam(query, "include_reactions", default: false)+ guard let result = try reader.getThread(selector, sinceRowid: sinceRowid, since: since, limit: Int(limit), includeReactions: includeReactions) else {+ throw Problem.notFound+ }+ return (response: .json(200, result), rows: result.data.messages.count)+ }++ public func problem(for error: Error) -> Problem {+ Problem.from(error, dbPath: dbPath)+ }++ static func selector(_ query: [String: String]) throws -> ThreadSelector {+ let present = ["thread_id", "chat_rowid", "chat_guid"].filter { query[$0] != nil }+ guard present.count == 1 else { throw Problem.badParam("give exactly one of thread_id, chat_rowid or chat_guid") }+ switch present[0] {+ case "chat_rowid":+ return .chatRowid(try intParam(query, "chat_rowid", 1...Int64.max)!)+ case "chat_guid":+ guard let guid = query["chat_guid"], !guid.isEmpty else { throw Problem.badParam("chat_guid must not be empty") }+ return .chatGuid(guid)+ default:+ guard let id = query["thread_id"], !id.isEmpty else { throw Problem.badParam("thread_id must not be empty") }+ return .identifier(id)+ }+ }++ static func intParam(_ query: [String: String], _ name: String, _ range: ClosedRange<Int64>) throws -> Int64? {+ guard let raw = query[name] else { return nil }+ guard let value = Int64(raw), range.contains(value) else {+ throw Problem.badParam("\(name) must be an integer from \(range.lowerBound) to \(range.upperBound)")+ }+ return value+ }++ static func boolParam(_ query: [String: String], _ name: String, default fallback: Bool) throws -> Bool {+ guard let raw = query[name] else { return fallback }+ switch raw.lowercased() {+ case "1", "true", "yes": return true+ case "0", "false", "no": return false+ default: throw Problem.badParam("\(name) must be true or false")+ }+ }+}diff --git a/apple/helper/Sources/HelperCore/Token.swift b/apple/helper/Sources/HelperCore/Token.swiftnew file mode 100644index 0000000000..cf33edea8a--- /dev/null+++ b/apple/helper/Sources/HelperCore/Token.swift@@ -0,0 +1,52 @@+import CryptoKit+import Foundation++public struct TokenGrant: Equatable {+ public let name: String+ public let verifier: [UInt8]+ public let scopes: Set<String>++ public init(name: String, verifier: [UInt8], scopes: Set<String>) {+ self.name = name+ self.verifier = verifier+ self.scopes = scopes+ }+}++public enum Token {+ public static func verifier(fromHex hex: String) -> [UInt8]? {+ let chars = Array(hex.lowercased())+ guard chars.count == 64 else { return nil }+ var bytes: [UInt8] = []+ for i in stride(from: 0, to: 64, by: 2) {+ guard let byte = UInt8(String(chars[i...i + 1]), radix: 16) else { return nil }+ bytes.append(byte)+ }+ return bytes+ }++ public static func hexDigest(_ token: String) -> String {+ SHA256.hash(data: Data(token.utf8)).map { String(format: "%02x", $0) }.joined()+ }++ public static func grant(authorization: String?, grants: [TokenGrant]) -> TokenGrant? {+ guard let authorization else { return nil }+ let parts = authorization.split(separator: " ", maxSplits: 1)+ guard parts.count == 2, parts[0].lowercased() == "bearer" else { return nil }+ let presented = parts[1].trimmingCharacters(in: .whitespaces)+ guard !presented.isEmpty else { return nil }+ let digest = Array(SHA256.hash(data: Data(presented.utf8)))+ var match: TokenGrant?+ for grant in grants where equal(digest, grant.verifier) && match == nil {+ match = grant+ }+ return match+ }++ static func equal(_ a: [UInt8], _ b: [UInt8]) -> Bool {+ guard a.count == b.count else { return false }+ var difference: UInt8 = 0+ for i in 0..<a.count { difference |= a[i] ^ b[i] }+ return difference == 0+ }+}diff --git a/apple/helper/Sources/HelperCore/TypedStream.swift b/apple/helper/Sources/HelperCore/TypedStream.swiftnew file mode 100644index 0000000000..13bbe887fa--- /dev/null+++ b/apple/helper/Sources/HelperCore/TypedStream.swift@@ -0,0 +1,69 @@+import Foundation++public enum BodyDecode: Equatable {+ case text(String)+ case undecodable+}++public enum TypedStream {+ static let header: [UInt8] = [0x04, 0x0B] + Array("streamtyped".utf8)+ static let stringClasses: [[UInt8]] = ["NSString", "NSMutableString"].map { [UInt8($0.utf8.count)] + Array($0.utf8) }+ static let stringType: [UInt8] = [0x84, 0x01, 0x2B]+ static let typeSearchWindow = 32++ public static func decodeText(_ data: [UInt8]) -> BodyDecode {+ guard data.count > header.count, Array(data[0..<header.count]) == header else { return .undecodable }+ guard let classEnd = earliestClassEnd(data) else { return .undecodable }+ guard let typeAt = find(stringType, in: data, from: classEnd, limit: classEnd + typeSearchWindow) else { return .undecodable }+ var cursor = typeAt + stringType.count+ guard let length = readLength(data, &cursor), length <= data.count - cursor else { return .undecodable }+ guard let text = String(bytes: data[cursor..<(cursor + length)], encoding: .utf8) else { return .undecodable }+ return .text(text)+ }++ static func earliestClassEnd(_ data: [UInt8]) -> Int? {+ var best: Int?+ for name in stringClasses {+ guard let at = find(name, in: data, from: header.count) else { continue }+ let end = at + name.count+ if let current = best, current <= end { continue }+ best = end+ }+ return best+ }++ static func find(_ needle: [UInt8], in data: [UInt8], from start: Int, limit: Int? = nil) -> Int? {+ let last = min(limit ?? data.count, data.count) - needle.count+ guard !needle.isEmpty, start >= 0, start <= last else { return nil }+ var i = start+ while i <= last {+ var j = 0+ while j < needle.count && data[i + j] == needle[j] { j += 1 }+ if j == needle.count { return i }+ i += 1+ }+ return nil+ }++ static func readLength(_ data: [UInt8], _ cursor: inout Int) -> Int? {+ guard cursor < data.count else { return nil }+ let marker = data[cursor]+ cursor += 1+ switch marker {+ case 0x81:+ guard data.count - cursor >= 2 else { return nil }+ let value = Int16(bitPattern: UInt16(data[cursor]) | UInt16(data[cursor + 1]) << 8)+ cursor += 2+ return value >= 0 ? Int(value) : nil+ case 0x82:+ guard data.count - cursor >= 4 else { return nil }+ var raw: UInt32 = 0+ for k in 0..<4 { raw |= UInt32(data[cursor + k]) << (8 * UInt32(k)) }+ cursor += 4+ let value = Int32(bitPattern: raw)+ return value >= 0 ? Int(value) : nil+ default:+ return marker < 0x80 ? Int(marker) : nil+ }+ }+}diff --git a/apple/helper/Sources/gbrain-apple-helper/main.swift b/apple/helper/Sources/gbrain-apple-helper/main.swiftnew file mode 100644index 0000000000..5e78a94d0f--- /dev/null+++ b/apple/helper/Sources/gbrain-apple-helper/main.swift@@ -0,0 +1,144 @@+import Darwin+import Foundation+import HelperCore++let arguments = Array(CommandLine.arguments.dropFirst())+let name = HelperIdentity.executable+let version = Bundle.main.object(forInfoDictionaryKey: "CFBundleShortVersionString") as? String ?? "dev"++func value(_ flag: String) -> String? {+ guard let index = arguments.firstIndex(of: flag), index + 1 < arguments.count else { return nil }+ return arguments[index + 1]+}++func values(_ flag: String) -> [String] {+ arguments.indices.filter { arguments[$0] == flag && $0 + 1 < arguments.count }.map { arguments[$0 + 1] }+}++func fail(_ message: String, code: Int32) -> Never {+ FileHandle.standardError.write(Data("\(name): \(message)\n".utf8))+ exit(code)+}++let configPath = value("--config") ?? Config.defaultPath++func loadConfig() -> (Config, ValidatedConfig) {+ do {+ let config = try Config.load(path: configPath)+ return (config, try config.validated())+ } catch {+ fail("\(error)", code: 78)+ }+}++func save(_ config: Config) {+ do {+ _ = try config.validated()+ try config.write(path: configPath)+ } catch {+ fail("\(error)", code: 78)+ }+}++func printJSON<T: Encodable>(_ value: T) {+ let encoder = JSONEncoder()+ encoder.keyEncodingStrategy = .convertToSnakeCase+ encoder.outputFormatting = [.sortedKeys]+ if let data = try? encoder.encode(value) { print(String(decoding: data, as: UTF8.self)) }+}++struct DBState: Encodable {+ let db: String+}++let usage = "usage: \(name) serve | init --port <n> [--enable <source>] [--db <path>] | validate-config | check [--db <path>] | token add --name <name> --verifier <sha256> [--scope <scope>]... | token remove --name <name> | --version"++switch arguments.first {+case "--version", "version":+ print("\(name) \(version)")++case "init":+ guard let portText = value("--port"), let port = Int(portText) else { fail("init needs --port", code: 64) }+ guard !FileManager.default.fileExists(atPath: configPath) else { fail("a config already exists at \(configPath)", code: 73) }+ let enable = values("--enable")+ for id in enable where !SourceCatalog.ids.contains(id) { fail("unknown source \(id)", code: 64) }+ let sources = Dictionary(uniqueKeysWithValues: SourceCatalog.ids.map { ($0, SourceSetting(enabled: enable.contains($0))) })+ save(Config(port: port, sources: sources, dbPath: value("--db")))+ print("config written to \(configPath)")++case "validate-config":+ _ = loadConfig()+ print("config ok")++case "token":+ _ = loadConfig()+ let lockPath = configPath + ".lock"+ let lockFile = Darwin.open(lockPath, O_CREAT | O_RDWR, 0o600)+ guard lockFile >= 0, flock(lockFile, LOCK_EX) == 0 else { fail("could not lock \(lockPath)", code: 75) }+ var config = loadConfig().0+ guard let tokenName = value("--name") else { fail(usage, code: 64) }+ switch arguments.dropFirst().first {+ case "add":+ guard let verifier = value("--verifier") else { fail("token add needs --verifier <sha256 of the token>", code: 64) }+ config.tokens.removeAll { $0.name == tokenName }+ config.tokens.append(TokenEntry(name: tokenName, verifier: verifier.lowercased(), scopes: values("--scope")))+ save(config)+ print("token \(tokenName) saved")+ case "remove":+ guard config.tokens.contains(where: { $0.name == tokenName }) else { fail("no token named \(tokenName)", code: 1) }+ config.tokens.removeAll { $0.name == tokenName }+ save(config)+ print("token \(tokenName) removed")+ default:+ fail(usage, code: 64)+ }++case "check":+ let path = value("--db") ?? (try? Config.load(path: configPath))?.dbPath ?? Config.defaultDBPath+ do {+ let reader = Reader(db: try ChatDB.open(path: path))+ try reader.healthCheck()+ printJSON(try reader.stats())+ } catch DBError.noAccess {+ printJSON(DBState(db: "no_access"))+ exit(1)+ } catch DBError.missing {+ printJSON(DBState(db: "missing"))+ exit(1)+ } catch DBError.schemaMismatch {+ printJSON(DBState(db: "schema_mismatch"))+ exit(1)+ } catch {+ printJSON(DBState(db: "error"))+ exit(1)+ }++case "serve":+ do {+ try Log.open(path: Log.defaultPath)+ } catch {+ Log.line("logging to stderr: \(error)")+ }+ let config: ValidatedConfig+ do {+ config = try Config.load(path: configPath).validated()+ } catch {+ Log.line("not serving: \(error)")+ exit(78)+ }+ let grants = LiveGrants(path: configPath)+ let settings = RouterSettings(listener: .loopback, liveGrants: grants.current, sources: [IMessageSource(dbPath: config.dbPath)],+ enabled: config.enabled, version: version)+ let server = Server(host: IPv4.parse("127.0.0.1")!, port: config.port, router: Router(settings: settings))+ do {+ try server.start()+ } catch {+ Log.line("not serving: \(error)")+ exit(75)+ }+ Log.line("serving \(name) \(version) on 127.0.0.1:\(server.boundPort)")+ server.serveForever()++default:+ fail(usage, code: 64)+}diff --git a/apple/helper/Tests/HelperCoreTests/BundleTests.swift b/apple/helper/Tests/HelperCoreTests/BundleTests.swiftnew file mode 100644index 0000000000..ac0fb7362f--- /dev/null+++ b/apple/helper/Tests/HelperCoreTests/BundleTests.swift@@ -0,0 +1,42 @@+import Foundation+import Testing+@testable import HelperCore++func bundleFile(_ name: String) throws -> [String: Any] {+ let root = URL(fileURLWithPath: #filePath).deletingLastPathComponent().deletingLastPathComponent().deletingLastPathComponent()+ let data = try Data(contentsOf: root.appendingPathComponent("Bundle").appendingPathComponent(name))+ guard let plist = try PropertyListSerialization.propertyList(from: data, format: nil) as? [String: Any] else {+ throw CocoaError(.propertyListReadCorrupt)+ }+ return plist+}++@Suite struct BundleTests {+ @Test func infoPlistNamesTheHelper() throws {+ let info = try bundleFile("Info.plist")+ #expect(info["CFBundleIdentifier"] as? String == HelperIdentity.bundleId)+ #expect(info["CFBundleExecutable"] as? String == HelperIdentity.executable)+ #expect(info["CFBundleName"] as? String == HelperIdentity.appName)+ #expect(info["CFBundleDisplayName"] as? String == HelperIdentity.appName)+ #expect(info["CFBundlePackageType"] as? String == "APPL")+ #expect(info["LSUIElement"] as? Bool == true)+ #expect(info["LSMinimumSystemVersion"] as? String == "13.0")+ #expect((info["NSAppleEventsUsageDescription"] as? String)?.isEmpty == false)+ #expect(info["CFBundleShortVersionString"] is String)+ }++ @Test func backgroundAgentRunsTheBundledProgram() throws {+ let agent = try bundleFile("\(HelperIdentity.bundleId).plist")+ #expect(agent["Label"] as? String == HelperIdentity.bundleId)+ #expect(agent["BundleProgram"] as? String == "Contents/MacOS/\(HelperIdentity.executable)")+ #expect(agent["ProgramArguments"] as? [String] == [HelperIdentity.executable, "serve"])+ #expect(agent["RunAtLoad"] as? Bool == true)+ #expect(agent["Program"] == nil)+ }++ @Test func entitlementsAllowAppleEventsOnly() throws {+ let entitlements = try bundleFile("\(HelperIdentity.executable).entitlements")+ #expect(entitlements.keys.sorted() == ["com.apple.security.automation.apple-events"])+ #expect(entitlements["com.apple.security.automation.apple-events"] as? Bool == true)+ }+}diff --git a/apple/helper/Tests/HelperCoreTests/ConfigTests.swift b/apple/helper/Tests/HelperCoreTests/ConfigTests.swiftnew file mode 100644index 0000000000..eea204a12e--- /dev/null+++ b/apple/helper/Tests/HelperCoreTests/ConfigTests.swift@@ -0,0 +1,81 @@+import Foundation+import Testing+@testable import HelperCore++@Suite struct ConfigTests {+ let verifier = Token.hexDigest(token)++ func config(port: Int = 8792, tokens: [TokenEntry]? = nil, sources: [String: SourceSetting] = ["imessage": SourceSetting(enabled: true)]) -> Config {+ Config(port: port, tokens: tokens ?? [TokenEntry(name: "gbrain", verifier: verifier, scopes: ["read:imessage"])], sources: sources)+ }++ @Test func validates() throws {+ let checked = try config().validated()+ #expect(checked.port == 8792)+ #expect(checked.enabled == ["imessage"])+ #expect(checked.grants.first?.scopes == ["read:imessage"])+ #expect(checked.dbPath == Config.defaultDBPath)+ #expect(try config(sources: ["imessage": SourceSetting(enabled: false)]).validated().enabled.isEmpty)+ #expect(try config(sources: [:]).validated().enabled.isEmpty)+ #expect(try config(tokens: [TokenEntry(name: "status", verifier: verifier, scopes: [])]).validated().grants.first?.scopes.isEmpty == true)+ }++ @Test func refusesBadConfigs() {+ let bad = [+ config(port: 80),+ config(port: 70000),+ config(tokens: [TokenEntry(name: "gbrain", verifier: "abc", scopes: [])]),+ config(tokens: [TokenEntry(name: "gbrain", verifier: verifier, scopes: ["write:imessage"])]),+ config(tokens: [TokenEntry(name: "", verifier: verifier, scopes: [])]),+ config(tokens: [TokenEntry(name: "a", verifier: verifier, scopes: []), TokenEntry(name: "a", verifier: verifier, scopes: [])]),+ config(sources: ["unknown-source": SourceSetting(enabled: true)]),+ ]+ for config in bad {+ #expect(throws: ConfigError.self) { try config.validated() }+ }+ }++ @Test func hasNoFieldThatOpensANetworkListener() throws {+ let encoded = try JSONSerialization.jsonObject(with: JSONEncoder().encode(config())) as? [String: Any] ?? [:]+ #expect(Set(encoded.keys) == ["port", "tokens", "sources"])+ let older = Data(#"{"port": 8792, "tokens": [], "sources": {}, "host": "100.100.0.1", "allowed_sources": ["100.102.0.7"]}"#.utf8)+ #expect(try JSONDecoder().decode(Config.self, from: older) == Config(port: 8792))+ }++ @Test func writesPrivateFiles() throws {+ let directory = FileManager.default.temporaryDirectory.appendingPathComponent("helper-config-\(UUID().uuidString)")+ defer { try? FileManager.default.removeItem(at: directory) }+ let path = directory.appendingPathComponent("config.json").path+ try config().write(path: path)+ #expect(try Config.load(path: path) == config())+ #expect(try FileManager.default.attributesOfItem(atPath: path)[.posixPermissions] as? Int == 0o600)+ #expect(try FileManager.default.attributesOfItem(atPath: directory.path)[.posixPermissions] as? Int == 0o700)+ let text = try String(contentsOfFile: path, encoding: .utf8)+ #expect(!text.contains(token))+ try config(tokens: []).write(path: path)+ #expect(try FileManager.default.contentsOfDirectory(atPath: directory.path) == ["config.json"])+ }++ @Test func liveGrantsFollowTheConfigFile() throws {+ let directory = FileManager.default.temporaryDirectory.appendingPathComponent("helper-config-\(UUID().uuidString)")+ defer { try? FileManager.default.removeItem(at: directory) }+ let path = directory.appendingPathComponent("config.json").path+ let live = LiveGrants(path: path)+ #expect(live.current().isEmpty)+ try config().write(path: path)+ #expect(live.current().map(\.name) == ["gbrain"])+ try config(tokens: []).write(path: path)+ #expect(live.current().isEmpty)+ try config().write(path: path)+ #expect(live.current().map(\.name) == ["gbrain"])+ #expect(FileManager.default.createFile(atPath: path, contents: Data("not json".utf8)))+ #expect(live.current().isEmpty)+ try FileManager.default.removeItem(atPath: path)+ #expect(live.current().isEmpty)+ }++ @Test func defaultPathsUseTheHelperNames() {+ #expect(Config.defaultPath.hasSuffix("/Library/Application Support/GBrain Helper/config.json"))+ #expect(Log.defaultPath.hasSuffix("/Library/Logs/gbrain-apple-helper.log"))+ }+}diff --git a/apple/helper/Tests/HelperCoreTests/ReaderTests.swift b/apple/helper/Tests/HelperCoreTests/ReaderTests.swiftnew file mode 100644index 0000000000..31bc35ae9b--- /dev/null+++ b/apple/helper/Tests/HelperCoreTests/ReaderTests.swift@@ -0,0 +1,184 @@+import Foundation+import SQLite3+import Testing+@testable import HelperCore++@Suite struct ReaderTests {+ @Test func listsNewestFirstAndMergesServices() throws {+ let fixture = Fixture.standard()+ let result = try reader(fixture).listThreads(limit: 30, includeLastMessage: true)+ #expect(result.data.map(\.threadId) == ["[email protected]", "chat900001", "+15550100", "12345"])+ let person = result.data[2]+ #expect(person.chatRowids == [1, 2])+ #expect(person.participants == ["+15550100"])+ #expect(person.participantCount == 2)+ #expect(person.isGroup == false)+ #expect(person.service == "iMessage,SMS")+ #expect(person.lastMessage?.text == "sms msg")+ let group = result.data[1]+ #expect(group.isGroup)+ #expect(group.displayName == "Test Group")+ #expect(group.participantCount == 3)+ #expect(group.lastMessage?.text == "group b")+ #expect(group.lastMessage?.from == "+15550111")+ }++ @Test func keepsTextPastAnEmbeddedNull() throws {+ let fixture = Fixture.standard()+ fixture.exec("UPDATE message SET text = CAST(X'6265666F7265006166746572' AS TEXT) WHERE ROWID = 1")+ let page = try #require(try reader(fixture).getThread(.identifier("+15550100"), sinceRowid: nil, since: nil, limit: 50, includeReactions: false))+ #expect(page.data.messages[0].text == "before\u{0}after")+ }++ @Test func listLimits() throws {+ let fixture = Fixture.standard()+ #expect(try reader(fixture).listThreads(limit: 2, includeLastMessage: false).data.count == 2)+ #expect(try reader(fixture).listThreads(limit: 2, includeLastMessage: false).data[0].lastMessage == nil)+ }++ @Test func getThreadMergesDecodesAndSkipsReactions() throws {+ let fixture = Fixture.standard()+ let page = try #require(try reader(fixture).getThread(.identifier("+15550100"), sinceRowid: nil, since: nil, limit: 50, includeReactions: false))+ #expect(page.data.messages.map(\.text) == ["hi one", "reply", "from body", "sms msg"])+ #expect(page.data.messages.map(\.id) == ["GUID-1", "GUID-2", "GUID-3", "GUID-5"])+ #expect(page.data.messages[1].isFromMe)+ #expect(page.data.messages[1].from == nil)+ #expect(page.data.messages[0].from == "+15550100")+ #expect(page.data.messages[0].fromIdentifier == "+15550100")+ #expect(page.data.messages[1].fromIdentifier == nil)+ #expect(page.data.messages[2].textStatus == "ok")+ #expect(page.data.messages[3].hasAttachments)+ #expect(page.data.messages[0].timestamp == "2026-01-01T00:01:00.000Z")+ #expect(page.data.contact == "+15550100")+ #expect(page.meta.hasMore == false)+ #expect(page.meta.nextSinceRowid == 5)+ let all = try #require(try reader(fixture).getThread(.identifier("+15550100"), sinceRowid: nil, since: nil, limit: 50, includeReactions: true))+ #expect(all.data.messages.count == 5)+ }++ @Test func rowidCursorPagesOldestFirst() throws {+ let fixture = Fixture.standard()+ let first = try #require(try reader(fixture).getThread(.identifier("+15550100"), sinceRowid: 1, since: nil, limit: 2, includeReactions: false))+ #expect(first.data.messages.map(\.rowid) == [2, 3])+ #expect(first.meta.hasMore)+ #expect(first.meta.nextSinceRowid == 3)+ let second = try #require(try reader(fixture).getThread(.identifier("+15550100"), sinceRowid: 3, since: nil, limit: 2, includeReactions: false))+ #expect(second.data.messages.map(\.rowid) == [5])+ #expect(second.meta.hasMore == false)+ let empty = try #require(try reader(fixture).getThread(.identifier("+15550100"), sinceRowid: 5, since: nil, limit: 2, includeReactions: false))+ #expect(empty.data.messages.isEmpty)+ #expect(empty.meta.nextSinceRowid == 5)+ }++ @Test func newestWithoutCursorAndTimeCursor() throws {+ let fixture = Fixture.standard()+ let newest = try #require(try reader(fixture).getThread(.identifier("+15550100"), sinceRowid: nil, since: nil, limit: 2, includeReactions: false))+ #expect(newest.data.messages.map(\.rowid) == [3, 5])+ #expect(newest.meta.hasMore)+ let since = AppleTime.date(fromRaw: minutes(2))+ let page = try #require(try reader(fixture).getThread(.identifier("+15550100"), sinceRowid: nil, since: since, limit: 50, includeReactions: false))+ #expect(page.data.messages.map(\.rowid) == [3, 5])+ }++ @Test func timeCursorPagesInRowOrder() throws {+ let fixture = Fixture.standard()+ let since = AppleTime.date(fromRaw: minutes(2))+ let first = try #require(try reader(fixture).getThread(.identifier("+15550100"), sinceRowid: nil, since: since, limit: 2, includeReactions: true))+ #expect(first.data.messages.map(\.rowid) == [3, 4])+ #expect(first.meta.hasMore)+ #expect(first.meta.nextSinceRowid == 4)+ let second = try #require(try reader(fixture).getThread(.identifier("+15550100"), sinceRowid: 4, since: since, limit: 2, includeReactions: true))+ #expect(second.data.messages.map(\.rowid) == [5])+ #expect(second.meta.hasMore == false)+ }++ @Test func messageInTwoMergedChatsTakesOneSlot() throws {+ let fixture = Fixture.standard()+ fixture.exec("INSERT INTO chat_message_join (chat_id, message_id) VALUES (2, 2)")+ let page = try #require(try reader(fixture).getThread(.identifier("+15550100"), sinceRowid: 1, since: nil, limit: 2, includeReactions: false))+ #expect(page.data.messages.map(\.rowid) == [2, 3])+ #expect(page.meta.hasMore)+ }++ @Test func undecodableEditNeverFallsBackToOldText() {+ let resolved = Reader.resolveText(text: "old words", body: TypedStream.header + [0x84, 0x01, 0x40, 0xFF, 0xFF], edited: true, retracted: false)+ #expect(resolved.text == "")+ #expect(resolved.status == "undecodable")+ }++ @Test func flagsUndecodableEditedAndRetracted() throws {+ let fixture = Fixture.standard()+ let page = try #require(try reader(fixture).getThread(.identifier("[email protected]"), sinceRowid: nil, since: nil, limit: 50, includeReactions: false))+ let byRow = Dictionary(uniqueKeysWithValues: page.data.messages.map { ($0.rowid, $0) })+ #expect(byRow[10]?.textStatus == "undecodable")+ #expect(byRow[10]?.text == "")+ #expect(byRow[11]?.text == "new words")+ #expect(byRow[11]?.editedAt == "2026-01-01T00:12:00.000Z")+ #expect(byRow[12]?.retracted == true)+ #expect(byRow[12]?.text == "")+ #expect(byRow[12]?.textStatus == "empty")+ #expect(page.meta.undecodable == 1)+ }++ @Test func selectorsKeepShortCodesApartFromRowids() throws {+ let fixture = Fixture.standard()+ let byIdentifier = try #require(try reader(fixture).getThread(.identifier("12345"), sinceRowid: nil, since: nil, limit: 50, includeReactions: false))+ #expect(byIdentifier.data.messages.map(\.text) == ["code 1234"])+ #expect(byIdentifier.data.messages[0].timestamp == "2025-12-31T23:50:00.000Z")+ #expect(try reader(fixture).getThread(.chatRowid(12345), sinceRowid: nil, since: nil, limit: 50, includeReactions: false) == nil)+ #expect(try reader(fixture).getThread(.chatRowid(4), sinceRowid: nil, since: nil, limit: 50, includeReactions: false)?.data.threadId == "12345")+ #expect(try reader(fixture).getThread(.chatGuid("SMS;-;12345"), sinceRowid: nil, since: nil, limit: 50, includeReactions: false)?.data.threadId == "12345")+ #expect(try reader(fixture).getThread(.identifier("[email protected]"), sinceRowid: nil, since: nil, limit: 50, includeReactions: false) != nil)+ #expect(try reader(fixture).getThread(.identifier("nobody"), sinceRowid: nil, since: nil, limit: 50, includeReactions: false) == nil)+ }++ @Test func seesRowsStillInTheWriteAheadLog() throws {+ let fixture = Fixture.standard(wal: true)+ fixture.message(13, chat: 3, text: "fresh in wal", date: minutes(20), handle: 3)+ let wal = try FileManager.default.attributesOfItem(atPath: fixture.path + "-wal")[.size] as? Int ?? 0+ #expect(wal > 0)+ let page = try #require(try reader(fixture).getThread(.identifier("chat900001"), sinceRowid: 7, since: nil, limit: 50, includeReactions: false))+ #expect(page.data.messages.map(\.text) == ["fresh in wal"])+ }++ @Test func connectionCannotWrite() throws {+ let fixture = Fixture.standard()+ let db = try ChatDB.open(path: fixture.path)+ #expect(throws: DBError.self) { try db.query("INSERT INTO handle (id) VALUES ('x')", []) { _ in } }+ #expect(throws: DBError.self) { try db.query("PRAGMA query_only = 0", []) { _ in }; try db.query("DELETE FROM message", []) { _ in } }+ }++ @Test func legacySchemaStillWorks() throws {+ let fixture = Fixture(legacy: true)+ fixture.handle(1, "+15550100")+ fixture.chat(1, guid: "iMessage;-;+15550100", identifier: "+15550100", handles: [1])+ fixture.message(1, chat: 1, text: "plain", date: minutes(1), handle: 1)+ let r = try reader(fixture)+ try r.healthCheck()+ #expect(try r.listThreads(limit: 5, includeLastMessage: true).data.first?.lastMessage?.text == "plain")+ }++ @Test func reportsSchemaMismatchMissingAndNoAccess() throws {+ let broken = Fixture(skipTable: "chat_handle_join")+ #expect(throws: DBError.schemaMismatch("table chat_handle_join is missing")) { try ChatDB.open(path: broken.path).validateSchema() }+ #expect(throws: DBError.missing) { try ChatDB.open(path: "/nonexistent/chat.db") }+ let locked = Fixture.standard()+ locked.close()+ chmod(locked.path, 0)+ defer { chmod(locked.path, 0o644) }+ #expect(throws: DBError.noAccess) { try ChatDB.open(path: locked.path) }+ }++ @Test func statsCountOnly() throws {+ let fixture = Fixture.standard()+ let stats = try reader(fixture).stats()+ #expect(stats.threads == 4)+ #expect(stats.threadIdShapes == ["plus_digits": 1, "chat_digits": 1, "short_code": 1, "email": 1])+ #expect(stats.recentMessagesScanned == 10)+ #expect(stats.undecodable == 1)+ #expect(stats.decodedFromBody == 1)+ #expect(stats.edited == 1)+ #expect(stats.editedTextDiffers == 1)+ #expect(stats.retracted == 1)+ }+}diff --git a/apple/helper/Tests/HelperCoreTests/RouterTests.swift b/apple/helper/Tests/HelperCoreTests/RouterTests.swiftnew file mode 100644index 0000000000..04fc3f6899--- /dev/null+++ b/apple/helper/Tests/HelperCoreTests/RouterTests.swift@@ -0,0 +1,286 @@+import Foundation+import Testing+@testable import HelperCore++let loopbackPeer = IPv4.parse("127.0.0.1")!+let tailnetPeer = IPv4.parse("100.102.0.7")!+let macTailnet = IPv4.parse("100.100.0.1")!+let strangerPeer = IPv4.parse("100.90.0.9")!+let token = "synthetic-test-token"+let statusToken = "synthetic-status-token"+let testPort: UInt16 = 8792+let goodHost = "127.0.0.1:8792"++func grants() -> [TokenGrant] {+ [TokenGrant(name: "reader", verifier: Token.verifier(fromHex: Token.hexDigest(token))!, scopes: [SourceCatalog.readScope("imessage")]),+ TokenGrant(name: "status", verifier: Token.verifier(fromHex: Token.hexDigest(statusToken))!, scopes: [])]+}++func router(dbPath: String, listener: Listener = .loopback, enabled: Set<String> = ["imessage"]) -> Router {+ let settings = RouterSettings(listener: listener, grants: grants(), sources: [IMessageSource(dbPath: dbPath)], enabled: enabled, version: "test")+ return Router(settings: settings, localAddresses: { [macTailnet] })+}++func request(_ target: String, auth: String? = "Bearer \(token)", host: String? = goodHost, origin: String? = nil, method: String = "GET") -> HTTPRequest {+ var raw = "\(method) \(target) HTTP/1.1\r\n"+ if let host { raw += "Host: \(host)\r\n" }+ if let origin { raw += "Origin: \(origin)\r\n" }+ if let auth { raw += "Authorization: \(auth)\r\n" }+ raw += "\r\n"+ guard case .request(let parsed) = HTTPParser.parse(Array(raw.utf8), maxBytes: 8192) else { fatalError("unparsed") }+ return parsed+}++func send(_ router: Router, _ request: HTTPRequest, peer: UInt32 = loopbackPeer) -> Routed {+ router.handle(request, peer: peer, port: testPort, requestId: "t")+}++func json(_ routed: Routed) -> [String: Any] {+ (try? JSONSerialization.jsonObject(with: Data(routed.response.body))) as? [String: Any] ?? [:]+}++func code(_ routed: Routed) -> String? {+ json(routed)["code"] as? String+}++func sources(_ routed: Routed) -> [[String: Any]] {+ json(routed)["data"] as? [[String: Any]] ?? []+}++@Suite struct LoopbackListenerTests {+ @Test func healthCarriesOnlyOkAndVersion() {+ let routed = send(router(dbPath: "/nonexistent/chat.db"), request("/health", auth: nil))+ #expect(routed.response.status == 200)+ #expect(Set(json(routed).keys) == ["ok", "version"])+ #expect(json(routed)["ok"] as? Bool == true)+ }++ @Test func admitsOnlyLoopbackPeers() {+ let r = router(dbPath: "/nonexistent/chat.db")+ #expect(r.admits(loopbackPeer))+ #expect(r.admits(IPv4.parse("127.0.0.2")!))+ for peer in [tailnetPeer, macTailnet, strangerPeer, IPv4.parse("192.168.1.5")!] {+ #expect(!r.admits(peer))+ #expect(code(send(r, request("/health", auth: nil), peer: peer)) == "forbidden_source")+ }+ }++ @Test func refusesEveryOtherHost() {+ let fixture = Fixture.standard()+ let r = router(dbPath: fixture.path)+ let hosts: [String?] = ["localhost:8792", "127.0.0.1", "127.0.0.1:8793", "attacker.example:8792", "[::1]:8792", "127.0.0.1:8792.attacker.example", "", nil]+ for host in hosts {+ for target in ["/health", "/v1/sources", "/v1/imessage/threads"] {+ let routed = send(r, request(target, host: host))+ #expect(routed.response.status == 403, "\(target) \(host ?? "no host")")+ #expect(code(routed) == "forbidden_host")+ }+ }+ }++ @Test func refusesAnyOriginAndSendsNoCORSHeaders() {+ let fixture = Fixture.standard()+ let r = router(dbPath: fixture.path)+ for origin in ["http://127.0.0.1:8792", "null", "https://attacker.example", ""] {+ for target in ["/health", "/v1/imessage/threads"] {+ let routed = send(r, request(target, origin: origin))+ #expect(routed.response.status == 403, "\(target) \(origin)")+ #expect(code(routed) == "forbidden_origin")+ }+ }+ let served = send(r, request("/v1/imessage/threads"))+ #expect(served.response.status == 200)+ #expect(!String(decoding: served.response.serialize(requestId: "t"), as: UTF8.self).lowercased().contains("access-control"))+ }++ @Test func checksTokens() {+ let fixture = Fixture.standard()+ let r = router(dbPath: fixture.path)+ #expect(send(r, request("/v1/imessage/threads", auth: nil)).response.status == 401)+ #expect(send(r, request("/v1/imessage/threads", auth: "Bearer wrong")).response.status == 401)+ #expect(send(r, request("/v1/imessage/threads", auth: "Basic \(token)")).response.status == 401)+ #expect(send(r, request("/v1/imessage/threads", auth: "Bearer")).response.status == 401)+ #expect(send(r, request("/v1/sources", auth: nil)).response.status == 401)+ #expect(send(r, request("/v1/imessage/threads")).response.status == 200)+ }++ @Test func dataRoutesNeedTheSourceScope() {+ let fixture = Fixture.standard()+ let r = router(dbPath: fixture.path)+ let routed = send(r, request("/v1/imessage/threads", auth: "Bearer \(statusToken)"))+ #expect(routed.response.status == 403)+ #expect(code(routed) == "insufficient_scope")+ #expect(send(r, request("/v1/sources", auth: "Bearer \(statusToken)")).response.status == 200)+ }++ @Test func disabledSourceIsRefusedAndNeverOpened() {+ let r = router(dbPath: "/nonexistent/chat.db", enabled: [])+ for target in ["/v1/imessage/threads", "/v1/imessage/thread?thread_id=%2B15550100"] {+ let routed = send(r, request(target))+ #expect(routed.response.status == 403)+ #expect(code(routed) == "source_disabled")+ }+ let listed = sources(send(r, request("/v1/sources")))+ #expect(listed.first?["enabled"] as? Bool == false)+ #expect(listed.first?["state"] as? String == "disabled")+ }++ @Test func sourcesListModulesWithoutData() {+ let fixture = Fixture.standard()+ let routed = send(router(dbPath: fixture.path), request("/v1/sources"))+ #expect(routed.response.status == 200)+ #expect(routed.rows == 0)+ let listed = sources(routed)+ #expect(listed.count == 1)+ #expect(listed.first?["id"] as? String == "imessage")+ #expect(listed.first?["enabled"] as? Bool == true)+ #expect(listed.first?["permission"] as? String == "full_disk_access")+ #expect(listed.first?["state"] as? String == "ok")+ #expect(listed.first?["scopes"] as? [String] == ["read:imessage"])+ #expect(listed.first?["routes"] as? [String] == ["/v1/imessage/threads", "/v1/imessage/thread"])+ let text = String(decoding: routed.response.body, as: UTF8.self)+ for data in ["15550100", "example.invalid", "Test Group", "hi one"] {+ #expect(!text.contains(data))+ }+ }++ @Test func listThreadsShape() {+ let fixture = Fixture.standard()+ let routed = send(router(dbPath: fixture.path), request("/v1/imessage/threads?limit=3"))+ let body = json(routed)+ let data = body["data"] as? [[String: Any]] ?? []+ #expect(data.count == 3)+ #expect(data.first?["thread_id"] as? String == "[email protected]")+ #expect((data.first?["last_message"] as? [String: Any])?["is_from_me"] as? Bool == false)+ #expect((body["meta"] as? [String: Any])?["count"] as? Int == 3)+ #expect(routed.rows == 3)+ #expect(routed.route == "/v1/imessage/threads")+ }++ @Test func plusInHandlesSurvivesRawAndEncoded() {+ let fixture = Fixture.standard()+ let r = router(dbPath: fixture.path)+ for target in ["/v1/imessage/thread?thread_id=+15550100", "/v1/imessage/thread?thread_id=%2B15550100"] {+ let routed = send(r, request(target))+ #expect(routed.response.status == 200)+ #expect(((json(routed)["data"] as? [String: Any])?["messages"] as? [Any])?.count == 4)+ }+ }++ @Test func sinceRowidPagesForward() {+ let fixture = Fixture.standard()+ let routed = send(router(dbPath: fixture.path), request("/v1/imessage/thread?thread_id=%2B15550100&since_rowid=2&limit=1"))+ let messages = (json(routed)["data"] as? [String: Any])?["messages"] as? [[String: Any]] ?? []+ #expect(messages.map { $0["rowid"] as? Int } == [3])+ #expect((json(routed)["meta"] as? [String: Any])?["next_since_rowid"] as? Int == 3)+ #expect((json(routed)["meta"] as? [String: Any])?["has_more"] as? Bool == true)+ }++ @Test func rejectsBadParameters() {+ let fixture = Fixture.standard()+ let r = router(dbPath: fixture.path)+ for target in ["/v1/imessage/thread", "/v1/imessage/thread?thread_id=a&chat_rowid=1", "/v1/imessage/thread?thread_id=",+ "/v1/imessage/thread?thread_id=x&limit=0", "/v1/imessage/thread?thread_id=x&limit=501",+ "/v1/imessage/thread?thread_id=x&since_iso=soon", "/v1/imessage/thread?thread_id=x&since_rowid=-1",+ "/v1/imessage/thread?chat_rowid=abc", "/v1/imessage/threads?limit=201", "/v1/imessage/threads?include_last_message=maybe"] {+ let routed = send(r, request(target))+ #expect(routed.response.status == 400, "\(target)")+ #expect(code(routed) == "bad_param")+ }+ }++ @Test func notFoundNeverEchoesTheHandle() {+ let fixture = Fixture.standard()+ let routed = router(dbPath: fixture.path).handle(request("/v1/imessage/thread?thread_id=%2B15559999"), peer: loopbackPeer, port: testPort, requestId: "rid1")+ #expect(routed.response.status == 404)+ #expect(!String(decoding: routed.response.body, as: UTF8.self).contains("15559999"))+ #expect(json(routed)["request_id"] as? String == "rid1")+ #expect(json(routed)["type"] as? String == Problem.docBase + "not_found")+ #expect(routed.response.contentType == "application/problem+json")+ }++ @Test func otherMethodsAndRoutes() {+ let fixture = Fixture.standard()+ let r = router(dbPath: fixture.path)+ #expect(send(r, request("/v1/imessage/threads", method: "POST")).response.status == 405)+ #expect(send(r, request("/v1/imessage/send")).response.status == 404)+ #expect(send(r, request("/v1/imessage/send", auth: nil)).response.status == 401)+ #expect(send(r, request("/v1/list_threads")).response.status == 404)+ #expect(send(r, request("/v1/imessage/send")).route == "unknown")+ }++ @Test func explainsMissingFullDiskAccess() {+ let fixture = Fixture.standard()+ fixture.close()+ chmod(fixture.path, 0)+ defer { chmod(fixture.path, 0o644) }+ let r = router(dbPath: fixture.path)+ let routed = send(r, request("/v1/imessage/threads"))+ #expect(routed.response.status == 503)+ let body = json(routed)+ #expect(body["code"] as? String == "no_access")+ #expect(body["retryable"] as? Bool == false)+ #expect((body["fix"] as? String)?.contains("Full Disk Access") == true)+ #expect((body["fix"] as? String)?.contains(HelperIdentity.bundleId) == true)+ #expect(sources(send(r, request("/v1/sources"))).first?["state"] as? String == "no_access")+ #expect(send(r, request("/health", auth: nil)).response.status == 200)+ let busy = HTTPResponse.problem(.dbBusy, requestId: "t")+ #expect(busy.extraHeaders.contains { $0.0 == "Retry-After" && $0.1 == "2" })+ }+}++@Suite struct TailnetListenerTests {+ @Test func admitsOnlyListedPeersThatAreNotThisMac() {+ let r = router(dbPath: "/nonexistent/chat.db", listener: .tailnet(allowed: [tailnetPeer, macTailnet, loopbackPeer]))+ #expect(r.admits(tailnetPeer))+ #expect(!r.admits(macTailnet))+ #expect(!r.admits(loopbackPeer))+ #expect(!r.admits(strangerPeer))+ }++ @Test func servesListedPeersWithAToken() {+ let fixture = Fixture.standard()+ let r = router(dbPath: fixture.path, listener: .tailnet(allowed: [tailnetPeer]))+ let host = "100.100.0.1:8792"+ #expect(send(r, request("/health", auth: nil, host: host), peer: tailnetPeer).response.status == 200)+ #expect(send(r, request("/v1/imessage/threads", host: host), peer: tailnetPeer).response.status == 200)+ #expect(send(r, request("/v1/imessage/threads", auth: nil, host: host), peer: tailnetPeer).response.status == 401)+ for peer in [macTailnet, loopbackPeer, strangerPeer] {+ #expect(code(send(r, request("/health", auth: nil), peer: peer)) == "forbidden_source")+ }+ }++ @Test func bindPolicyNeedsATailnetUtunAddress() {+ let host = IPv4.parse("100.100.0.1")!+ #expect(BindPolicy.problem(host: host, interfaces: [host: "utun4"]) == nil)+ #expect(BindPolicy.problem(host: host, interfaces: [host: "en0"]) != nil)+ #expect(BindPolicy.problem(host: host, interfaces: [:]) != nil)+ #expect(BindPolicy.problem(host: IPv4.parse("0.0.0.0")!, interfaces: [:]) != nil)+ #expect(BindPolicy.problem(host: IPv4.parse("192.168.1.5")!, interfaces: [IPv4.parse("192.168.1.5")!: "utun1"]) != nil)+ }+}++@Suite struct TokenTests {+ @Test func picksTheMatchingGrant() {+ let all = grants()+ #expect(Token.grant(authorization: "Bearer \(token)", grants: all)?.name == "reader")+ #expect(Token.grant(authorization: "bearer \(statusToken)", grants: all)?.name == "status")+ #expect(Token.grant(authorization: "Bearer \(token)x", grants: all) == nil)+ #expect(Token.grant(authorization: nil, grants: all) == nil)+ #expect(Token.grant(authorization: "Bearer \(token)", grants: []) == nil)+ #expect(Token.verifier(fromHex: "zz" + String(repeating: "0", count: 62)) == nil)+ #expect(Token.verifier(fromHex: Token.hexDigest(token).uppercased()) == Token.verifier(fromHex: Token.hexDigest(token)))+ }+}++@Suite struct LiveGrantTests {+ @Test func aRemovedTokenStopsWorkingOnTheNextRequest() {+ let fixture = Fixture.standard()+ var current = grants()+ let settings = RouterSettings(listener: .loopback, liveGrants: { current }, sources: [IMessageSource(dbPath: fixture.path)], enabled: ["imessage"], version: "test")+ let r = Router(settings: settings, localAddresses: { [macTailnet] })+ #expect(send(r, request("/v1/sources")).response.status == 200)+ current = []+ #expect(code(send(r, request("/v1/sources"))) == "unauthorized")+ }+}diff --git a/apple/helper/Tests/HelperCoreTests/ServerTests.swift b/apple/helper/Tests/HelperCoreTests/ServerTests.swiftnew file mode 100644index 0000000000..25ac8d9623--- /dev/null+++ b/apple/helper/Tests/HelperCoreTests/ServerTests.swift@@ -0,0 +1,102 @@+import Darwin+import Foundation+import Testing+@testable import HelperCore++func connect(_ port: UInt16) -> Int32 {+ let fd = socket(AF_INET, SOCK_STREAM, 0)+ var address = sockaddr_in()+ address.sin_len = UInt8(MemoryLayout<sockaddr_in>.size)+ address.sin_family = sa_family_t(AF_INET)+ address.sin_port = port.bigEndian+ address.sin_addr = in_addr(s_addr: IPv4.parse("127.0.0.1")!.bigEndian)+ let rc = withUnsafePointer(to: &address) { $0.withMemoryRebound(to: sockaddr.self, capacity: 1) { Darwin.connect(fd, $0, socklen_t(MemoryLayout<sockaddr_in>.size)) } }+ precondition(rc == 0)+ var timeout = timeval(tv_sec: 5, tv_usec: 0)+ setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, &timeout, socklen_t(MemoryLayout<timeval>.size))+ return fd+}++func sendText(_ fd: Int32, _ text: String) {+ let bytes = Array(text.utf8)+ _ = bytes.withUnsafeBytes { Darwin.send(fd, $0.baseAddress, $0.count, 0) }+}++func readAll(_ fd: Int32) -> String {+ var result: [UInt8] = []+ var chunk = [UInt8](repeating: 0, count: 4096)+ while true {+ let count = recv(fd, &chunk, chunk.count, 0)+ if count <= 0 { break }+ result.append(contentsOf: chunk[0..<count])+ }+ close(fd)+ return String(decoding: result, as: UTF8.self)+}++func status(_ response: String) -> Int {+ Int(response.split(separator: " ").dropFirst().first ?? "") ?? 0+}++func get(_ port: UInt16, _ target: String, host: String? = nil, extra: String = "") -> String {+ let fd = connect(port)+ sendText(fd, "GET \(target) HTTP/1.1\r\nHost: \(host ?? "127.0.0.1:\(port)")\r\n\(extra)\r\n")+ return readAll(fd)+}++@Suite(.serialized) struct ServerTests {+ func running(_ fixture: Fixture, timeout: TimeInterval = 10) throws -> Server {+ let server = Server(host: loopbackPeer, port: 0, router: router(dbPath: fixture.path), maxConcurrent: 4, ioTimeout: timeout)+ try server.start()+ Thread.detachNewThread { server.serveForever() }+ return server+ }++ @Test func servesLoopbackRequestsAddressedToItsOwnPort() throws {+ let fixture = Fixture.standard()+ let server = try running(fixture)+ defer { server.stop() }+ let port = server.boundPort+ let health = get(port, "/health")+ #expect(status(health) == 200)+ #expect(health.contains("Cache-Control: no-store"))+ #expect(!health.lowercased().contains("access-control"))+ #expect(status(get(port, "/v1/imessage/threads", extra: "Authorization: Bearer \(token)\r\n")) == 200)+ #expect(status(get(port, "/v1/imessage/threads", extra: "Authorization: Bearer \(token)\r\nOrigin: http://attacker.example\r\n")) == 403)+ #expect(status(get(port, "/health", host: "localhost:\(port)")) == 403)+ #expect(status(get(port, "/health", host: "127.0.0.1:\(port &+ 1)")) == 403)+ }++ @Test func refusesTheFifthConcurrentRequest() throws {+ let fixture = Fixture.standard()+ let server = try running(fixture, timeout: 3)+ defer { server.stop() }+ let held = (0..<4).map { _ in connect(server.boundPort) }+ for fd in held { sendText(fd, "GET /health HTTP/1.1\r\n") }+ Thread.sleep(forTimeInterval: 0.5)+ let fifth = connect(server.boundPort)+ sendText(fifth, "GET /health HTTP/1.1\r\n\r\n")+ let response = readAll(fifth)+ #expect(status(response) == 503)+ #expect(response.contains("overloaded"))+ #expect(response.contains("Retry-After: 1"))+ for fd in held { close(fd) }+ }++ @Test func dropsSlowClientsAndOversizedRequests() throws {+ let fixture = Fixture.standard()+ let server = try running(fixture, timeout: 1)+ defer { server.stop() }+ let slow = connect(server.boundPort)+ sendText(slow, "GET /health HTTP/1.1\r\n")+ let started = Date()+ #expect(readAll(slow).isEmpty)+ #expect(Date().timeIntervalSince(started) < 3)+ let big = connect(server.boundPort)+ sendText(big, "GET /health?" + String(repeating: "a", count: 8192 - 12))+ #expect(status(readAll(big)) == 413)+ let bad = connect(server.boundPort)+ sendText(bad, "NONSENSE\r\n\r\n")+ #expect(status(readAll(bad)) == 400)+ }+}diff --git a/apple/helper/Tests/HelperCoreTests/Support.swift b/apple/helper/Tests/HelperCoreTests/Support.swiftnew file mode 100644index 0000000000..10bd695271--- /dev/null+++ b/apple/helper/Tests/HelperCoreTests/Support.swift@@ -0,0 +1,163 @@+import Foundation+import SQLite3+@testable import HelperCore++let transient = unsafeBitCast(-1, to: sqlite3_destructor_type.self)+let base = AppleTime.raw(from: AppleTime.parseISO("2026-01-01T00:00:00Z")!)++func minutes(_ n: Int64) -> Int64 { base + n * 60_000_000_000 }++func archive(_ string: String, attributes: Bool = false) -> [UInt8] {+ let value: NSAttributedString = attributes+ ? NSMutableAttributedString(string: string, attributes: [NSAttributedString.Key("__kIMMessagePartAttributeName"): NSNumber(value: 0)])+ : NSAttributedString(string: string)+ return Array(NSArchiver.archivedData(withRootObject: value))+}++func hex(_ string: String) -> [UInt8] {+ var bytes: [UInt8] = []+ var index = string.startIndex+ while index < string.endIndex {+ let next = string.index(index, offsetBy: 2)+ bytes.append(UInt8(string[index..<next], radix: 16)!)+ index = next+ }+ return bytes+}++struct SplitMix: RandomNumberGenerator {+ var state: UInt64+ mutating func next() -> UInt64 {+ state &+= 0x9E37_79B9_7F4A_7C15+ var z = state+ z = (z ^ (z >> 30)) &* 0xBF58_476D_1CE4_E5B9+ z = (z ^ (z >> 27)) &* 0x94D0_49BB_1331_11EB+ return z ^ (z >> 31)+ }+}++final class Fixture {+ let directory: URL+ let path: String+ var db: OpaquePointer?++ init(wal: Bool = false, legacy: Bool = false, skipTable: String? = nil) {+ directory = FileManager.default.temporaryDirectory.appendingPathComponent("helper-fixture-\(UUID().uuidString)")+ try! FileManager.default.createDirectory(at: directory, withIntermediateDirectories: true)+ path = directory.appendingPathComponent("chat.db").path+ precondition(sqlite3_open(path, &db) == SQLITE_OK)+ if wal { exec("PRAGMA journal_mode=WAL; PRAGMA wal_autocheckpoint=0;") }+ let messageExtra = legacy ? "" : ", attributedBody BLOB, associated_message_type INTEGER DEFAULT 0, item_type INTEGER DEFAULT 0, cache_has_attachments INTEGER DEFAULT 0, date_edited INTEGER DEFAULT 0, date_retracted INTEGER DEFAULT 0"+ let joinExtra = legacy ? "" : ", message_date INTEGER DEFAULT 0"+ let tables = [+ "handle": "CREATE TABLE handle (ROWID INTEGER PRIMARY KEY AUTOINCREMENT UNIQUE, id TEXT NOT NULL, service TEXT)",+ "chat": "CREATE TABLE chat (ROWID INTEGER PRIMARY KEY AUTOINCREMENT, guid TEXT UNIQUE NOT NULL, style INTEGER, chat_identifier TEXT, service_name TEXT, display_name TEXT)",+ "chat_handle_join": "CREATE TABLE chat_handle_join (chat_id INTEGER, handle_id INTEGER, UNIQUE(chat_id, handle_id))",+ "message": "CREATE TABLE message (ROWID INTEGER PRIMARY KEY AUTOINCREMENT, guid TEXT UNIQUE NOT NULL, text TEXT, handle_id INTEGER DEFAULT 0, date INTEGER, is_from_me INTEGER DEFAULT 0\(messageExtra))",+ "chat_message_join": "CREATE TABLE chat_message_join (chat_id INTEGER, message_id INTEGER\(joinExtra), PRIMARY KEY (chat_id, message_id))",+ ]+ for (name, sql) in tables where name != skipTable { exec(sql) }+ }++ deinit {+ close()+ try? FileManager.default.removeItem(at: directory)+ }++ func close() {+ if let db { sqlite3_close(db) }+ db = nil+ }++ func exec(_ sql: String) {+ var error: UnsafeMutablePointer<CChar>?+ let rc = sqlite3_exec(db, sql, nil, nil, &error)+ precondition(rc == SQLITE_OK, error.map { String(cString: $0) } ?? "exec failed")+ }++ func handle(_ rowid: Int64, _ id: String) {+ exec("INSERT INTO handle (ROWID, id, service) VALUES (\(rowid), '\(id)', 'iMessage')")+ }++ func chat(_ rowid: Int64, guid: String, identifier: String, name: String? = nil, service: String = "iMessage", style: Int = 45, handles: [Int64]) {+ let display = name.map { "'\($0)'" } ?? "NULL"+ exec("INSERT INTO chat (ROWID, guid, style, chat_identifier, service_name, display_name) VALUES (\(rowid), '\(guid)', \(style), '\(identifier)', '\(service)', \(display))")+ for handle in handles { exec("INSERT INTO chat_handle_join (chat_id, handle_id) VALUES (\(rowid), \(handle))") }+ }++ @discardableResult+ func message(_ rowid: Int64, chat: Int64, text: String?, body: [UInt8]? = nil, date: Int64, fromMe: Bool = false, handle: Int64 = 0,+ association: Int = 0, itemType: Int = 0, attachments: Bool = false, edited: Int64 = 0, retracted: Int64 = 0) -> Int64 {+ let legacy = !columns("message").contains("attributedBody")+ let sql = legacy+ ? "INSERT INTO message (ROWID, guid, text, handle_id, date, is_from_me) VALUES (?, ?, ?, ?, ?, ?)"+ : "INSERT INTO message (ROWID, guid, text, handle_id, date, is_from_me, attributedBody, associated_message_type, item_type, cache_has_attachments, date_edited, date_retracted) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"+ var statement: OpaquePointer?+ precondition(sqlite3_prepare_v2(db, sql, -1, &statement, nil) == SQLITE_OK)+ sqlite3_bind_int64(statement, 1, rowid)+ sqlite3_bind_text(statement, 2, "GUID-\(rowid)", -1, transient)+ if let text { sqlite3_bind_text(statement, 3, text, -1, transient) } else { sqlite3_bind_null(statement, 3) }+ sqlite3_bind_int64(statement, 4, handle)+ sqlite3_bind_int64(statement, 5, date)+ sqlite3_bind_int64(statement, 6, fromMe ? 1 : 0)+ if !legacy {+ if let body { _ = body.withUnsafeBytes { sqlite3_bind_blob(statement, 7, $0.baseAddress, Int32(body.count), transient) } } else { sqlite3_bind_null(statement, 7) }+ sqlite3_bind_int64(statement, 8, Int64(association))+ sqlite3_bind_int64(statement, 9, Int64(itemType))+ sqlite3_bind_int64(statement, 10, attachments ? 1 : 0)+ sqlite3_bind_int64(statement, 11, edited)+ sqlite3_bind_int64(statement, 12, retracted)+ }+ precondition(sqlite3_step(statement) == SQLITE_DONE)+ sqlite3_finalize(statement)+ let joinDate = columns("chat_message_join").contains("message_date")+ exec(joinDate+ ? "INSERT INTO chat_message_join (chat_id, message_id, message_date) VALUES (\(chat), \(rowid), \(date))"+ : "INSERT INTO chat_message_join (chat_id, message_id) VALUES (\(chat), \(rowid))")+ return rowid+ }++ func columns(_ table: String) -> Set<String> {+ var statement: OpaquePointer?+ var names = Set<String>()+ guard sqlite3_prepare_v2(db, "PRAGMA table_info(\(table))", -1, &statement, nil) == SQLITE_OK else { return names }+ while sqlite3_step(statement) == SQLITE_ROW { names.insert(String(cString: sqlite3_column_text(statement, 1))) }+ sqlite3_finalize(statement)+ return names+ }++ static func standard(wal: Bool = false) -> Fixture {+ let fixture = Fixture(wal: wal)+ fixture.populate()+ return fixture+ }++ func populate() {+ handle(1, "+15550100")+ handle(2, "[email protected]")+ handle(3, "+15550111")+ handle(4, "12345")+ chat(1, guid: "iMessage;-;+15550100", identifier: "+15550100", handles: [1])+ chat(2, guid: "SMS;-;+15550100", identifier: "+15550100", service: "SMS", handles: [1])+ chat(3, guid: "iMessage;+;chat900001", identifier: "chat900001", name: "Test Group", style: 43, handles: [2, 3])+ chat(4, guid: "SMS;-;12345", identifier: "12345", service: "SMS", handles: [4])+ chat(5, guid: "iMessage;-;[email protected]", identifier: "[email protected]", handles: [2])++ message(1, chat: 1, text: "hi one", date: minutes(1), handle: 1)+ message(2, chat: 1, text: "reply", date: minutes(2), fromMe: true, handle: 1)+ message(3, chat: 1, text: nil, body: archive("from body"), date: minutes(3), handle: 1)+ message(4, chat: 1, text: "Loved a message", date: minutes(5), handle: 1, association: 2000)+ message(5, chat: 2, text: "sms msg", date: minutes(4), handle: 1, attachments: true)+ message(6, chat: 3, text: "group a", date: minutes(6), handle: 2)+ message(7, chat: 3, text: "group b", date: minutes(7), handle: 3)+ message(8, chat: 3, text: nil, date: minutes(8), handle: 2, itemType: 2)+ message(9, chat: 4, text: "code 1234", date: (base / 1_000_000_000) - 600, handle: 4)+ message(10, chat: 5, text: nil, body: TypedStream.header + [0x84, 0x01, 0x40, 0xFF, 0xFF], date: minutes(9), handle: 2)+ message(11, chat: 5, text: "old words", body: archive("new words"), date: minutes(10), handle: 2, edited: minutes(12))+ message(12, chat: 5, text: "gone", date: minutes(11), handle: 2, retracted: minutes(13))+ }+}++func reader(_ fixture: Fixture) throws -> Reader {+ Reader(db: try ChatDB.open(path: fixture.path))+}diff --git a/apple/helper/Tests/HelperCoreTests/TypedStreamTests.swift b/apple/helper/Tests/HelperCoreTests/TypedStreamTests.swiftnew file mode 100644index 0000000000..bfdd9fc082--- /dev/null+++ b/apple/helper/Tests/HelperCoreTests/TypedStreamTests.swift@@ -0,0 +1,76 @@+import Foundation+import Testing+@testable import HelperCore++@Suite struct TypedStreamTests {+ static let hello = hex("040b73747265616d747970656481e803840140848484124e5341747472696275746564537472696e67008484084e534f626a656374008592848484084e53537472696e67019484012b0548656c6c6f86")++ @Test func decodesKnownVector() {+ #expect(TypedStream.decodeText(Self.hello) == .text("Hello"))+ }++ @Test(arguments: [0, 1, 127, 128, 255, 256, 32767, 32768, 70000])+ func decodesArchivedLengths(length: Int) {+ let string = String(repeating: "a", count: length)+ #expect(TypedStream.decodeText(archive(string)) == .text(string))+ #expect(TypedStream.decodeText(archive(string, attributes: true)) == .text(string))+ }++ @Test func decodesUnicode() {+ for string in ["héllo wörld", "👋🏽 hi there", "日本語のテキスト", "line one\nline two", "\u{FFFC}with attachment"] {+ #expect(TypedStream.decodeText(archive(string)) == .text(string))+ }+ }++ @Test func agreesWithFoundation() {+ for string in ["short", String(repeating: "é", count: 300), "🙂🙃"] {+ let blob = archive(string, attributes: true)+ let foundation = (NSUnarchiver.unarchiveObject(with: Data(blob)) as? NSAttributedString)?.string+ #expect(TypedStream.decodeText(blob) == .text(foundation ?? "<nil>"))+ }+ }++ @Test func rejectsGarbage() {+ #expect(TypedStream.decodeText([]) == .undecodable)+ #expect(TypedStream.decodeText(TypedStream.header) == .undecodable)+ #expect(TypedStream.decodeText(Array("not a typedstream".utf8)) == .undecodable)+ #expect(TypedStream.decodeText(TypedStream.header + [0x08] + Array("NSString".utf8) + [0x01, 0x94, 0x84, 0x01, 0x2B, 0x7F]) == .undecodable)+ #expect(TypedStream.decodeText(TypedStream.header + [0x08] + Array("NSString".utf8) + [0x01, 0x94, 0x84, 0x01, 0x2B, 0x82, 0xFF, 0xFF, 0xFF, 0xFF]) == .undecodable)+ #expect(TypedStream.decodeText(TypedStream.header + [0x08] + Array("NSString".utf8) + [0x01, 0x94, 0x84, 0x01, 0x2B, 0x02, 0xC3, 0x28]) == .undecodable)+ }++ @Test func survivesTruncationAndMutation() {+ let blob = archive("a message long enough to matter, with ünïcödé", attributes: true)+ for end in 0..<blob.count { _ = TypedStream.decodeText(Array(blob[0..<end])) }+ var random = SplitMix(state: 42)+ for _ in 0..<5000 {+ var mutated = blob+ for _ in 0..<Int.random(in: 1...4, using: &random) {+ mutated[Int.random(in: 0..<mutated.count, using: &random)] = UInt8.random(in: 0...255, using: &random)+ }+ _ = TypedStream.decodeText(mutated)+ }+ for _ in 0..<2000 {+ let tail = (0..<Int.random(in: 0...64, using: &random)).map { _ in UInt8.random(in: 0...255, using: &random) }+ _ = TypedStream.decodeText(TypedStream.header + tail)+ }+ }+}++@Suite struct AppleTimeTests {+ @Test func convertsNanosecondsAndSeconds() {+ #expect(AppleTime.iso(fromRaw: 788_918_400_000_000_000) == "2026-01-01T00:00:00.000Z")+ #expect(AppleTime.iso(fromRaw: 788_918_400) == "2026-01-01T00:00:00.000Z")+ #expect(AppleTime.raw(from: AppleTime.parseISO("2026-01-01T00:00:00Z")!) == 788_918_400_000_000_000)+ #expect(AppleTime.parseISO("2026-01-01T00:00:00.250Z") != nil)+ #expect(AppleTime.parseISO("yesterday") == nil)+ }++ @Test func extremeTimesClampInsteadOfCrashing() {+ #expect(AppleTime.raw(from: Date(timeIntervalSince1970: -62_135_596_800)) == Int64.min)+ #expect(AppleTime.raw(from: Date(timeIntervalSince1970: 253_402_214_400)) == Int64.max)+ #expect(AppleTime.nanoseconds(fromRaw: Int64.min) == Int64.min)+ #expect(AppleTime.nanoseconds(fromRaw: 50_000_000_000) == Int64.max)+ #expect(AppleTime.nanoseconds(fromRaw: -50_000_000_000) == Int64.min)+ }+}diff --git a/apple/helper/scripts/signing-probe.sh b/apple/helper/scripts/signing-probe.shnew file mode 100755index 0000000000..ad7874cc0a--- /dev/null+++ b/apple/helper/scripts/signing-probe.sh@@ -0,0 +1,162 @@+#!/usr/bin/env bash+# Headless signing probe for GBrain Helper.app, run by .github/workflows/apple-helper.yml.+#+# The question: on a Mac with no Apple developer account and nobody at the+# screen, can a key and self-signed certificate made on that Mac sign the app+# (hardened runtime, designated requirement pinned to the bundle id and the+# certificate) with no dialog, or does the certificate first need to be trusted+# for code signing ("Always Trust"), which asks for a password once?+#+# Attempts, in order, stopping at the first that signs and verifies:+# A. untrusted certificate, keychain named only with --keychain+# B. untrusted certificate, keychain also on the user's search list+# C. certificate trusted for code signing (admin trust settings via sudo)+# Every codesign call runs under a 120-second alarm, so a dialog waiting for a+# person fails the attempt instead of hanging the job. The answer goes to the+# job summary. Exits non-zero only when no attempt worked.+set -euo pipefail++here="$(cd "$(dirname "$0")/.." && pwd)"+work="$(mktemp -d)"+keychain="$work/gbrain-helper-signing.keychain-db"+app="$work/GBrain Helper.app"+bundle_id="io.gbrain.apple-helper"+program="gbrain-apple-helper"+report="$work/report.md"+saved_keychains=()+trusted=0+outcome=""++cleanup() {+ if [ "${#saved_keychains[@]}" -gt 0 ]; then+ security list-keychains -d user -s "${saved_keychains[@]}" || true+ fi+ if [ "$trusted" = 1 ]; then+ sudo security remove-trusted-cert -d "$work/cert.pem" >/dev/null 2>&1 || true+ fi+ security delete-keychain "$keychain" >/dev/null 2>&1 || true+ rm -rf "$work"+}+trap cleanup EXIT++limited() {+ perl -e 'alarm shift; exec @ARGV or die "exec: $!\n"' 120 "$@"+}++note() {+ printf '%s\n' "$*" | tee -a "$report"+}++macos="$(sw_vers -productVersion) ($(sw_vers -buildVersion)), $(uname -m)"+note "## GBrain Helper signing probe"+note ""+note "- macOS $macos"+note "- $(swift --version 2>&1 | head -n 1)"++swift build -c release --package-path "$here"+bin_dir="$(swift build -c release --package-path "$here" --show-bin-path)"+mkdir -p "$app/Contents/MacOS" "$app/Contents/Library/LaunchAgents"+cp "$bin_dir/$program" "$app/Contents/MacOS/$program"+cp "$here/Bundle/Info.plist" "$app/Contents/Info.plist"+cp "$here/Bundle/$bundle_id.plist" "$app/Contents/Library/LaunchAgents/$bundle_id.plist"++unlock="$(/usr/bin/openssl rand -hex 24)"+security create-keychain -p "$unlock" "$keychain"+security set-keychain-settings -lut 3600 "$keychain"+security unlock-keychain -p "$unlock" "$keychain"++cat >"$work/cert.cnf" <<'EOF'+[req]+distinguished_name = dn+x509_extensions = ext+prompt = no+[dn]+CN = GBrain Helper Local Signing+[ext]+basicConstraints = critical,CA:false+keyUsage = critical,digitalSignature+extendedKeyUsage = critical,codeSigning+subjectKeyIdentifier = hash+EOF+/usr/bin/openssl req -new -x509 -newkey rsa:2048 -nodes -days 3650 -config "$work/cert.cnf" \+ -keyout "$work/key.pem" -out "$work/cert.pem" 2>/dev/null+legacy=""+case "$(/usr/bin/openssl version)" in+ OpenSSL\ 3*) legacy="-legacy" ;;+esac+p12_unlock="$(/usr/bin/openssl rand -hex 16)"+# shellcheck disable=SC2086+/usr/bin/openssl pkcs12 -export $legacy -inkey "$work/key.pem" -in "$work/cert.pem" \+ -name "GBrain Helper Local Signing" -out "$work/identity.p12" -passout "pass:$p12_unlock"+security import "$work/identity.p12" -k "$keychain" -P "$p12_unlock" -T /usr/bin/codesign >/dev/null+security set-key-partition-list -S apple-tool:,apple: -s -k "$unlock" "$keychain" >/dev/null+rm -f "$work/key.pem" "$work/identity.p12"++leaf="$(/usr/bin/openssl x509 -in "$work/cert.pem" -outform der | shasum -a 1 | awk '{print toupper($1)}')"+requirement="identifier \"$bundle_id\" and certificate leaf = H\"$leaf\""+listed="$(security find-identity -p codesigning "$keychain" | grep -c "$leaf" || true)"+valid="$(security find-identity -v -p codesigning "$keychain" | grep -c "$leaf" || true)"+note "- Certificate: self-signed, code-signing key use, 10 years, own keychain; listed as a signing identity: $listed, listed as valid: $valid"++sign() {+ limited codesign --force --keychain "$keychain" --sign "$leaf" --options runtime --timestamp=none \+ --entitlements "$here/Bundle/$program.entitlements" "-r=designated => $requirement" "$app" 2>"$work/codesign.err"+}++verify() {+ codesign --verify --strict --verbose=2 "$app" 2>>"$work/codesign.err" &&+ codesign --verify --strict "-R=$requirement" "$app" 2>>"$work/codesign.err" &&+ codesign -d -r- "$app" 2>&1 | grep -qiF "certificate leaf = H\"$leaf\""+}++attempt() {+ if sign && verify; then+ outcome="$1"+ note "- $1: signed and verified"+ return 0+ fi+ note "- $1: failed: $(tr '\n' ' ' <"$work/codesign.err" | cut -c1-300)"+ return 1+}++if ! attempt "A (untrusted certificate, keychain named with --keychain)"; then+ while IFS= read -r entry; do+ entry="${entry#"${entry%%[![:space:]]*}"}"+ entry="${entry#\"}"+ entry="${entry%\"}"+ [ -n "$entry" ] && saved_keychains+=("$entry")+ done <<KEYCHAINS+$(security list-keychains -d user)+KEYCHAINS+ security list-keychains -d user -s "$keychain" ${saved_keychains[@]+"${saved_keychains[@]}"}+ if ! attempt "B (untrusted certificate, keychain on the search list)"; then+ if limited sudo security add-trusted-cert -d -r trustRoot -p codeSign -k /Library/Keychains/System.keychain "$work/cert.pem"; then+ trusted=1+ attempt "C (certificate trusted for code signing)" || true+ else+ note "- C: could not mark the certificate trusted without a dialog"+ fi+ fi+fi++if [ -n "$outcome" ]; then+ note "- Signed program runs: $("$app/Contents/MacOS/$program" --version 2>&1 || echo failed)"+ before="$(codesign -d -r- "$app" 2>&1 | grep designated || true)"+ cp "$bin_dir/$program" "$app/Contents/MacOS/$program"+ if sign && verify && [ "$before" = "$(codesign -d -r- "$app" 2>&1 | grep designated || true)" ]; then+ note "- Re-signing a fresh copy keeps the same designated requirement, so grants that match it carry over"+ else+ note "- Re-signing a fresh copy changed or broke the designated requirement"+ fi+fi++note ""+case "$outcome" in+ A*|B*) note "**Answer:** an untrusted self-signed certificate signs the helper with no dialog on macOS $macos. No Always Trust step is needed." ;;+ C*) note "**Answer:** on macOS $macos signing needed the certificate trusted for code signing first, so setup needs the one-time Always Trust step." ;;+ *) note "**Answer:** no attempt signed the helper headlessly on macOS $macos, so setup falls back to ad hoc signing." ;;+esac+if [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then+ cat "$report" >>"$GITHUB_STEP_SUMMARY"+fi+[ -n "$outcome" ]diff --git a/changelog.d/apple-helper-port.md b/changelog.d/apple-helper-port.mdnew file mode 100644index 0000000000..5e53f001c7--- /dev/null+++ b/changelog.d/apple-helper-port.md@@ -0,0 +1,5 @@+**A small macOS helper that gives gbrain read-only access to your Messages history now lives in `apple/helper/`. Nothing in gbrain calls it yet.**++### Added+- `apple/helper/` is a Swift program that runs on your Mac under your own account. It answers read-only questions about your Messages history on a local port, and only for tokens you create with its `token` command. A removed token stops working on the next request. It is ported from the maintainer's private prototype, with the maintainer's consent. You build it yourself with Swift 6 (Xcode 16); a later release adds a gbrain command that builds and installs it for you.+- A macOS workflow, `Apple helper`, builds and tests the helper on every change under `apple/`. It is not a required check, and Linux and Windows installs never build or load the helper.diff --git a/docs/ci-event-parity.md b/docs/ci-event-parity.mdindex cadb19181b..c219a7a6d9 100644--- a/docs/ci-event-parity.md+++ b/docs/ci-event-parity.md@@ -43,6 +43,7 @@ checks ([below](#full-ci-on-pull-requests)), never per-cell matrix names.| `heavy-tests.yml` | `grok-door` (label or input only) | Recorded decision: no scheduled run until the `XAI_API_KEY` secret exists | None yet, by decision || `heavy-tests.yml` | `opencode-door-canary` | Named exception: schedule-only, non-gating canary | Itself || `macos-validation.yml` | Label-gated on pull requests | Named exception | Nightly macOS validation schedule |+| `apple-helper.yml` | Pull requests and master pushes that touch `apple/**` or the workflow; not a required check | Named exception: the Swift helper is inert files to every other workflow | Every push to master that touches those paths || `scale-tier.yml` | Path- or label-gated 10k on pull requests | Named exception | Nightly Scale tier schedule (10k-50k) || `upgrade-matrix.yml` | Pull requests run N-1 and N-50, keyless repo-less cells only, against the head; the full matrix (N-1, N-10, N-50, 30 days, stable × both engines × keys × brain shapes) is schedule and dispatch only, and those refuse any ref but master ([docs](ci-upgrade-matrix.md)) | Named exception | Nightly Upgrade matrix schedule || `semgrep.yml` | PR diff scan vs full-tree scan | Named exception | Weekly Semgrep schedule |