Your Android phone and your Windows PC, working as one. Local-first, end-to-end encrypted, no account.
Architecture
NectarLink is built around a shared Rust core library (nectarlink-core) that implements peer discovery, cryptographic identity, QUIC session management, capability negotiation, persistent storage, and media/file framing. Native platform shells on Windows (desktop/app) and Android (android/app) bind directly to this core and provide OS integration and UI rendering.
System Topology
+-------------------------------------------------------------------------------+
| Windows PC |
| |
| +-------------------------------------------------------------------------+ |
| | desktop/app (nectarlink-desktop) | |
| | +---------------------------+ +-----------------------------------+ | |
| | | Qt 6.12 QML UI | | Windows OS Integrations | | |
| | | (desktop/app/qml/*.qml) | | WASAPI, Media Foundation, WinRT, | | |
| | +-------------+-------------+ | DXGI Desktop Duplication, SendInput| | |
| | | cxx-qt 0.10 +-----------------+-----------------+ | |
| | +-------------v-----------------------------------v-----------------+ | |
| | | Rust Bridge & Services (desktop/app/src/bridge/, services) | | |
| | +-------------+-----------------------------------+-----------------+ | |
| | | | Shared Memory | |
| | | | NLVC v2 | |
| | | +-------------v-----------------+ | |
| | | | desktop/vcam | | |
| | | | (nectarlink_vcam.dll IMFMedia)| | |
| | | +-------------------------------+ | |
| | +-------------v-----------------------------------------------------+ | |
| | | core/nectarlink-core (incl. adb.rs & usb.rs) + nectarlink-protocol| | |
| | +-------------+-------------------+-------------------+-------------+ | |
| +----------------|-------------------|-------------------|----------------+ |
+-------------------|-------------------|-------------------|-------------------+
| LAN QUIC | USB ADB TCP | Relay QUIC
| (UDP + mDNS) | (127.0.0.1:53319) | (away_mode)
+-------------------v-------------------v-------------------v-------------------+
| Android Device |
| |
| +-------------------------------------------------------------------------+ |
| | android/app (app.nectarlink.android) | |
| | +-------------------------------------------------------------------+ | |
| | | core/nectarlink-ffi (UniFFI 0.32 → libnectarlink_ffi.so) | | |
| | | embeds core/nectarlink-core + core/nectarlink-protocol | | |
| | +-------------+-----------------------------------+-----------------+ | |
| | | Kotlin Callbacks / Events | Loopback TCP | |
| | +-------------v-----------------+ +-------------v-----------------+ | |
| | | Jetpack Compose UI & | | Elevated Helper (app_process) | | |
| | | ConnectionService (Foreground)| | InputServer / AppDisplay | | |
| | | NotificationListener, Camera2 | | (UID 2000 shell via ADB TLS) | | |
| | +-------------------------------+ +-------------------------------+ | |
| +-------------------------------------------------------------------------+ |
+-------------------------------------------------------------------------------+
Workspace Crates and Components
1. core/nectarlink-protocol
Pure wire-protocol crate with no runtime IO or platform dependencies beyond tokio::io::{AsyncRead, AsyncWrite} for stream framing.
- Envelope and Framing (
src/frame.rs,src/messages.rs): Implements unsigned LEB128 length-prefixed CBOREnvelopeencoding (ciborium), with a 1 MiB frame ceiling (MAX_FRAME_LEN = 1,048,576bytes). - Capabilities (
src/capabilities.rs): Defines the canonical string identifiers for all v0 protocol capabilities (msg.v0,clip.v0,files.v0,mirror.v0,webcam.v0,storage.v0,sms.v0,calls.v0,contacts.v0,media.v0,photos.v0,recorder.v0,toggles.v0,actions.v0,deck.v0,remote.v0,remote.input.v0,mirror.pc.v0,mirror.apps.v0,mirror.audio.v0,commands.v0, andlocalsend.v2). - Pairing Primitives (
src/pairing.rs): Defines thenectarlink://pairURI parser/serializer,ALPN_PAIR("nectarlink-pair/0"),ALPN_SESSION("nectarlink/0"), QR HMAC proof calculation, and 6-digit Short Authentication String (SAS) derivation. - Video Packetizer (
src/video.rs): Encodes and decodes real-time video packets (VideoKind::Config,VideoKind::Frame,VideoKind::Keyframe) with microsecond timestamps and an 8 MiB packet limit (MAX_VIDEO_PACKET).
2. core/nectarlink-core
The central networking, USB/ADB transport, persistence, and feature runtime.
- Node and Session Engine (
src/node.rs,src/session.rs): Manages theiroh::Endpoint, accepts incomingALPN_PAIRandALPN_SESSIONconnections, runs thehello/hello.updatecapability handshake, maintains per-peer connection state, and multiplexes control messages, bulk streams, and QUIC datagrams. - Zero-Shell-Out ADB & USB Tunnel (
src/adb.rs,src/usb.rs): Speaks the ADB host protocol directly over TCP (127.0.0.1:5037) to track attached USB Android devices viahost:track-deviceswithout spawningadb.exesubprocesses per poll, configures port forwards (host-serial:<serial>:forward:tcp:0;tcp:53319), and opens a TCP-carried UDP tunnel (NLQUheader) soirohQUIC datagrams traverse the USB cable with full TLS 1.3 encryption and congestion control. - Persistence (
src/store.rs): Embedded SQLite database (nectarlink.db, schema version 7) storing paired devices, per-device capability toggles, key-value settings, unified timeline entries,MessagingStylechat threads/messages, and the persistent FIFO send queue (send_queue). - Resumable Transfers (
src/transfer.rs): Implementsfiles.v0multi-file transfers with BLAKE3 integrity verification (blake3),.partbyte-offset resume, pause/resume/cancel semantics, and automatic queueing when a peer is offline or mid-disconnect. - Perceptual Color Engine (
src/palette.rs): Extracts Material You HCT + OKLab 15-role light and dark color palettes (ThemeScheme) from album cover art. - Encrypted Clipboard Vault (
src/clipboard_history.rs): Stores recent text and image clips in encrypted files (clipboard_history.encwithNLCB\x01header andclipboard_images/<id>.encwithNLCI\x01header) sealed directly with the platformKeyProtector(Windows DPAPI or Android Keystore AES-256-GCM). - Interoperability (
src/localsend.rs,src/handoff.rs): Implements LocalSend v2 HTTP/HTTPS discovery and file transfer compatibility (UDP/TCP 53317) and cross-device reading/document continuation (handoff).
3. core/nectarlink-ffi
UniFFI 0.32 bridge (src/lib.rs) compiled via cargo-ndk into libnectarlink_ffi.so for arm64-v8a and x86_64.
- Wraps
nectarlink_core::Nodein an FFI-safeNectarlinkNodehandle. - Marshals async Rust events into Kotlin
EventCallbackinterfaces (NodeEvent) and exposes synchronous/asynchronous methods for pairing, messaging, file transfers, send queue inspection, media sessions, and stream creation. - Accepts a Kotlin-implemented
ForeignKeyProtectorso Android'sKeystoreKeyProtector(AES-256-GCM insideAndroidKeyStore, hardware-backed when supported by the device) sealsidentity.keyandclipboard_history.encinside Rust.
4. cli (nectarlink-cli)
Command-line binary (nectarlink) in top-level cli/ for headless operation, scripted integration testing, notify-when task completion alerts, and diagnostic commands (id, pair, qr-host, send, status).
5. desktop/app (nectarlink-desktop)
The Windows desktop client combining a Rust service host with a Qt 6.12 Quick/QML user interface bridged via cxx-qt 0.10.
- Qt Bridges (
desktop/app/src/bridge/):node_bridge.rs: Main UI controller exposing connection state, device list, timeline, notifications, SMS, calls, photos, storage browser, clipboard, and send queue models to QML.theme_tokens.rs: Design system typography, spacing, elevation, and dynamic OKLab media palette bindings.mirror_item.rs(MirrorDisplayQQuickItem): Hardware-accelerated D3D11/Media Foundation H.264/H.265 video decoder and QQuick rendering node for phone screen mirroring and remote app windows.
- Windows Platform Modules (
desktop/app/src/):audio_out.rsandpc_audio.rs: Low-latency WASAPI audio playback (for phone audio mirroring) and WASAPI loopback capture (for streaming PC system audio to the phone).pc_mirror.rsandremote_input.rs: DXGI Desktop Duplication screen capture with Media Foundation H.264 encoding, plus Win32SendInputsynthesis for remote trackpad/keyboard control.media.rs: WindowsGlobalSystemMediaTransportControlsSessionManager(SMTC) integration for bidirectional Now Playing metadata and transport controls.ble_proximity.rs: Windows WinRT BLE watcher and GATT advertisement reader for RSSI-smoothed proximity presence and auto-lock.local_api.rs: Opt-in localhost HTTP/SSE server (127.0.0.1:53320) for CLI, PowerToys, Raycast, and Stream Deck integrations.share_target.rsandshell_integration.rs: Windows Share Target activation and Explorer context-menu verb registration (Send to Phone with NectarLink).setup.rs(withbridge/setup.rsandqml/setup/): The installer's and uninstaller's window. The NSIS installer (desktop/installer/nectarlink.nsi) unpacks the app and starts it with--setup; the uninstall entry starts the installed app with--remove. The window runs the NSIS installer or uninstaller silently with administrator rights and shows its progress.updater.rs: Automatic release update checker (https://api.github.com/repos/thtbee/nectarlink/releases/latest) and SHA-256 verified silent NSIS installer runner on installed desktop copies.
6. desktop/vcam (nectarlink-vcam)
A standalone Windows Media Foundation virtual camera COM DLL (nectarlink_vcam.dll) registered via IMFVirtualCamera or DirectShow/Media Foundation class factories.
- Communicates with
nectarlink-desktopacross a lock-free sequence-numbered shared memory region (Global\NectarlinkVcamFrame_v2,Local\NectarlinkVcamFrame_v2, orLocal\NectarlinkVirtualCamera_v2defined indesktop/vcam/src/shm.rs). - Supports
RGB32(PIXEL_FORMAT_RGB32 = 0) andNV12(PIXEL_FORMAT_NV12 = 1) frames up to3840x2160(MAX_WIDTH = 3840,MAX_HEIGHT = 2160,MAX_FRAME_BYTES = 33,177,600) with a 64-byteSharedHeader(signature0x4E4C5643/"NLVC",VERSION = 2). - Writers update payload bytes between odd and even sequence numbers (
seq), allowing camera consumers (Zoom, Teams, Meet, OBS) to read frames without cross-process mutexes.
7. android/app (app.nectarlink.android)
The Android application written in Kotlin with Jetpack Compose (compileSdk = 37, targetSdk = 37, minSdk = 26).
- Foreground Service (
service/ConnectionService.kt): Hosts theNectarlinkNodesingleton, maintains wake/Wi-Fi locks as needed, and coordinates feature handlers (SmsHandler,CallHelper,ContactsHelper,MediaSessionHelper,PhotosHelper,StorageHelper,AudioRecorderHelper,TogglesHelper,WebcamStreamer,ScreenMirrorStreamer,PcMirrorClient,AudioMirrorPlayer). - Notification Listener (
service/NotificationBridge.kt): ImplementsNotificationListenerServiceto stream active notifications,MessagingStyleconversation threads, inline replies, and action invocations. - Elevated Helper (
elevated/Elevated.kt,elevated/InputServer.kt,elevated/AppDisplay.kt,elevated/AdbMdns.kt): An optional privileged helper process launched viaapp_process(app.nectarlink.android.elevated.InputServer <token>) undershell(UID 2000via on-device Wireless Debugging TLS pairing or USB ADB). It binds a loopback TCP socket (127.0.0.1:<ephemeral_port>) and requires a 24-byte (48-hex-char) per-sessionSecureRandomtoken on the first line (verified in constant time withMessageDigest.isEqual), injects touch/key/scroll events throughandroid.hardware.input.IInputManagerand/dev/uhid, and creates virtual displays (DisplayManager.createVirtualDisplayinAppDisplay.kt) for per-app window streaming (mirror.apps.v0).
Discovery and Transport Architecture
NectarLink uses iroh (1.3) to run a single QUIC + TLS 1.3 endpoint per device. Every paired connection is classified into one of three ConnectionPath variants (core/nectarlink-core/src/node.rs):
| Path | Discovery Mechanism | Transport Path | Latency & Bandwidth Profile |
|---|---|---|---|
ConnectionPath::Lan |
mDNS (iroh-mdns-address-lookup, service name "nectarlink") + cached direct socket addresses from peers.last_addrs |
Direct UDP QUIC between LAN IPv4/IPv6 sockets | Lowest latency, full Wi-Fi / Ethernet throughput |
ConnectionPath::Usb |
core/nectarlink-core/src/adb.rs device tracker (127.0.0.1:5037) forwarding to 127.0.0.1:53319 (DEFAULT_USB_PORT) |
QUIC UDP datagrams framed over TCP (NLQU header in src/usb.rs) via adb forward |
Zero Wi-Fi dependency, deterministic wired latency |
ConnectionPath::Relay |
Opt-in Connect over mobile data (away_mode = true) enabling n0_dns Pkarr lookup and RelayMode::Default |
End-to-end TLS 1.3 encrypted QUIC relayed via iroh DERP/relay servers |
Works across cellular and different networks; subject to relay capability guards |
QUIC Transport Parameters
Configured in core/nectarlink-core/src/node.rs:
KEEP_ALIVE_INTERVAL:1,000 ms(1 second)IDLE_TIMEOUT:8s(detects network drops and triggers path failover within seconds)STREAM_WINDOW:16 MiBper streamCONNECTION_WINDOW:32 MiBper connection
Automatic Path Failover and Deduping
When both USB and Wi-Fi are available, or when a phone transitions between Wi-Fi and a Connect over mobile data (away_mode) relay path, both peers may attempt simultaneous connections. Node resolves concurrent dials deterministically and updates the active session handle in place so upper-layer features (Timeline, SendQueue, Media, Notifications) continue operating against the peer's DeviceId.
Perceptual OKLab Color Engine
Implemented in core/nectarlink-core/src/palette.rs, the color engine derives cohesive light and dark UI palettes (ThemeScheme) from album artwork (extract_theme_from_encoded_image, extract_theme_from_rgba).
Encoded Image / RGBA Buffer
│
├─► 1. Border Trim (trim_borders: strips up to 22% uniform/black/white letterbox bars)
│
├─► 2. Area-Box Downsample (downsample_rgba: max 112x112 = 12,544 pixels)
│
├─► 3. Alpha & Near-Black Filter (ignores alpha < 128 and CIE L* < 1.5)
│
├─► 4. Celebi Quantization in CIE L*a*b* (QuantizerCelebi, up to 128 clusters)
│
├─► 5. Achromatic Check & 360-deg Hue Neighborhood Scoring
│ (weights population^0.38 * chroma * mid-tone lightness; penalizes skin-tone bands)
│
├─► 6. OKLab Centroid Seed Calculation (sRGB → linear RGB → LMS → OKLab mean)
│
└─► 7. HCT TonalPalette Generation + WCAG AA/AAA Contrast Enforcement
Produces 15-role dark & light RolePalette (cached in 64-entry LRU)
Pipeline Stages
- Letterbox and Border Trimming (
trim_borders): Scans inward from all four edges (up to 22% of width/height) and removes rows/columns that are near-black, near-white, or low-variance solid frames so letterboxed video thumbnails do not skew the dominant color. - Area-Box Downsampling (
downsample_rgba/downsample_rgb): Averages pixel blocks into at most112x112(MAX_SAMPLE_SIDE = 112) pixels, suppressing compression noise while preserving proportional color area. - Celebi Clustering (
QuantizerCelebi): Quantizes filtered pixels in CIE L*a*b* space into at most 128 clusters (QUANTIZE_COLORS = 128). - Monochrome Detection (
is_mostly_achromatic): If at least 92% of non-dark pixels have HCT chroma< 8.0and no vibrant accent exists, the engine selects the neutral zinc seed (ACHROMATIC_SEED_RGB = 0x5E5E62) to avoid inventing false color casts on black-and-white covers. - Hue Family Scoring (
select_perceptual_seed): Evaluates each candidate cluster across a 15-degree Gaussian hue neighborhood, weightingpopulation_ratio^0.38, HCT chroma peak (48.0), mid-tone lightness preference, and a skin-tone de-bias factor (35..52deg hue with moderate chroma) so background/graphic art beats portrait skin tones. - OKLab Centroid (
oklab_centroid_rgb): Computes the population-weighted mean of all clusters within 25 degrees of the winning hue in perceptual OKLab space (srgb_to_oklab→oklab_to_srgb), preventing muddy RGB averaging. - Tonal Palette and Contrast Enforcement (
scheme_from_seed,enforce_contrast): Builds six HCTTonalPaletteramps (primary,secondary,tertiary,neutral,neutral_variant,error) and derives 15 semantic tokens (surface,surface_container,surface_bright,primary,on_primary,primary_container,on_primary_container,secondary,on_secondary,secondary_container,tertiary,on_tertiary,on_surface,on_surface_variant,outline) with guaranteed WCAG contrast ratios (>= 4.5:1for primary/surface text,>= 3.0:1for muted text) and a 64-entry FNV-1a hashed LRU cache (CACHE_CAPACITY = 64).