Native Apple Dev Cockpit

Your terminal, browser, and devices —
one cockpit, four screens.

blau is a four-app native Apple ecosystem built around Pilot, a macOS command-center with real Ghostty terminals, a WebKit browser, and git tooling. Your iPhone drives the cursor, your Watch presses Enter, and your iPad mirrors the screen over the air — no servers, no accounts, no cloud in the path.

Get it on GitHub See the architecture
4 apps
macOS / iOS / watchOS / iPadOS
2 transports
control + media, both peer-to-peer
0 servers
discovery and data stay on your network
pilot.app — terminal · browser · device mirror · git inspector
Pilot for macOS: a command-center showing live Ghostty terminals, a WebKit browser pane, and a git inspector side by side.
The ecosystem

Four apps. One Mac does the work.

Pilot runs everything on the desktop. The other three are honest peripherals — each does one job, natively, on its own device.

Pilot

macOS · the cockpit

The command-center. Split panes of real Ghostty terminals, a WebKit browser, a USB-tethered iPhone mirror, and a git / GitHub inspector — with notes and freehand annotation on top.

Copilot

iOS

Mirrors Pilot's workspace list, drives the Mac cursor from a touch trackpad, switches panes, and transcribes your voice on-device to paste into the active terminal.

Wingman

watchOS

A double-pinch on your wrist becomes an Enter keypress on the Mac. One gesture, one job, with haptic confirmation.

Plotter

iPadOS

Receives Pilot's screen as a live HEVC stream and lets you mark it up with Apple Pencil — strokes round-trip back onto the Mac. Includes a widget and Live Activity.

+ blau.app — this site, built with Astro

Inside Pilot

Six real panes, rendered by the platform.

No Electron, no web-view IDE. The terminal is libghostty rendered through Metal; the browser is WKWebView; everything else is SwiftUI and AppKit.

Pilot for macOS showing terminal, browser, device mirror and git panes side by side
pilot.app — six native panes in one resizable window

  1. Ghostty terminal panes

    Real libghostty — the same GPU-accelerated, Metal-rendered engine as the standalone Ghostty app — in resizable, collapsible splits. Per-pane working directory tracked over OSC 7; sessions persist across relaunch.

  2. WebKit browser pane

    A full WKWebView with address bar, back / forward / reload, Web Inspector, and downloads. It scans your repo for package.json and framework configs to offer one-click dev-server links — Next 3000, Vite 5173, Astro 4321.

  3. USB iPhone mirror

    Mirrors a tethered iPhone via AVFoundation and CoreMediaIO, QuickTime-style. Record to a .mov on your Desktop or grab a screenshot to clipboard, without leaving the window.

  4. Git & GitHub inspector

    Recent commits from git log, Actions runs and open issues from the gh CLI. Click an issue and Pilot pastes an agent-ready ‘implement #N’ prompt into the active terminal — and badges any workspace when its background Action finishes.

  5. Markdown notes

    A live GitHub-Flavored-Markdown editor with split-into-lines multi-cursor, clickable task checkboxes, code-block copy, and .env secrets masked until you click to reveal.

  6. Freehand ink overlay

    Draw straight over the active pane with a native AppKit canvas — color palette, eraser, undo, clear. SwiftData keeps your workspaces, panes, browser state, and notes across launches.

LIVE MIRROR · PILOT → PLOTTER

Your Mac screen, on the iPad, in real time.

Pilot captures its own window with ScreenCaptureKit, encodes HEVC on Apple's dedicated media engine with VideoToolbox, and streams it to Plotter over a direct device-to-device Wi-Fi link. Plotter hardware-decodes it, and your Apple Pencil strokes are normalized to the video frame and sent back — Pilot renders them over its own window. Undo and clear sync both ways.

  1. 01 Capture

    ScreenCaptureKit grabs Pilot's window as IOSurface-backed frames, up to 4K / 60fps, skipping idle frames.

  2. 02 Encode

    VideoToolbox compresses to HEVC on the media engine — same picture in roughly half the bits of H.264.

  3. 03 Stream

    FrameLink sends length-prefixed packets over Network.framework on the local link; capture only runs while a Plotter is connected.

  4. 04 Annotate

    PencilKit strokes on the iPad round-trip to the Mac; Plotter reports link quality so Pilot adapts its bitrate.

Hardware all the way down — capture, encode, and decode never touch your CPU.

Pilot's macOS window mirrored live onto an iPad with Apple Pencil annotation
HEVC · 60fps · link-up
Copilot workspaces list on iPhone with an on-screen trackpad
on-device · 0 bytes audio sent
Voice & Control

Talk to your terminal. Nothing leaves the device.

Hold Copilot's volume-down button to record; WhisperKit transcribes your speech locally on the iPhone with Apple's Core ML — no audio ever crosses the wire. Only the finished text is sent, and Pilot pastes it into the active terminal. Hold volume-up to send Enter.

PUSH-TO-TALK

On-device transcription

WhisperKit (openai_whisper-base) runs Whisper on Apple Silicon via Core ML. Audio is transcribed on the phone; just the text is delivered.

TRACKPAD

Cursor from your pocket

Copilot's touch trackpad sends relative mouse deltas and clicks; Pilot posts real CGEvents through the Accessibility API.

VOLUME NAV

Hardware-button workspaces

A KVO state machine on outputVolume turns the volume buttons into list navigation — the system volume HUD is suppressed by a hidden MPVolumeView.

WATCH

Double-pinch = Enter

Wingman sends a double-pinch over WatchConnectivity to Copilot, which relays it to Pilot as an Enter keypress, with haptic confirmation.

import WhisperKit — on-device, MIT-licensed, auditable. No cloud STT.
ARCHITECTURE

Two transports. Zero servers.

Devices find each other over Bonjour and talk directly. Control sync — workspace state, selection, trackpad, voice text, device status — runs over MultipeerConnectivity with an encrypted MCSession. The high-bandwidth screen mirror and annotation channels run over Apple's Network.framework as length-prefixed packets on the local link. Discovery and data never leave your network. There is no backend to breach, no telemetry hop, no account to create.

topology · live
  • control · MultipeerConnectivity
  • media · Network.framework
MAC Pilot center hub
  • IPHONE Copilot Pilot ⇄ Copilot — MultipeerConnectivity · MCSession encryptionPreference: .required
  • WATCH Wingman Copilot ⇄ Wingman — WatchConnectivity · double-pinch
  • IPAD Plotter Pilot → Plotter — Network.framework · HEVC over Bonjour/AWDL TCP
  • IPAD Plotter Plotter → Pilot — Network.framework · Pencil strokes + link quality

Control and media are deliberately separate transports — reliable sync over MultipeerConnectivity, low-latency video over Network.framework.

Built on

First-party frameworks, end to end.

No cross-platform runtime sits between blau and the OS. Every screen is native, and the hard parts lean on the tools Apple ships its own platforms on.

SwiftUI · Swift 6.0 · macOS / iOS / watchOS / iPadOS

Open source

Read the source. Run the cockpit.

Four native apps, two peer-to-peer transports, and not a single server. Clone it, generate the project with XcodeGen, and build.

View on GitHub Read the architecture