hongkongkiwi · Oct 25, 2026 · Oct 25, 2026 · Oct 25, 2026 · Oct 25, 2026 · Oct 25, 2026
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -49,7 +49,7 @@ jobs:
       - name: Install Linux native dependencies
         run: |
           sudo apt-get update
-          sudo apt-get install -y libwebkit2gtk-4.1-dev build-essential curl wget file libxdo-dev libssl-dev libdbus-1-dev libayatana-appindicator3-dev librsvg2-dev libfuse2 patchelf xvfb dbus-x11 xdg-utils
+          sudo apt-get install -y libwebkit2gtk-4.1-dev build-essential curl wget file libxdo-dev libssl-dev libdbus-1-dev libayatana-appindicator3-dev librsvg2-dev libasound2-dev libfuse2 patchelf xvfb dbus-x11 xdg-utils
       # Tauri embeds the built frontend, so dist/ must exist before cargo runs.
       - run: npm ci
       - run: npm run build

diff --git a/AGENTS.md b/AGENTS.md
@@ -31,6 +31,7 @@
 - The GitHub view (`state.view === 'github'`, header button beside Board, hidden while `githubEnabled()` is false) is `src/github-hub.ts` with `src/github-hub.css`, mounted lazily by `syncGithubHub()` in `main.ts` into `#github-hub`; full-width like the board (layout rules in `styles.css`). Native side `src-tauri/src/github_hub.rs` (`github_repos`, `github_repo_pulls`, `github_repo_issues` by `owner/name`; `github_project_repos` maps projects by `origin` remote; `github_clone_repo`). Agent prompts and thread titles are in `src/github-hub-prompts.ts` and mention `#123`, so `github.contextFor` attaches the item; `startHubTask()` in `main.ts` creates, links, titles and (optionally) sends the task. Preview data: `src/preview-github-hub.ts`.
 - Branch/worktree UI lives in `src/worktrees.ts` (context bar, `#branch-chip` menu, dialogs, cleanup offer); GitHub in `src/github.ts`; a task's linked PR renders as the inspector's PR tab from `src/github-pull.ts` (native side `github_pull.rs`), and after every run end `github_detect_pull_request` links a PR URL the turn mentioned when it is on the checkout's branch or was created during the turn (an open link is never replaced). Both receive callbacks from `main.ts` instead of importing its state. `linkChip()` and every GitHub affordance return nothing while `githubEnabled()` is false.
 - Token saver (`src/token-saver.ts`, panel in `src/token-saver-panel.ts`): per-thread packs (Caveman Lite/Full/Ultra, Lean code, Quiet tools, RTK when `token_saver_tools` finds `rtk`) become one folded `Response style (PandaCode token saver…` block appended at send, only when the packs change and every `REMIND_EVERY` messages, so every harness gets it without paying for it each turn. Keep pack text short; cite benchmarks in `saves`, mark estimates `(est.)`.
+- Voice dictation ([docs/voice.md](docs/voice.md)): the composer mic (`src/voice.ts`, button beside the token-saver leaf, `data-action="voice"`) records natively — Apple Speech on macOS (`src-tauri/src/voice/apple.rs`, needs the bundled app; `src-tauri/Info.plist` carries the usage strings) or sherpa-onnx two-stage local models elsewhere (`voice/sherpa.rs`): streaming zipformer partials + `punct-en` restoration live, the chosen quality model (`voice/model.rs`, pinned catalog, SHA-256, size caps, `voice-download` progress, nothing bundled) re-decodes on stop unless Fast mode. Live partials replace the previous partial in the prompt via the `startHubTask` insertion pattern and never auto-send; the recording bar mounts on `body` so `renderComposer` re-renders never kill a take; Esc cancels; Settings → Voice (`src/voice-settings.ts`, tab between Agents and Providers) holds enable, engine, model catalog and tap/hold. Commands `voice_*` in `voice/commands.rs`; events `voice-partial`/`voice-result`/`voice-error`/`voice-level`/`voice-download` (typed in `src/bridge.ts`). Mobile: `mobile/src/ui/speech.ts` over `expo-speech-recognition` (pure helpers in `speech-text.ts`), mic `Round` in `Composer.tsx`, on-device preferred with an On-device only switch in the settings tab; only the final transcript goes over Connect.
 - Local VMs (Apple silicon): `src-tauri/src/vm*` drives Tart (`vm/tart.rs`; one-click install of the notarized release in `vm/setup.rs`, team `9M2P8L4D89`) on Virtualization.framework. VMs run headless with `--vnc-experimental`; `vm/rfb.rs` is a small RFB client (that server answers only when the screen changed, so captures are incremental with a quiet window) and `vm/screen.rs` maps clicks, drags, scrolls, typing and key combos, scaling screenshots to 1280 px wide. Commands run through `tart exec` (Tart Guest Agent, no SSH). New VMs start right away, and a Linux VM without a desktop gets one in the background whenever PandaCode starts it (`vm/desktop.rs`). The panel draws frames on a canvas: the app CSP allows no `blob:` images, and `tests/vm.spec.ts` runs under that CSP. Linux guests start at 2 GB; idle VMs are suspended (macOS) or stopped (Linux) after `vm::IDLE`, and on quit. Agents get `pandacode_vm_*` tools (`vm/tools.rs`); the inspector's VM tab is `src/vm-panel.ts`, mocked in `src/preview-vm.ts`. Live tests are `#[ignore]`d and take `PANDACODE_VM`/`PANDACODE_VNC`.
 - Cloud sandboxes and deploys live in `src-tauri/src/cloud*`. Machine IDs are `provider:native-id` (`daytona:`, `cloudflare:`, `aws:`, `agentcore:`), so the `pandacode_vm_*` tools and the VM panel route by prefix (`cloud::split`). Keys go through `secret_store.rs` (shared with bots, providers and connections: on macOS one 0600 `secrets.json` in app data, never the Keychain, with old Keychain items moved over once on first read; Secret Service/Credential Manager elsewhere) under `app.pandacode.cloud`; non-secret settings are one JSON value beside them (`cloud::Settings`). Daytona is REST (`cloud/daytona.rs`, toolbox computer use). Docker (`cloud/docker.rs`, non-keyed like AWS/gcloud) runs one task machine per name on a selectable engine (Docker, Podman (`--url` endpoints, `rm --ignore`, array JSON lists), Apple Containers, or the `lxc` LXD/Incus client — which uses its own configured remotes; ownership is the `pc-` name prefix there, labels elsewhere) and endpoint (local daemon, `unix://`, `ssh://` over the shared SSH transport, or TLS `tcp://` for Docker). VPS providers (DigitalOcean, Vultr, Hetzner, and RunPod pods) are plain Linux boxes over the shared SSH transport `cloud/ssh.rs`: one ed25519 keypair in the secure store (`vps_key`, materialized 0600 on use), system `ssh`/`scp`, trust-on-first-use known_hosts in app data, machines tagged/labelled `pandacode` (RunPod instead filters its unfiltered pod list by the `pc-` name prefix), per-machine addresses cached in `ssh::address_cache` with one refresh-and-retry (transport failures carry the `ssh::UNREACHABLE` marker), recycled addresses forgotten from known_hosts (port-aware). Cloudflare deploys `cloud/cloudflare_worker/` (Sandbox SDK 1.0 container Worker, bearer token, 10 min idle) with Wrangler; Cloudflare cannot list Durable Objects, so `cloudflare_sandboxes` remembers names. AWS uses the user's AWS CLI login: EC2 desktops via SSM only (no inbound ports; VNC on the instance's localhost, carried by an SSM port-forward in `cloud/aws_screen.rs`), and AgentCore code interpreters (`cloud/agentcore.rs` signs InvokeCodeInterpreter with SigV4 and reads the event stream). Deploys: `pandacode_cloud_cli` (`cloud/deploy.rs`, approval-gated) runs vercel, wrangler, railway, aws or gcloud in the checkout with credentials only in that process's environment and scrubbed from output (Vercel, Wrangler and Railway use an installed CLI or a pinned `npx` package fetched on first use, never bundled; Railway runs only deploy subcommands in `RAILWAY_ALLOWED`; aws and gcloud use only the user's installed CLI and login, with `aws_refusal`/`gcloud_refusal` and `scrub_aws`/`scrub_gcloud`); the header Deploy menu (`src/deploy.ts`) starts a task with a platform brief. Settings → Agents → Cloud sandboxes is `src/cloud-settings.ts`; preview mocks are in `src/preview-vm.ts`. HTTP tests use `cloud::http::mock`.
 - Cloud agents ([docs/cloud-agents.md](docs/cloud-agents.md)): `src-tauri/src/cloud/sandbox.rs` is the `Sandbox` trait every provider implements (`cloud/<provider>.rs`; Railway and AgentCore can't host agents). `src-tauri/src/cloud_agents/` runs the ordinary provider pipeline over a remote transport: `transport.rs` swaps the harness for PandaCode's own binary in bridge mode (`bridge.rs`, `handle_cloud_bridge` in `main.rs`), which drives `runner.sh` on the machine through byte-acknowledged `inbox` appends and polled `out`; `resume.rs` re-attaches active runs (`sessions.cloud`, schema 27) at startup; `tasks.rs` holds the IPC, prepare/before/after-run steps; `setup.rs`, `credentials.rs` and `sync.rs` install the harness, copy logins and move git bundles to and from the task's mirror worktree. UI: `src/cloud-agents.ts`, the Run on menu in `src/remote-target.ts`, keyed provider forms in `src/cloud-settings.ts`; mocks in `src/preview.ts`/`src/preview-vm.ts`.

diff --git a/README.md b/README.md
@@ -162,6 +162,9 @@ information:
 | PandaCode Connect (optional): GitHub sign-in, then the relay that forwards your devices' end-to-end encrypted messages | `github.com`, the PandaCode relay | Only after you sign in under Settings → Remote access; nothing before |
 | Push notifications (optional, mobile app): the relay sends a generic alert ("A task finished") with a `pandacode://inbox` link to your phone's device token, never task content | Apple Push Notification service, Firebase Cloud Messaging (sent by the PandaCode relay) | Only after you allow notifications in the mobile app while signed in to Connect |
 | Netlify token check (optional): verifies the token and project you enter | `api.netlify.com` | Only when you save a Netlify token in Settings → Agents → Cloud sandboxes |
+| Voice model download (optional): the dictation model you picked, verified by size and SHA-256 | `huggingface.co`, one `github.com` release | Only when you press Download in Settings → Voice; never bundled, never automatic |
+| Mobile voice input uses the phone's own speech recognition; with On-device only off, the phone's speech service may process dictation | Your device's speech service (Apple/Google) | Only while you dictate; PandaCode's relay only ever carries the final text |
+| Online dictation (optional): your recording goes to the provider you configured | That provider's API (OpenAI, Groq, OpenRouter, Together, Mistral, Alibaba Qwen, MiniMax, Deepgram, or your own endpoint) | Only when you pick Online mode, save a key and dictate; never configured by default |
 
 ## Install
 

diff --git a/docs/ipc.md b/docs/ipc.md
@@ -364,3 +364,20 @@ notice. Worker snapshots use the worker's session ID. See
 Task-scoped Run Script discovery, execution, and preview readiness commands are
 documented in [Run and preview](run-project.md). Execution uses the existing
 managed terminal event/lifecycle contract; it never interpolates shell commands.
+
+## Voice dictation
+
+`voice_state()` returns `{settings, models, recording, appleAvailable,
+microphoneAvailable}`; `voice_save_settings(settings)` normalizes and saves
+`voice.json`, returning the normalized settings. `voice_models()` lists the
+catalog with install state; `voice_download_model({id})` streams one model in
+the background, emitting `voice-download {id, status, progress, error?}`.
+`voice_start()` refuses when voice is off, a recording is already running or
+the required models are missing; `voice_stop()` finalizes and
+`voice_cancel()` discards. Online modes add `voice_providers()` (the preset
+registry), `voice_set_key({provider, key})` (secure store; empty key forgets),
+`voice_keys()` (provider ids with saved keys) and
+`voice_test_provider({provider})` (a silence probe through the real
+endpoint). Events: `voice-partial {text}` replaces the live
+text, `voice-result {text, engine, fallback}` settles the transcript,
+`voice-error {message}`, `voice-level f32`. See [voice](voice.md).
diff --git a/docs/voice.md b/docs/voice.md
@@ -0,0 +1,157 @@
+# Voice dictation
+
+A mic button in the composer records your voice and streams the transcript
+into the prompt for review before you send — the dictation model (record →
+review → send), never auto-send and never a voice conversation. Everything
+runs on your computer by default: nothing is bundled, nothing phones home, and
+online transcription exists only if you configure a provider yourself.
+
+## How it works
+
+Native side, `src-tauri/src/voice/`:
+
+- `engine` dispatch by setting: **Apple Speech** (macOS) or **local models**
+  via sherpa-onnx (all platforms).
+- `capture.rs` records the default input device through cpal, downmixes to
+  mono and resamples to 16 kHz; the Apple engine captures with AVAudioEngine
+  instead and feeds `SFSpeechAudioBufferRecognitionRequest` directly.
+- `model.rs` is the catalog of downloadable models; `commands.rs` holds the
+  IPC commands and the recording loop; `settings.rs` is `voice.json`.
+
+## Engines
+
+### Apple Speech (macOS, default)
+
+The OS's own recognizer: no download, live punctuated partials. On-device
+recognition is required whenever the locale supports it
+(`requiresOnDeviceRecognition`); otherwise Apple's service handles it behind
+its own consent prompt, and PandaCode itself still sends nothing anywhere.
+Needs a bundled build (the usage strings live in `src-tauri/Info.plist`, which
+Tauri merges into the generated bundle); an unbundled dev binary refuses
+instead of crashing.
+
+### Local models (sherpa-onnx, all platforms)
+
+Two passes, sherpa-onnx's documented streaming pattern:
+
+1. **Live pass** — a streaming zipformer (`live-en` English, `live-zh-en`
+   Chinese/English) produces partial text while you speak, restored to
+   punctuation and casing by the small `punct-en` model.
+2. **Quality pass** — when recording stops, the chosen offline model
+   re-decodes the whole buffer for the final transcript. **Fast mode** skips
+   this pass and keeps the streaming text.
+
+## Models (downloaded on demand, never bundled)
+
+Settings → Voice lists the catalog with size, languages and license. Files
+download from pinned URLs (Hugging Face repos under `csukuangfj/` plus one
+k2-fsa GitHub release for the punctuation model) with a size cap, SHA-256
+verification where the source publishes digests, and progress events; they
+install under `<app data>/voice/models/<id>/`. Downloaded models stay ready;
+switching the quality model downloads that one on demand.
+
+| Model | Size | Languages | License |
+| --- | --- | --- | --- |
+| Moonshine Base / Tiny | ≈287 / 123 MB | en | MIT |
+| SenseVoice Small (int8) | ≈239 MB | zh, en, ja, ko, yue | FunASR Model License (attribution required) |
+| Parakeet TDT 0.6B v3 (int8) | ≈670 MB | 25 European | CC-BY-4.0 |
+| Whisper Small (int8) | ≈375 MB | 99 | MIT |
+| Live EN / bilingual ZH-EN streamers | ≈73 / 196 MB | en / zh-en | Apache-2.0 |
+| Online punctuation (EN) | ≈31 MB | en | Apache-2.0 |
+
+Defaults follow the language setting: English → Moonshine Base, Chinese/Japanese
+/Korean → SenseVoice, anything else → SenseVoice then Whisper Small. Model
+credits belong to their authors (Useful Sensors, Alibaba FunAudioLLM, NVIDIA,
+OpenAI; runtimes by the k2-fsa sherpa-onnx project).
+
+## Commands and events
+
+Commands (all camelCase args, errors reject as strings): `voice_state`,
+`voice_save_settings`, `voice_models`, `voice_download_model`,
+`voice_start`, `voice_stop`, `voice_cancel`, `voice_providers`,
+`voice_set_key`, `voice_keys`, `voice_test_provider`.
+
+Events: `voice-partial {text}` (replaces the previous live text),
+`voice-result {text, engine, fallback}`, `voice-error {message}`,
+`voice-level f32` (≈10 Hz, 0..1), `voice-download {id, status, progress,
+error?}`. One recording runs at a time; stop/cancel are fire-and-forget and
+the worker clears the session itself.
+
+## Interaction
+
+Tap-to-talk (default) or hold-to-talk, per setting. Esc cancels. The recording
+bar (level meter, timer, Done/Cancel) mounts on the document body so composer
+re-renders never kill a recording, and dictated text works while an agent runs
+(it queues or steers like typed text).
+
+## Online providers (opt-in)
+
+Settings → Voice has three modes: **offline** (default), **online**, and
+**online with offline fallback**. Online transcription exists only when you
+configure it: pick a provider, save its API key (secure store, never the
+webview), and choose a model. Audio then goes to that provider — while you
+speak for the streaming ones, once on stop for the multipart ones — and
+nowhere else.
+
+| Provider | Transport | Live partials | Default model |
+| --- | --- | --- | --- |
+| OpenAI | `POST /v1/audio/transcriptions` | no | `gpt-4o-mini-transcribe` |
+| Groq | OpenAI-compatible multipart | no | `whisper-large-v3-turbo` |
+| OpenRouter | OpenAI-compatible multipart | no | `openai/gpt-4o-mini-transcribe` (25+ models) |
+| Together | OpenAI-compatible multipart | no | `openai/whisper-large-v3` |
+| Mistral | OpenAI-compatible multipart | no | `voxtral-mini-transcribe-3` |
+| Alibaba Qwen | OpenAI-compatible `chat/completions` (base64 audio part) | no | `qwen3-asr-flash` |
+| MiniMax | `POST /v1/speech_to_text` (OpenAI-style multipart, `language` header) | no | `asr-1.0` |
+| Deepgram | streaming WebSocket | yes | `nova-3` |
+| OpenAI Realtime | streaming WebSocket (Together-compatible) | yes | `gpt-4o-mini-transcribe` |
+| Custom | any OpenAI-compatible endpoint (self-hosted incl. localhost) | no | yours |
+
+With fallback on, a failed online attempt (connection error, timeout, mid-stream
+error) keeps the recording and settles it with the local quality model — the
+result is marked as a fallback and a toast says so. Streaming providers send
+16 kHz PCM (Deepgram) or 24 kHz PCM16 (OpenAI Realtime); batch providers send
+one request when you stop — a 16 kHz WAV for the multipart ones, base64 WAV in
+a JSON `input_audio` part for Qwen. Every transport carries the dictation
+language as an ISO-639-1 hint when one is picked (the multipart `language`
+field, MiniMax's `language` header, Qwen's `asr_options.language`, Deepgram's
+`language` query parameter, the realtime session's transcription language);
+**Detect automatically** sends nothing and lets the provider detect.
+
+Two region notes: MiniMax documents both `api.minimax.io` and `api.minimax.cn`
+and which one works depends on where your account was created — set the
+endpoint override to the other host if a request is refused; its
+speech-to-text requires a paid account. Alibaba Qwen defaults to the
+international Singapore compatible-mode host; a China account overrides the
+endpoint to `https://dashscope.aliyuncs.com/compatible-mode/v1` (or its
+workspace host), and clips are capped at 5 minutes. Keys:
+`voice_set_key`/`voice_keys`/`voice_test_provider`
+(the test sends half a second of silence through the real endpoint).
+
+## Mobile
+
+The app uses the phone's own speech recognition (`expo-speech-recognition` →
+Apple Speech / Android `SpeechRecognizer`): on-device when the language
+supports it, with an **On-device only** setting that forbids the OS's network
+service. Partials stream into the composer; only the final transcript is sent,
+as ordinary text over Connect. No audio ever reaches PandaCode's relay, and
+the mic hides on devices with no speech service.
+
+## Privacy
+
+Audio goes exactly where you chose: nowhere (local models), to Apple/Google's
+on-device stack, to the OS's speech service only if you allow it, or — in a
+later online mode you configure yourself — to that provider. The one automatic
+network request Voice ever makes is a model download you clicked. Both are in
+the README privacy table.
+
+## Verification
+
+- `cargo test --lib voice::` — 23 unit tests (settings round-trip, resample,
+  level math, download verification, install detection, language defaults).
+- `npx playwright test tests/voice.spec.ts` — 4 tests (mic button states,
+  live partials into the prompt, cancel, model download and selection, hold
+  mode) against the preview mocks; `tests/settings.spec.ts` covers the tab.
+- `npx tsc --noEmit`, `npm test` for the desktop webview; `cd mobile &&
+  npx tsc --noEmit && npm test && npx expo lint` (4 speech tests).
+- Live checks (ignored tests, need hardware): `PANDACODE_MIC=1 cargo test
+  --lib -- --ignored voice` exercises real capture and transcription.