Captioneer/plan_smoke_test.md

---
name: Desktop smoke harness
overview: Add a Distrobox-based nested Wayland desktop smoke test that boots a headless compositor, plays synthesized speech into a temporary PipeWire null sink, runs captioneer-desktop, and fails on errors or missing captions — fast enough for manual runs, realistic enough to catch integration breakages.
todos:
  - id: smoke-lib
    content: Add scripts/smoke/lib.sh (null sink, espeak fixture, wait helpers, cleanup traps)
    status: completed
  - id: smoke-script
    content: Add scripts/smoke/desktop-smoke.sh orchestrating labwc + captioneer-desktop + assertions
    status: completed
  - id: smoke-deps
    content: On-demand Distrobox package install for labwc/espeak-ng (and ffmpeg if needed)
    status: completed
  - id: smoke-docs-ci
    content: Document in README; ShellCheck the new scripts from scripts/ci/test.sh
    status: in_progress
isProject: false
---

# Desktop smoke harness

## Approach (chosen default)

**Not a full QEMU VM.** A nested Wayland compositor inside the existing Distrobox is the better tradeoff: boots in seconds, still exercises GTK, Layer Shell, Pulse capture, model worker, and caption logging. Full VMs add minutes of boot/image management without much extra signal for this app.

```mermaid
flowchart TD
  host[Host PipeWire] --> nullSink[temp null sink]
  script[desktop-smoke.sh] --> box[captioneer-dev Distrobox]
  box --> labwc[labwc headless Wayland]
  box --> app[captioneer-desktop]
  nullSink --> play[paplay espeak WAV]
  nullSink --> app
  app --> log[stderr diagnostics]
  app --> transcript[caption-log dir]
  script --> assert[pass/fail report]
  log --> assert
  transcript --> assert
```

## What you will run

```bash
./scripts/download-models.sh   # once, if models/ missing
./scripts/distrobox/build.sh   # ensure build/captioneer-desktop exists
./scripts/smoke/desktop-smoke.sh
```

Exit `0` = pass. Prints a short report under `build/smoke/`.

## Harness layout

| Path | Role |
| --- | --- |
| [`scripts/smoke/desktop-smoke.sh`](scripts/smoke/desktop-smoke.sh) | Orchestrator (manual entrypoint) |
| [`scripts/smoke/lib.sh`](scripts/smoke/lib.sh) | Shared helpers: null sink, espeak fixture, wait/assert |
| Extend [`scripts/distrobox/common.sh`](scripts/distrobox/common.sh) | Add smoke packages: `labwc`, `espeak-ng`, `ffmpeg` (or keep install-on-demand in smoke script to avoid bloating every build) |

**Package strategy:** install smoke deps on demand inside `desktop-smoke.sh` via `sudo dnf install -y` in the Distrobox (same pattern as `ensure_dependencies`), so normal `build.sh` / `run.sh` stay lean.

## Test sequence

1. **Preflight**
   - Require Distrobox, `build/captioneer-desktop`, `models/parakeet-tdt-v2/`, `models/silero_vad_v5.onnx`
   - Create work dir `build/smoke/<timestamp>/`

2. **Audio sandbox (host-visible via Distrobox Pulse integration)**
   - `pactl load-module module-null-sink sink_name=captioneer-smoke`
   - Save/restore previous default sink
   - `pactl set-default-sink captioneer-smoke`
   - Synthesize a short clear English line with `espeak-ng` → WAV in the work dir
   - Loop-play with `paplay` for ~15–20s so VAD/fixed mode has time to finalize

3. **Nested desktop**
   - Start `labwc` (wlroots, Layer Shell capable) with `WLR_BACKENDS=headless WLR_LIBINPUT_NO_DEVICES=1`
   - Export `WAYLAND_DISPLAY` to that socket; `GSK_RENDERER=cairo`

4. **Launch Captioneer**
   ```bash
   ./build/captioneer-desktop \
     --no-config \
     --start-captions=true \
     --output=both \
     --stderr-diagnostics=true \
     --caption-log=true \
     --caption-log-dir="$workdir/transcripts" \
     --models-dir=models \
     --mode=vad \
     --model-auto-restart=false
   ```
   - Redirect stderr to `$workdir/captioneer.stderr.log`
   - Present main window first (already true in current [`src/gui/app.go`](src/gui/app.go)), so compositor + GUI come up before capture

5. **Settle + stop**
   - Wait until stderr contains `Model worker ready` (timeout ~60s)
   - Keep playing speech ~15s more
   - Send clean shutdown (`gdbus` `app.exit` or SIGINT)
   - Wait for exit; unload null sink; kill labwc

6. **Assertions (fail hard)**
   - Process exited 0 (or SIGINT-clean)
   - stderr has `Model worker ready` and `Capturing from:`
   - No `ERROR` lines in stderr (allow listed WARNINGs only if needed, e.g. always-on-top on Wayland)
   - At least one non-empty caption log file under the smoke transcript dir **or**, if recognition is too flaky on synthetic speech, fall back to requiring `INFO [lifecycle]` start/stop without ERROR and treat caption text as a soft check printed in the report

**Caption reliability default:** require caption log content. If espeak+VAD proves flaky in practice, switch the harness to `--mode=fixed --chunk-duration=1s` (still in the same script) before softening the assertion — fixed mode is more predictable with continuous playback.

## Report

Write `$workdir/report.txt` with: compositor, sink name, key log greps, transcript excerpt, pass/fail. Always print the report path on exit.

## Docs / CI wiring

- Short “Manual desktop smoke” subsection in [`README.md`](README.md) Development section
- Add `scripts/smoke/desktop-smoke.sh` + `lib.sh` to ShellCheck in [`scripts/ci/test.sh`](scripts/ci/test.sh)
- **Do not** add this to Gitea CI yet (needs models + nested compositor + audio; too heavy/flaky for every PR). Manual-only until it proves stable.

## Out of scope

- Full QEMU/libvirt guest image
- Automating CTRL+drag overlay positioning
- Pixel screenshot OCR of the overlay bubble