Captioneer/plan_smoke_test.md

name, overview, todos, isProject

name	overview	todos	isProject
Desktop smoke harness	Add a Distrobox-based nested Wayland desktop smoke test that boots a headless compositor, plays synthesized speech into a temporary PipeWire null sink, runs captioneer-desktop, and fails on errors or missing captions — fast enough for manual runs, realistic enough to catch integration breakages.	id content status smoke-lib Add scripts/smoke/lib.sh (null sink, espeak fixture, wait helpers, cleanup traps) completed id content status smoke-script Add scripts/smoke/desktop-smoke.sh orchestrating labwc + captioneer-desktop + assertions completed id content status smoke-deps On-demand Distrobox package install for labwc/espeak-ng (and ffmpeg if needed) completed id content status smoke-docs-ci Document in README; ShellCheck the new scripts from scripts/ci/test.sh in_progress	false

Desktop smoke harness

Approach (chosen default)

Not a full QEMU VM. A nested Wayland compositor inside the existing Distrobox is the better tradeoff: boots in seconds, still exercises GTK, Layer Shell, Pulse capture, model worker, and caption logging. Full VMs add minutes of boot/image management without much extra signal for this app.

flowchart TD
  host[Host PipeWire] --> nullSink[temp null sink]
  script[desktop-smoke.sh] --> box[captioneer-dev Distrobox]
  box --> labwc[labwc headless Wayland]
  box --> app[captioneer-desktop]
  nullSink --> play[paplay espeak WAV]
  nullSink --> app
  app --> log[stderr diagnostics]
  app --> transcript[caption-log dir]
  script --> assert[pass/fail report]
  log --> assert
  transcript --> assert

What you will run

./scripts/download-models.sh   # once, if models/ missing
./scripts/distrobox/build.sh   # ensure build/captioneer-desktop exists
./scripts/smoke/desktop-smoke.sh

Exit 0 = pass. Prints a short report under build/smoke/.

Harness layout

Path	Role
scripts/smoke/desktop-smoke.sh	Orchestrator (manual entrypoint)
scripts/smoke/lib.sh	Shared helpers: null sink, espeak fixture, wait/assert
Extend scripts/distrobox/common.sh	Add smoke packages: labwc, espeak-ng, ffmpeg (or keep install-on-demand in smoke script to avoid bloating every build)

Package strategy: install smoke deps on demand inside desktop-smoke.sh via sudo dnf install -y in the Distrobox (same pattern as ensure_dependencies), so normal build.sh / run.sh stay lean.

Test sequence

1. Preflight

   • Require Distrobox, build/captioneer-desktop, models/parakeet-tdt-v2/, models/silero_vad_v5.onnx
   • Create work dir build/smoke/<timestamp>/

2. Audio sandbox (host-visible via Distrobox Pulse integration)

   • pactl load-module module-null-sink sink_name=captioneer-smoke
   • Save/restore previous default sink
   • pactl set-default-sink captioneer-smoke
   • Synthesize a short clear English line with espeak-ng → WAV in the work dir
   • Loop-play with paplay for ~15–20s so VAD/fixed mode has time to finalize

3. Nested desktop

   • Start labwc (wlroots, Layer Shell capable) with WLR_BACKENDS=headless WLR_LIBINPUT_NO_DEVICES=1
   • Export WAYLAND_DISPLAY to that socket; GSK_RENDERER=cairo

4. Launch Captioneer

   ./build/captioneer-desktop \
     --no-config \
     --start-captions=true \
     --output=both \
     --stderr-diagnostics=true \
     --caption-log=true \
     --caption-log-dir="$workdir/transcripts" \
     --models-dir=models \
     --mode=vad \
     --model-auto-restart=false
   

   • Redirect stderr to $workdir/captioneer.stderr.log
   • Present main window first (already true in current src/gui/app.go), so compositor + GUI come up before capture

5. Settle + stop

   • Wait until stderr contains Model worker ready (timeout ~60s)
   • Keep playing speech ~15s more
   • Send clean shutdown (gdbus app.exit or SIGINT)
   • Wait for exit; unload null sink; kill labwc

6. Assertions (fail hard)

   • Process exited 0 (or SIGINT-clean)
   • stderr has Model worker ready and Capturing from:
   • No ERROR lines in stderr (allow listed WARNINGs only if needed, e.g. always-on-top on Wayland)
   • At least one non-empty caption log file under the smoke transcript dir or, if recognition is too flaky on synthetic speech, fall back to requiring INFO [lifecycle] start/stop without ERROR and treat caption text as a soft check printed in the report

Caption reliability default: require caption log content. If espeak+VAD proves flaky in practice, switch the harness to --mode=fixed --chunk-duration=1s (still in the same script) before softening the assertion — fixed mode is more predictable with continuous playback.

Report

Write $workdir/report.txt with: compositor, sink name, key log greps, transcript excerpt, pass/fail. Always print the report path on exit.

Docs / CI wiring

• Short “Manual desktop smoke” subsection in README.md Development section
• Add scripts/smoke/desktop-smoke.sh + lib.sh to ShellCheck in scripts/ci/test.sh
• Do not add this to Gitea CI yet (needs models + nested compositor + audio; too heavy/flaky for every PR). Manual-only until it proves stable.

Out of scope

• Full QEMU/libvirt guest image
• Automating CTRL+drag overlay positioning
• Pixel screenshot OCR of the overlay bubble