Captioneer/README.md

Captioneer

Captioneer creates local, live English captions for audio playing through the current default PulseAudio or PipeWire output device. It captures the sink's monitor source and transcribes it with Parakeet TDT v2; VAD mode uses Silero to find speech boundaries.

Audio and captions stay on your machine. Nothing is uploaded after the one-time model download.

Captioneer provides two commands:

• captioneer is the lightweight, terminal-only command and has no GTK dependency.
• captioneer-desktop adds a GTK4 subtitle overlay and can output to the overlay, terminal, or both. It is built with the gtk build tag.

Setup

The terminal command requires Linux x86_64, Go 1.26.2 or newer, pactl, and parec. The desktop command additionally needs GTK4 and GTK4 Layer Shell development/runtime libraries. PipeWire works through its PulseAudio compatibility layer.

Download the large recognition models once:

./scripts/download-models.sh

This creates models/parakeet-tdt-v2/ and models/silero_vad_v5.onnx. They are intentionally excluded from Git.

Terminal captions

VAD mode is recommended for normal use. It redraws a provisional caption while speech is active and commits a timestamped caption after the utterance ends:

go run ./src/cmd/captioneer --mode=vad

Fixed mode transcribes independent, exact-duration chunks:

go run ./src/cmd/captioneer --mode=fixed --chunk-duration=1s

Build and run the terminal binary:

go build -o build/captioneer ./src/cmd/captioneer
./build/captioneer --mode=vad

Final captions use stdout; diagnostics and overload warnings use stderr. This makes redirection useful:

./build/captioneer --mode=vad > captions.txt

Press Ctrl+C to stop. Captioneer stops parec and flushes pending audio before exiting.

Desktop overlay

The simplest Bazzite development command is:

./scripts/distrobox/run.sh --mode=vad --output=overlay

Use --output=both to keep terminal output as well:

./scripts/distrobox/run.sh --mode=vad --output=both

In VAD mode, a 500 ms pause finalizes the current caption and starts the next line. The overlay keeps up to six completed lines above the active provisional caption; each completed line remains visible for at least three seconds. If a seventh line arrives sooner, the hard six-line cap removes the oldest. Text is left-aligned inside the centered, fixed-width dark bubble. Pointer input passes through the window by default. Empty completed captions hide the overlay.

To build outside the helper script, install GTK4 and GTK4 Layer Shell development packages, then run:

go build -tags gtk -o build/captioneer-desktop ./src/cmd/captioneer-desktop
./build/captioneer-desktop --mode=vad --output=overlay

Wayland and X11 behavior

On KDE Plasma Wayland, GTK4 Layer Shell places Captioneer in the overlay layer near the bottom center without taking keyboard focus. With automatic monitor selection, the compositor chooses the recently active output. A numeric monitor index or connector name can be selected explicitly.

A normal GTK window does work on Wayland. What generic Wayland does not provide is a portable way for that window to force itself above all other windows or choose an exact screen position. If Layer Shell is unavailable, Captioneer uses a normal undecorated window and prints an explicit warning; compositor window rules may improve its stacking behavior. It never silently switches to XWayland—use --overlay-backend=x11 when that route is wanted.

On X11/XWayland, Captioneer selects the monitor containing the mouse cursor in automatic mode and requests notification, keep-above, skip-taskbar, and skip-pager window-manager hints.

Layer Shell and X11 hints are requests, not universal guarantees. Same-layer windows, exclusive-fullscreen games, compositor policy, screenshots, screen sharing, and recording software can all behave differently. Captioneer does not attempt capture exclusion in this version.

Command-line options

--mode is required for both commands. captioneer-desktop also requires --output.

Transcription options

Option	Default	Meaning
`--mode=vad	fixed`	required
--chunk-duration=1s	1s	Fixed chunk length or VAD preview refresh interval. Accepts Go durations such as 500ms or 2s.
--models-dir=models	models	Directory containing parakeet-tdt-v2/ and silero_vad_v5.onnx.
--threads=2	2	CPU threads used by recognition and VAD.
--preview-threshold-dbfs=-45	-45	RMS threshold that starts a provisional VAD caption. Raise it to reject background audio; lower it for quiet speech.

Desktop-only options

Option	Default	Meaning
`--output=terminal	overlay	both`
`--overlay-backend=auto	wayland	x11`
--overlay-opacity=0.90	0.90	Dark bubble opacity from 0 to 1.
--overlay-font-size=28	28	Font size in logical pixels.
--overlay-bottom-margin=100	100	Bottom margin in logical pixels.
--overlay-max-width=1200	1200	Fixed caption-bubble width; it is capped at 80% of the selected monitor.
--overlay-monitor=auto	auto	auto, a zero-based monitor index, or a connector name such as DP-1.
--overlay-final-timeout=3s	3s	Completed-caption lifetime; it must be 0s or at least 3s. 0s keeps lines until the six-line cap removes them.
--overlay-click-through=true	true	Use an empty input region where the display backend supports it.

Examples:

# Larger text, longer final-caption visibility
./scripts/distrobox/run.sh --mode=vad --output=overlay \
  --overlay-font-size=34 --overlay-final-timeout=6s

# Explicit XWayland/X11 route on the cursor's monitor
./scripts/distrobox/run.sh --mode=vad --output=both --overlay-backend=x11

# Fixed two-second captions on a named monitor
./scripts/distrobox/run.sh --mode=fixed --chunk-duration=2s \
  --output=overlay --overlay-monitor=DP-1

Bazzite development with Distrobox

The helper scripts keep GTK development packages off the immutable host. They create and reuse captioneer-dev from the pinned registry.fedoraproject.org/fedora-toolbox:44 image, install only missing packages, enter this mounted repository, and reuse the host's models/ directory.

Run the desktop command from source:

./scripts/distrobox/run.sh --mode=vad --output=overlay

Build both binaries into the host-visible build/ directory:

./scripts/distrobox/build.sh

The results are build/captioneer and build/captioneer-desktop. The terminal binary can normally run directly on Bazzite. The desktop binary is dynamically linked to GTK4 Layer Shell; run it through run.sh unless matching runtime libraries are available on the host.

The scripts recognize these environment variables:

Variable	Default	Meaning
CAPTIONEER_DISTROBOX_NAME	captioneer-dev	Dedicated Distrobox container name.
CAPTIONEER_DISTROBOX_IMAGE	registry.fedoraproject.org/fedora-toolbox:44	Container image. Override it to test a newer Fedora release deliberately.
GSK_RENDERER	cairo in run.sh only	GTK renderer. The safe software default avoids host/container GPU DMABUF mismatches; an explicit value is preserved.

Captioneer itself has no environment-based application configuration; use the command-line flags above. The Fedora image is pinned for reproducibility and will not advance automatically. The name and image overrides apply when creating or selecting the container; an existing named container is reused. Update the image intentionally when Fedora 44 is no longer supported. Distrobox provides the normal home, display, session bus, PipeWire/PulseAudio, and runtime-directory integration; the scripts add no broad socket or device mounts.

Troubleshooting

• Missing model files: run ./scripts/download-models.sh or correct --models-dir.
• pactl or parec missing: install PulseAudio utilities, or use the Distrobox scripts.
• No captions: confirm the current sink with pactl get-default-sink; Captioneer listens to <sink>.monitor.
• Processing overload warning: inference fell behind and the oldest 100 ms packet was dropped. Try more --threads or a longer --chunk-duration.
• Overlay is not always above other windows: read the backend warning. On Wayland this usually means the compositor lacks Layer Shell; add a compositor rule for the Captioneer Overlay window if supported.
• Overlay initialization fails: --output=overlay exits with the error. --output=both warns and continues terminal-only.
• Desktop binary misses shared libraries on Bazzite: use scripts/distrobox/run.sh; container-built GTK binaries are not self-contained.

Development

The transcription pipeline emits presentation-neutral events. output/terminal and output/overlay consume those events independently; GTK stays on its main OS thread while capture and recognition run on a worker.

src/app/                 capture/model/process lifecycle
src/captions/            caption events and fixed/VAD coordination
src/output/terminal/     terminal renderer
src/output/overlay/      private GTK4/Layer Shell CGO adapter
src/cmd/captioneer/      terminal command
src/cmd/captioneer-desktop/ GTK-tagged desktop command

Run non-GTK verification on the host:

go test ./...
go build -o build/captioneer ./src/cmd/captioneer

Run full verification in the prepared Distrobox:

repo_root="$(pwd -P)"
distrobox enter --no-workdir captioneer-dev -- bash -lc '
  cd -- "$1"
  go test -tags gtk ./...
  shellcheck scripts/distrobox/common.sh scripts/distrobox/build.sh scripts/distrobox/run.sh
' captioneer-verify "$repo_root"
./scripts/distrobox/build.sh

Run this from the repository root. The build and run helpers resolve that path automatically.

License

Captioneer is licensed under the Apache License 2.0. Sherpa-Onnx is Apache-2.0 licensed, GTK4 is LGPL-2.1-or-later, and GTK4 Layer Shell is MIT licensed. Captioneer links to the system GTK libraries rather than copying their source into the project.

Version 1 intentionally has no animation, persisted caption history, runtime dragging, background-free mode, capture exclusion, Flatpak, desktop entry, or autostart integration.