Captioneer/README.md

Captioneer

Captioneer creates local, live English captions for audio playing through the current default PulseAudio or PipeWire output device. It captures the sink's monitor source and transcribes it with Parakeet TDT v2; VAD mode uses Silero to find speech boundaries.

Audio and captions stay on your machine. Nothing is uploaded after the one-time model download.

Captioneer provides two commands:

• captioneer is the lightweight, terminal-only command and has no GTK dependency.
• captioneer-desktop is the GTK4 desktop utility. It manages caption processing, the overlay, caption history, the application Console, and saved settings. It is built with the gtk build tag.

Setup

The terminal command requires Linux x86_64, Go 1.26.2 or newer, pactl, and parec. Building the desktop command from source additionally needs GTK4, GTK4 Layer Shell, GObject Introspection, and X11 development libraries. Published release archives carry the desktop runtime libraries. PipeWire works through its PulseAudio compatibility layer.

Download the large recognition models once:

./scripts/download-models.sh

This creates models/parakeet-tdt-v2/ and models/silero_vad_v5.onnx. They are intentionally excluded from Git.

Terminal captions

VAD mode is recommended for normal use. It redraws a provisional caption only after Silero detects speech and commits a timestamped caption after the utterance ends:

go run ./src/cmd/captioneer --mode=vad

Fixed mode transcribes independent, exact-duration chunks:

go run ./src/cmd/captioneer --mode=fixed --chunk-duration=1s

Build and run the terminal binary:

go build -o build/captioneer ./src/cmd/captioneer
./build/captioneer --mode=vad

Final captions use stdout; diagnostics and overload warnings use stderr. This makes redirection useful:

./build/captioneer --mode=vad > captions.txt

Press Ctrl+C to stop. Captioneer stops parec and flushes pending audio before exiting.

Sherpa recognition, Silero VAD, and audio capture run in an isolated worker process. The terminal or GTK process stays responsive if native inference crashes or hangs: Captioneer restarts the worker automatically after a failure, and Ctrl+C force-kills only that worker if it cannot flush within the configured shutdown timeout. Send SIGHUP to the parent Captioneer PID to hot-reload the models without closing the terminal or overlay:

kill -HUP <captioneer-parent-pid>

An automatic or manual restart begins a fresh capture/transcription session; audio being processed by the failed worker cannot be recovered.

Desktop application

The simplest Bazzite development command is:

./scripts/distrobox/run.sh

The main window has three top-level tabs:

• Captions keeps up to 500 final captions and one mutable live/provisional row. Scrolling upward pauses automatic scrolling; Jump to Latest resumes it. Selected text or the full in-memory transcript can be copied.
• Console keeps up to 2,000 timestamped diagnostic entries, including startup, capture, model, overload, warning, error, and shutdown messages. It has independent scrolling, copying, and clearing controls.
• Settings exposes the meaningful CLI configuration with validation and folder pickers. Advanced recovery settings are collapsed by default. Apply & Restart Captions appears when a recognition change requires restarting a running pipeline.

The header contains the running state, current processing mode, a Start/Stop control, and native GTK window controls. Closing the main window exits Captioneer in this stage. Captions start automatically by default, but this can be changed under Settings → General. The overlay, GUI history, stdout captions, stderr diagnostics, and timestamped caption files can all be enabled independently.

Caption logs are final captions only. Each session creates a new file named like captioneer-2026-07-17_13-45-20.000.log; every line contains its creation timestamp, audio-relative start/end time, and caption. Lines are flushed immediately. The default directory is $XDG_STATE_HOME/captioneer/transcripts, or ~/.local/state/captioneer/transcripts when XDG_STATE_HOME is unset.

The interface deliberately uses one compact olive/gray-green, early-2000s desktop theme. It uses GTK client-side decoration for native dragging, double-click behavior, keyboard access, and window controls rather than imitating those behaviors manually. Replace assets/icon/icon.png to change the header/release icon while keeping the same path.

Caption overlay behavior

In VAD mode, a 500 ms pause finalizes the current speech segment. Captioneer splits that transcript again at sentence punctuation and at word-safe boundaries of at most 64 characters, so continuous speech does not become one enormous paragraph. The live provisional view keeps the latest three digestible chunks and grows vertically without ellipsizing them. Completed chunks remain visible for at least three seconds, receive extra time based on their word count, and gradually fade from white to a restrained gray as they age. When several chunks finalize together, their removal is staggered in reading order instead of deleting the whole block at once. Up to six completed chunks remain above the provisional text; if a seventh arrives, the oldest is removed. Blank recognition results never erase completed text before its reading lifetime expires.

The expensive Parakeet recognizer stays idle during silence and audio that Silero does not classify as speech. A small rolling preview window also prevents long continuous audio from making each refresh progressively slower. Silero itself continues receiving the live stream because it needs the quiet frames to detect when speech ends and when it starts again.

Text is left-aligned inside the centered, fixed-width dark bubble and wraps at its configured width instead of extending off-screen. Pointer input passes through the window by default.

To build outside the helper script, install GTK4, GTK4 Layer Shell, GObject Introspection, and X11 development packages, then run:

go build -tags gtk -o build/captioneer-desktop ./src/cmd/captioneer-desktop
./build/captioneer-desktop

Wayland and X11 behavior

On KDE Plasma Wayland, GTK4 Layer Shell places Captioneer in the overlay layer near the bottom center without taking keyboard focus. With automatic monitor selection, the compositor chooses the recently active output. A numeric monitor index or connector name can be selected explicitly.

A normal GTK window does work on Wayland. What generic Wayland does not provide is a portable way for that window to force itself above all other windows or choose an exact screen position. If Layer Shell is unavailable, Captioneer uses a normal undecorated window and prints an explicit warning; compositor window rules may improve its stacking behavior. It never silently switches to XWayland—use --overlay-backend=x11 when that route is wanted.

On X11/XWayland, Captioneer selects the monitor containing the mouse cursor in automatic mode and requests notification, keep-above, skip-taskbar, and skip-pager window-manager hints.

Layer Shell and X11 hints are requests, not universal guarantees. Same-layer windows, exclusive-fullscreen games, compositor policy, screenshots, screen sharing, and recording software can all behave differently. Captioneer does not attempt capture exclusion in this version.

Command-line options

Both commands use built-in defaults, then the saved configuration, then explicit command-line flags. No mode or output flag is required. The compatibility flag captioneer-desktop --output=terminal|overlay|both remains available and changes caption destinations for that launch only.

Configuration options

Option	Default	Meaning
--config=PATH	XDG path	Read configuration from another JSON file.
--no-config	false	Ignore saved settings and start from built-in defaults for this launch.

Transcription options

Option	Default	Meaning
`--mode=vad	fixed`	vad
--chunk-duration=1s	1s	Fixed chunk length or VAD preview refresh interval. Accepts Go durations such as 500ms or 2s.
--models-dir=models	models	Directory containing parakeet-tdt-v2/ and silero_vad_v5.onnx.
--threads=2	2	CPU threads used by recognition and VAD.
--preview-threshold-dbfs=-45	-45	RMS gate below which recognition stays idle. Raise it to reject quiet background noise; lower it for quiet speech. In fixed mode, gated chunks are not sent to Parakeet.
--vad-threshold=0.5	0.5	Silero speech confidence from 0 to 1. Raise it to reject more music/noise; lower it if real speech is missed. Applies to VAD mode.
--model-auto-restart=true	true	Restart the isolated model worker after a crash, capture failure, or timeout. Set false to report the failure and exit instead.
--model-timeout=30s	30s	Maximum model startup, continuously busy processing time, or missing-heartbeat period before the worker is considered hung. 0s disables hang detection.
--model-shutdown-timeout=2s	2s	Time allowed for pending-audio flush and model cleanup after Ctrl+C or reload before the worker is force-killed.

Desktop-only options

Option	Default	Meaning
`--output=terminal	overlay	both`
`--gui-history=true	false`	true
`--stderr-diagnostics=true	false`	true
`--start-captions=true	false`	true
`--main-window-always-on-top=true	false`	false
`--caption-log=true	false`	false
--caption-log-dir=PATH	XDG state path	Directory for caption session logs.
`--overlay-backend=auto	wayland	x11`
--overlay-opacity=0.90	0.90	Dark bubble opacity from 0 to 1.
--overlay-font-size=28	28	Font size in logical pixels.
--overlay-font-family=Sans	Sans	GTK/Pango font family for final and provisional captions.
--overlay-bottom-margin=100	100	Bottom margin in logical pixels.
--overlay-max-width=1200	1200	Fixed caption-bubble width; it is capped at 80% of the selected monitor.
--overlay-monitor=auto	auto	auto, a zero-based monitor index, or a connector name such as DP-1.
--overlay-final-timeout=3s	3s	Completed-caption lifetime; it must be 0s or at least 3s. 0s keeps lines until the six-line cap removes them.
--overlay-click-through=true	true	Use an empty input region where the display backend supports it.

Examples:

# Larger text, a custom font, and longer final-caption visibility
./scripts/distrobox/run.sh --mode=vad \
  --overlay-font-size=34 --overlay-font-family="Noto Sans" --overlay-final-timeout=6s

# Explicit XWayland/X11 route on the cursor's monitor
./scripts/distrobox/run.sh --mode=vad --output=both --overlay-backend=x11

# Fixed two-second captions on a named monitor
./scripts/distrobox/run.sh --mode=fixed --chunk-duration=2s \
  --output=overlay --overlay-monitor=DP-1

# Be more conservative when instrumental music causes false speech detections
./scripts/distrobox/run.sh --mode=vad \
  --vad-threshold=0.65 --preview-threshold-dbfs=-40

Saved configuration

The GUI saves a readable, versioned JSON file at:

$XDG_CONFIG_HOME/captioneer/config.json

When XDG_CONFIG_HOME is unset, the normal Linux fallback is ~/.config/captioneer/config.json. Saving is atomic and creates the directory/file with user-only permissions. Both captioneer and captioneer-desktop load this file, so ordinary terminal use benefits from the same recognition settings without repeating flags. Explicit flags always win and do not silently rewrite the saved file.

Precedence is unambiguous:

built-in defaults < saved JSON configuration < explicit command-line flags

Use --no-config for a clean one-off launch or --config=/path/to/config.json for a separate profile. The GUI's Help → Open Configuration Folder opens the active file's parent directory. Caption history itself remains in memory and does not survive application restarts; timestamped caption logging is the opt-in persistent transcript path.

Bazzite development with Distrobox

The helper scripts keep GTK development packages off the immutable host. They create and reuse captioneer-dev from the pinned registry.fedoraproject.org/fedora-toolbox:44 image, install only missing packages, enter this mounted repository, and reuse the host's models/ directory.

Run the desktop command from source:

./scripts/distrobox/run.sh

Build both binaries into the host-visible build/ directory:

./scripts/distrobox/build.sh

The results are build/captioneer and build/captioneer-desktop. The terminal binary can normally run directly on Bazzite. The desktop binary is dynamically linked to GTK4 Layer Shell; run it through run.sh unless matching runtime libraries are available on the host.

The scripts recognize these environment variables:

Variable	Default	Meaning
CAPTIONEER_DISTROBOX_NAME	captioneer-dev	Dedicated Distrobox container name.
CAPTIONEER_DISTROBOX_IMAGE	registry.fedoraproject.org/fedora-toolbox:44	Container image. Override it to test a newer Fedora release deliberately.
GSK_RENDERER	cairo in run.sh only	GTK renderer. The safe software default avoids host/container GPU DMABUF mismatches; an explicit value is preserved.

Captioneer uses XDG configuration/state locations rather than application-specific environment variables. Use the GUI, JSON file, or command-line flags above. The Fedora image is pinned for reproducibility and will not advance automatically. The name and image overrides apply when creating or selecting the container; an existing named container is reused. Update the image intentionally when Fedora 44 is no longer supported. Distrobox provides the normal home, display, session bus, PipeWire/PulseAudio, and runtime-directory integration; the scripts add no broad socket or device mounts.

Troubleshooting

• Missing model files: run ./scripts/download-models.sh or correct --models-dir.
• pactl or parec missing: install PulseAudio utilities, or use the Distrobox scripts.
• No captions: confirm the current sink with pactl get-default-sink; Captioneer listens to <sink>.monitor.
• Music/noise produces captions: use VAD mode and try --vad-threshold=0.65. Increase it in small steps if false detections continue. --preview-threshold-dbfs=-40 can also reject quiet background audio, but an RMS gate cannot distinguish loud music from speech. Singing may still be treated as speech.
• Processing overload warning: inference fell behind and the oldest 100 ms packet was dropped. Try more --threads or a longer --chunk-duration.
• Overlay is not always above other windows: read the backend warning. On Wayland this usually means the compositor lacks Layer Shell; add a compositor rule for the Captioneer Overlay window if supported.
• Overlay initialization fails: disable Settings → Outputs → Caption overlay and use GUI history or stdout; details remain available in the Console.
• Always on top does not work for the main window: this is a best-effort X11 feature. Generic Wayland intentionally leaves stacking policy to the compositor, and Captioneer reports that limitation in the Console.
• Settings do not apply: validation appears beside the Apply button. Recognition changes restart a running pipeline; an overlay backend change is saved but requires restarting the whole application.
• Caption log is missing: enable Save timestamped caption log, verify its directory, then look for the dated per-session file shown in the Console.
• A locally built desktop binary misses shared libraries: use scripts/distrobox/run.sh or install the matching GTK4 and GTK4 Layer Shell runtime packages. The desktop command in a published release carries these application libraries in its private lib/ directory; keep the extracted directory together.
• The model worker repeatedly restarts: inspect the warning on stderr. Slow machines may need a larger --model-timeout; use --model-auto-restart=false when diagnosing a persistent crash so Captioneer exits on the first failure.

Development

The transcription pipeline emits presentation-neutral events. output/terminal and output/overlay consume those events independently; GTK stays on its main OS thread while capture, VAD, and recognition run in a supervised child process. Caption events and health messages cross a private pipe, and the child is placed in its own process group so its parec process can be stopped or replaced with it.

src/app/                 capture/model/process lifecycle
src/captions/            caption events and fixed/VAD coordination
src/config/              defaults, CLI precedence, and XDG JSON persistence
src/diagnostics/         bounded structured application log
src/gui/                 GTK4 main window, pages, settings, and styling
src/output/history/      bounded GUI caption history
src/output/terminal/     terminal renderer
src/output/overlay/      private GTK4/Layer Shell CGO adapter
src/output/transcript/   timestamped final-caption files
src/cmd/captioneer/      terminal command
src/cmd/captioneer-desktop/ GTK-tagged desktop command

Run non-GTK verification on the host:

go test ./...
go build -o build/captioneer ./src/cmd/captioneer

Run full verification in the prepared Distrobox:

repo_root="$(pwd -P)"
distrobox enter --no-workdir captioneer-dev -- bash -lc '
  cd -- "$1"
  go test -tags gtk ./...
  shellcheck scripts/distrobox/common.sh scripts/distrobox/build.sh scripts/distrobox/run.sh
' captioneer-verify "$repo_root"
./scripts/distrobox/build.sh

Run this from the repository root. The build and run helpers resolve that path automatically.

Manual desktop smoke test

scripts/smoke/desktop-smoke.sh boots a nested headless Wayland desktop (sway) inside the captioneer-dev Distrobox, routes synthesized espeak-ng speech through a temporary PipeWire null sink, runs the real build/captioneer-desktop binary against it, and fails on ERROR diagnostics, an unclean shutdown, or an empty caption transcript.

./scripts/download-models.sh      # once, if models/ is missing
./scripts/distrobox/build.sh      # produce build/captioneer-desktop
./scripts/smoke/desktop-smoke.sh

Exit 0 means pass; a report is written under build/smoke/<timestamp>/report.txt together with the application logs and caption transcripts. The harness installs sway and espeak-ng into the Distrobox on demand and restores the previous default sink when it finishes. If VAD recognition of synthetic speech proves flaky, rerun with CAPTIONEER_SMOKE_MODE=fixed. This test is manual-only and intentionally not part of CI: it needs downloaded models, a nested compositor, and live audio routing.

Gitea Actions and releases

The Tests workflow runs the normal and GTK-tagged Go tests, go vet, and ShellCheck for every pull request and branch push. To make it mandatory for pull requests, open the repository's Settings → Branches, edit the protection rule for master, enable required status checks, and select Tests / Go and GTK tests after its first run. Workflow files can report the check, but the branch-protection rule is what prevents merging when it fails.

Pushing a tag that exactly follows vMAJOR.MINOR.BUG runs the same test suite first, then builds and publishes a Gitea release:

git tag v1.3.519
git push origin v1.3.519

The release contains a checksummed Linux x86_64 archive with captioneer, captioneer-desktop, and a private runtime containing GTK4, GTK4 Layer Shell, Sherpa/ONNX, and their linked non-glibc dependencies. Extract the whole archive and run ./captioneer-desktop; do not move the executable away from its adjacent lib/ directory. Models, pactl, parec, host fonts, the display server, graphics drivers, and glibc remain host-provided.

The release workflow uses Gitea's built-in GITEA_TOKEN; no personal token is required. In Settings → Actions → General, allow the workflow token Read access to code and Write access to releases. Gitea clamps workflow permissions to those repository settings, so a more restrictive maximum produces an actionable HTTP 403 during publishing.

License

Captioneer is licensed under the Apache License 2.0. Sherpa-Onnx is Apache-2.0 licensed, GTK4 is LGPL-2.1-or-later, and GTK4 Layer Shell is MIT licensed. Release archives dynamically link and redistribute the required shared libraries; RUNTIME-LIBRARIES.tsv records their source packages and third-party-licenses/ contains the installed license notices supplied by those packages. Local source builds continue to link against the system libraries.

This first GUI stage intentionally has no system tray, start-at-login behavior, global shortcuts, notifications, transcript database, cloud sync, plugin system, remote control, automatic updates, multiple themes, or full terminal emulator. Closing the main window exits the application; background availability belongs to the later tray phase.