kato/Captioneer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 4 Wiki Activity

chore: add Apache License 2.0 and ignore build directory

Browse Source
This commit is contained in:
kato
2026-07-16 17:24:50 +03:00
parent 45e4bce2a0
commit 7affa0dbc8
23 changed files with 1814 additions and 157 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+152 -66
View File
@@ -1,119 +1,205 @@
# Captioneer
Local, live English captions for the audio playing through your default PulseAudio/PipeWire output device. It captures that output's monitor source, runs the Parakeet TDT v2 model locally, and writes captions to the terminal.
Captioneer creates local, live English captions for audio playing through the current default PulseAudio or PipeWire output device. It captures the sink's monitor source and transcribes it with Parakeet TDT v2; VAD mode uses Silero to find speech boundaries.
Everything runs on your machine after the one-time model download. No audio is uploaded anywhere.
Audio and captions stay on your machine. Nothing is uploaded after the one-time model download.
## Requirements
Captioneer provides two commands:
- Linux on `amd64` / x86_64
- Go 1.26.2 or newer
- `pactl` and `parec` available on `PATH` (on Debian/Ubuntu: `sudo apt install pulseaudio-utils`)
- `curl` and `tar` to download the models
- A working PulseAudio-compatible server, including PipeWire's PulseAudio compatibility layer
The application listens to the monitor of the current default audio sink. Check it with:
```bash
pactl get-default-sink
```
- `captioneer` is the lightweight, terminal-only command and has no GTK dependency.
- `captioneer-desktop` adds a GTK4 subtitle overlay and can output to the overlay, terminal, or both. It is built with the `gtk` build tag.
## Setup
Clone the project, then download the required models once:
The terminal command requires Linux x86_64, Go 1.26.2 or newer, `pactl`, and `parec`. The desktop command additionally needs GTK4 and GTK4 Layer Shell development/runtime libraries. PipeWire works through its PulseAudio compatibility layer.
Download the large recognition models once:
```bash
git clone <your-repository-url>
cd captioneer
./scripts/download-models.sh
```
The download installs the Parakeet TDT v2 int8 model and Silero VAD into `models/`. They are intentionally not committed to Git because they are large.
This creates `models/parakeet-tdt-v2/` and `models/silero_vad_v5.onnx`. They are intentionally excluded from Git.
## Run
## Terminal captions
Use VAD mode for normal live captions. It shows a temporary `…` caption while speech is active, then prints a timestamped final caption after 0.5 seconds of silence:
VAD mode is recommended for normal use. It redraws a provisional caption while speech is active and commits a timestamped caption after the utterance ends:
```bash
go run ./src/cmd/captioneer --mode=vad
```
Use fixed mode to decode exact time slices, even through silence:
Fixed mode transcribes independent, exact-duration chunks:
```bash
go run ./src/cmd/captioneer --mode=fixed --chunk-duration=1s
```
Build a reusable executable instead of using `go run`:
Build and run the terminal binary:
```bash
go build -o captioneer ./src/cmd/captioneer
./captioneer --mode=vad
go build -o build/captioneer ./src/cmd/captioneer
./build/captioneer --mode=vad
```
Press <kbd>Ctrl</kbd>+<kbd>C</kbd> to stop. Pending audio is flushed before exit.
Final captions use stdout; diagnostics and overload warnings use stderr. This makes redirection useful:
## Options
```bash
./build/captioneer --mode=vad > captions.txt
```
`--mode` is required. All other options have defaults.
Press <kbd>Ctrl</kbd>+<kbd>C</kbd> to stop. Captioneer stops `parec` and flushes pending audio before exiting.
| Option | Default | Description |
## Desktop overlay
The simplest Bazzite development command is:
```bash
./scripts/distrobox/run.sh --mode=vad --output=overlay
```
Use `--output=both` to keep terminal output as well:
```bash
./scripts/distrobox/run.sh --mode=vad --output=both
```
The overlay shows the latest final caption above the active provisional caption. Final captions remain for four seconds by default. The dark background is 90% opaque, text is centered and limited to two lines per row, and pointer input passes through the window by default. Empty completed captions hide the overlay.
To build outside the helper script, install GTK4 and GTK4 Layer Shell development packages, then run:
```bash
go build -tags gtk -o build/captioneer-desktop ./src/cmd/captioneer-desktop
./build/captioneer-desktop --mode=vad --output=overlay
```
### Wayland and X11 behavior
On KDE Plasma Wayland, GTK4 Layer Shell places Captioneer in the overlay layer near the bottom center without taking keyboard focus. With automatic monitor selection, the compositor chooses the recently active output. A numeric monitor index or connector name can be selected explicitly.
A normal GTK window does work on Wayland. What generic Wayland does **not** provide is a portable way for that window to force itself above all other windows or choose an exact screen position. If Layer Shell is unavailable, Captioneer uses a normal undecorated window and prints an explicit warning; compositor window rules may improve its stacking behavior. It never silently switches to XWayland—use `--overlay-backend=x11` when that route is wanted.
On X11/XWayland, Captioneer selects the monitor containing the mouse cursor in automatic mode and requests notification, keep-above, skip-taskbar, and skip-pager window-manager hints.
Layer Shell and X11 hints are requests, not universal guarantees. Same-layer windows, exclusive-fullscreen games, compositor policy, screenshots, screen sharing, and recording software can all behave differently. Captioneer does not attempt capture exclusion in this version.
## Command-line options
`--mode` is required for both commands. `captioneer-desktop` also requires `--output`.
### Transcription options
| Option | Default | Meaning |
| --- | --- | --- |
| `--mode=vad` | — | Speech-aware captions. This is the recommended everyday mode. |
| `--mode=fixed` | — | Decode every fixed-duration audio chunk. |
| `--chunk-duration=1s` | `1s` | Fixed chunk length, or VAD preview refresh interval. Go duration syntax is used, e.g. `500ms`, `1s`, `2s`. |
| `--mode=vad|fixed` | required | Speech-aware utterances or independent fixed chunks. |
| `--chunk-duration=1s` | `1s` | Fixed chunk length or VAD preview refresh interval. Accepts Go durations such as `500ms` or `2s`. |
| `--models-dir=models` | `models` | Directory containing `parakeet-tdt-v2/` and `silero_vad_v5.onnx`. |
| `--threads=2` | `2` | CPU threads used by Parakeet and VAD. Increase only if your CPU has spare capacity. |
| `--preview-threshold-dbfs=-45` | `-45` | RMS level that begins a provisional VAD caption. Raise it (for example, `-35`) if background audio starts previews; lower it (for example, `-55`) if quiet speech does not. |
| `--threads=2` | `2` | CPU threads used by recognition and VAD. |
| `--preview-threshold-dbfs=-45` | `-45` | RMS threshold that starts a provisional VAD caption. Raise it to reject background audio; lower it for quiet speech. |
### Desktop-only options
| Option | Default | Meaning |
| --- | --- | --- |
| `--output=terminal|overlay|both` | required | Select desktop-command renderers. `both` falls back to terminal if GTK initialization fails. |
| `--overlay-backend=auto|wayland|x11` | `auto` | Let GTK detect the backend, or require one explicitly. |
| `--overlay-opacity=0.90` | `0.90` | Dark bubble opacity from `0` to `1`. |
| `--overlay-font-size=28` | `28` | Font size in logical pixels. |
| `--overlay-bottom-margin=100` | `100` | Bottom margin in logical pixels. |
| `--overlay-max-width=1200` | `1200` | Width cap; the overlay also limits itself to 80% of the selected monitor. |
| `--overlay-monitor=auto` | `auto` | `auto`, a zero-based monitor index, or a connector name such as `DP-1`. |
| `--overlay-final-timeout=4s` | `4s` | Time before the final row disappears. `0s` keeps it until replaced or hidden. |
| `--overlay-click-through=true` | `true` | Use an empty input region where the display backend supports it. |
Examples:
```bash
# Faster fixed chunks
go run ./src/cmd/captioneer --mode=fixed --chunk-duration=500ms
# Larger text, longer final-caption visibility
./scripts/distrobox/run.sh --mode=vad --output=overlay \
--overlay-font-size=34 --overlay-final-timeout=6s
# Fewer VAD preview redraws and four CPU threads
go run ./src/cmd/captioneer --mode=vad --chunk-duration=2s --threads=4
# Explicit XWayland/X11 route on the cursor's monitor
./scripts/distrobox/run.sh --mode=vad --output=both --overlay-backend=x11
# Store or reuse models outside this repository
go run ./src/cmd/captioneer --mode=vad --models-dir=/path/to/models
# Fixed two-second captions on a named monitor
./scripts/distrobox/run.sh --mode=fixed --chunk-duration=2s \
--output=overlay --overlay-monitor=DP-1
```
There are no environment variables to configure. All supported configuration is supplied with command-line options.
## Bazzite development with Distrobox
## Source layout
The helper scripts keep GTK development packages off the immutable host. They create and reuse `captioneer-dev` from the pinned `registry.fedoraproject.org/fedora-toolbox:44` image, install only missing packages, enter this mounted repository, and reuse the host's `models/` directory.
All Go code lives in `src/` as focused packages:
- `cmd/captioneer` is the executable entrypoint.
- `config` defines command-line options.
- `audio` contains PCM conversion and audio calculations.
- `capture` reads the default system-audio monitor.
- `models` owns Parakeet and Silero model resources.
- `captions` handles fixed chunks, VAD segments, and terminal output.
## Output and troubleshooting
Final captions go to standard output in this form:
```text
[12.34s-15.67s] This is a completed caption.
```
Startup messages and overload warnings go to standard error, so you can save final captions separately:
Run the desktop command from source:
```bash
go run ./src/cmd/captioneer --mode=vad > captions.txt
./scripts/distrobox/run.sh --mode=vad --output=overlay
```
- **Missing model files:** run `./scripts/download-models.sh` again.
- **`pactl` or `parec` not found:** install `pulseaudio-utils` and start a PulseAudio-compatible audio server.
- **No captions while audio plays:** verify the default sink with `pactl get-default-sink`; the app records that sink's `.monitor` source.
- **Overload warning:** recognition is behind real time. Try `--threads=4` (or another suitable value) or `--chunk-duration=2s`.
Build both binaries into the host-visible `build/` directory:
## Verify changes
```bash
./scripts/distrobox/build.sh
```
The results are `build/captioneer` and `build/captioneer-desktop`. The terminal binary can normally run directly on Bazzite. The desktop binary is dynamically linked to GTK4 Layer Shell; run it through `run.sh` unless matching runtime libraries are available on the host.
The scripts recognize these environment variables:
| Variable | Default | Meaning |
| --- | --- | --- |
| `CAPTIONEER_DISTROBOX_NAME` | `captioneer-dev` | Dedicated Distrobox container name. |
| `CAPTIONEER_DISTROBOX_IMAGE` | `registry.fedoraproject.org/fedora-toolbox:44` | Container image. Override it to test a newer Fedora release deliberately. |
| `GSK_RENDERER` | `cairo` in `run.sh` only | GTK renderer. The safe software default avoids host/container GPU DMABUF mismatches; an explicit value is preserved. |
Captioneer itself has no environment-based application configuration; use the command-line flags above. The Fedora image is pinned for reproducibility and will not advance automatically. The name and image overrides apply when creating or selecting the container; an existing named container is reused. Update the image intentionally when Fedora 44 is no longer supported. Distrobox provides the normal home, display, session bus, PipeWire/PulseAudio, and runtime-directory integration; the scripts add no broad socket or device mounts.
## Troubleshooting
- **Missing model files:** run `./scripts/download-models.sh` or correct `--models-dir`.
- **`pactl` or `parec` missing:** install PulseAudio utilities, or use the Distrobox scripts.
- **No captions:** confirm the current sink with `pactl get-default-sink`; Captioneer listens to `<sink>.monitor`.
- **Processing overload warning:** inference fell behind and the oldest 100 ms packet was dropped. Try more `--threads` or a longer `--chunk-duration`.
- **Overlay is not always above other windows:** read the backend warning. On Wayland this usually means the compositor lacks Layer Shell; add a compositor rule for the `Captioneer Overlay` window if supported.
- **Overlay initialization fails:** `--output=overlay` exits with the error. `--output=both` warns and continues terminal-only.
- **Desktop binary misses shared libraries on Bazzite:** use `scripts/distrobox/run.sh`; container-built GTK binaries are not self-contained.
## Development
The transcription pipeline emits presentation-neutral events. `output/terminal` and `output/overlay` consume those events independently; GTK stays on its main OS thread while capture and recognition run on a worker.
```text
src/app/ capture/model/process lifecycle
src/captions/ caption events and fixed/VAD coordination
src/output/terminal/ terminal renderer
src/output/overlay/ private GTK4/Layer Shell CGO adapter
src/cmd/captioneer/ terminal command
src/cmd/captioneer-desktop/ GTK-tagged desktop command
```
Run non-GTK verification on the host:
```bash
go test ./...
go build -o captioneer ./src/cmd/captioneer
go build -o build/captioneer ./src/cmd/captioneer
```
Run full verification in the prepared Distrobox:
```bash
repo_root="$(pwd -P)"
distrobox enter --no-workdir captioneer-dev -- bash -lc '
cd -- "$1"
go test -tags gtk ./...
shellcheck scripts/distrobox/common.sh scripts/distrobox/build.sh scripts/distrobox/run.sh
' captioneer-verify "$repo_root"
./scripts/distrobox/build.sh
```
Run this from the repository root. The build and run helpers resolve that path automatically.
## License
Captioneer is licensed under the [Apache License 2.0](LICENSE). Sherpa-Onnx is Apache-2.0 licensed, GTK4 is LGPL-2.1-or-later, and GTK4 Layer Shell is MIT licensed. Captioneer links to the system GTK libraries rather than copying their source into the project.
Version 1 intentionally has no animation, caption history, runtime dragging, background-free mode, capture exclusion, Flatpak, desktop entry, or autostart integration.