kato/Captioneer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 4 Wiki Activity

docs: update README with detailed desktop features and setup requirements

Browse Source
This commit is contained in:
kato
2026-07-17 14:41:10 +03:00
parent 55fbd357b0
commit 54fb100d3c
35 changed files with 3469 additions and 162 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+66 -19
View File
@@ -7,11 +7,11 @@ Audio and captions stay on your machine. Nothing is uploaded after the one-time
Captioneer provides two commands:
- `captioneer` is the lightweight, terminal-only command and has no GTK dependency.
- `captioneer-desktop` adds a GTK4 subtitle overlay and can output to the overlay, terminal, or both. It is built with the `gtk` build tag.
- `captioneer-desktop` is the GTK4 desktop utility. It manages caption processing, the overlay, caption history, the application Console, and saved settings. It is built with the `gtk` build tag.
## Setup
The terminal command requires Linux x86_64, Go 1.26.2 or newer, `pactl`, and `parec`. Building the desktop command from source additionally needs GTK4 and GTK4 Layer Shell development libraries. Published release archives carry the desktop runtime libraries. PipeWire works through its PulseAudio compatibility layer.
The terminal command requires Linux x86_64, Go 1.26.2 or newer, `pactl`, and `parec`. Building the desktop command from source additionally needs GTK4, GTK4 Layer Shell, GObject Introspection, and X11 development libraries. Published release archives carry the desktop runtime libraries. PipeWire works through its PulseAudio compatibility layer.
Download the large recognition models once:
@@ -58,19 +58,27 @@ kill -HUP <captioneer-parent-pid>
An automatic or manual restart begins a fresh capture/transcription session; audio being processed by the failed worker cannot be recovered.
## Desktop overlay
## Desktop application
The simplest Bazzite development command is:
```bash
./scripts/distrobox/run.sh --mode=vad --output=overlay
./scripts/distrobox/run.sh
```
Use `--output=both` to keep terminal output as well:
The main window has three top-level tabs:
```bash
./scripts/distrobox/run.sh --mode=vad --output=both
```
- **Captions** keeps up to 500 final captions and one mutable live/provisional row. Scrolling upward pauses automatic scrolling; **Jump to Latest** resumes it. Selected text or the full in-memory transcript can be copied.
- **Console** keeps up to 2,000 timestamped diagnostic entries, including startup, capture, model, overload, warning, error, and shutdown messages. It has independent scrolling, copying, and clearing controls.
- **Settings** exposes the meaningful CLI configuration with validation and folder pickers. Advanced recovery settings are collapsed by default. **Apply & Restart Captions** appears when a recognition change requires restarting a running pipeline.
The header contains the running state, current processing mode, a Start/Stop control, and native GTK window controls. Closing the main window exits Captioneer in this stage. Captions start automatically by default, but this can be changed under **Settings → General**. The overlay, GUI history, stdout captions, stderr diagnostics, and timestamped caption files can all be enabled independently.
Caption logs are final captions only. Each session creates a new file named like `captioneer-2026-07-17_13-45-20.000.log`; every line contains its creation timestamp, audio-relative start/end time, and caption. Lines are flushed immediately. The default directory is `$XDG_STATE_HOME/captioneer/transcripts`, or `~/.local/state/captioneer/transcripts` when `XDG_STATE_HOME` is unset.
The interface deliberately uses one compact olive/gray-green, early-2000s desktop theme. It uses GTK client-side decoration for native dragging, double-click behavior, keyboard access, and window controls rather than imitating those behaviors manually. Replace [`assets/icon/icon.png`](assets/icon/icon.png) to change the header/release icon while keeping the same path.
### Caption overlay behavior
In VAD mode, a 500 ms pause finalizes the current speech segment. Captioneer splits that transcript again at sentence punctuation and at word-safe boundaries of at most 64 characters, so continuous speech does not become one enormous paragraph. The live provisional view keeps the latest three digestible chunks and grows vertically without ellipsizing them. Completed chunks remain visible for at least three seconds, receive extra time based on their word count, and gradually fade from white to a restrained gray as they age. When several chunks finalize together, their removal is staggered in reading order instead of deleting the whole block at once. Up to six completed chunks remain above the provisional text; if a seventh arrives, the oldest is removed. Blank recognition results never erase completed text before its reading lifetime expires.
@@ -78,11 +86,11 @@ The expensive Parakeet recognizer stays idle during silence and audio that Siler
Text is left-aligned inside the centered, fixed-width dark bubble and wraps at its configured width instead of extending off-screen. Pointer input passes through the window by default.
To build outside the helper script, install GTK4 and GTK4 Layer Shell development packages, then run:
To build outside the helper script, install GTK4, GTK4 Layer Shell, GObject Introspection, and X11 development packages, then run:
```bash
go build -tags gtk -o build/captioneer-desktop ./src/cmd/captioneer-desktop
./build/captioneer-desktop --mode=vad --output=overlay
./build/captioneer-desktop
```
### Wayland and X11 behavior
@@ -97,13 +105,20 @@ Layer Shell and X11 hints are requests, not universal guarantees. Same-layer win
## Command-line options
`--mode` is required for both commands. `captioneer-desktop` also requires `--output`.
Both commands use built-in defaults, then the saved configuration, then explicit command-line flags. No mode or output flag is required. The compatibility flag `captioneer-desktop --output=terminal|overlay|both` remains available and changes caption destinations for that launch only.
### Configuration options
| Option | Default | Meaning |
| --- | --- | --- |
| `--config=PATH` | XDG path | Read configuration from another JSON file. |
| `--no-config` | `false` | Ignore saved settings and start from built-in defaults for this launch. |
### Transcription options
| Option | Default | Meaning |
| --- | --- | --- |
| `--mode=vad|fixed` | required | Speech-aware utterances or independent fixed chunks. |
| `--mode=vad|fixed` | `vad` | Speech-aware utterances or independent fixed chunks. |
| `--chunk-duration=1s` | `1s` | Fixed chunk length or VAD preview refresh interval. Accepts Go durations such as `500ms` or `2s`. |
| `--models-dir=models` | `models` | Directory containing `parakeet-tdt-v2/` and `silero_vad_v5.onnx`. |
| `--threads=2` | `2` | CPU threads used by recognition and VAD. |
@@ -117,7 +132,13 @@ Layer Shell and X11 hints are requests, not universal guarantees. Same-layer win
| Option | Default | Meaning |
| --- | --- | --- |
| `--output=terminal|overlay|both` | required | Select desktop-command renderers. `both` falls back to terminal if GTK initialization fails. |
| `--output=terminal|overlay|both` | saved outputs | Compatibility preset for overlay and stdout caption destinations. |
| `--gui-history=true|false` | `true` | Retain captions in the bounded GUI history. |
| `--stderr-diagnostics=true|false` | `true` | Mirror structured diagnostics to stderr as well as the Console. |
| `--start-captions=true|false` | `true` | Start processing when the desktop window opens. |
| `--main-window-always-on-top=true|false` | `false` | Best-effort keep-above request for the main window. Supported through X11 window-manager hints; Wayland may reject it. |
| `--caption-log=true|false` | `false` | Save finalized captions to a timestamped session file. |
| `--caption-log-dir=PATH` | XDG state path | Directory for caption session logs. |
| `--overlay-backend=auto|wayland|x11` | `auto` | Let GTK detect the backend, or require one explicitly. |
| `--overlay-opacity=0.90` | `0.90` | Dark bubble opacity from `0` to `1`. |
| `--overlay-font-size=28` | `28` | Font size in logical pixels. |
@@ -132,7 +153,7 @@ Examples:
```bash
# Larger text, a custom font, and longer final-caption visibility
./scripts/distrobox/run.sh --mode=vad --output=overlay \
./scripts/distrobox/run.sh --mode=vad \
--overlay-font-size=34 --overlay-font-family="Noto Sans" --overlay-final-timeout=6s
# Explicit XWayland/X11 route on the cursor's monitor
@@ -143,10 +164,28 @@ Examples:
--output=overlay --overlay-monitor=DP-1
# Be more conservative when instrumental music causes false speech detections
./scripts/distrobox/run.sh --mode=vad --output=overlay \
./scripts/distrobox/run.sh --mode=vad \
--vad-threshold=0.65 --preview-threshold-dbfs=-40
```
## Saved configuration
The GUI saves a readable, versioned JSON file at:
```text
$XDG_CONFIG_HOME/captioneer/config.json
```
When `XDG_CONFIG_HOME` is unset, the normal Linux fallback is `~/.config/captioneer/config.json`. Saving is atomic and creates the directory/file with user-only permissions. Both `captioneer` and `captioneer-desktop` load this file, so ordinary terminal use benefits from the same recognition settings without repeating flags. Explicit flags always win and do not silently rewrite the saved file.
Precedence is unambiguous:
```text
built-in defaults < saved JSON configuration < explicit command-line flags
```
Use `--no-config` for a clean one-off launch or `--config=/path/to/config.json` for a separate profile. The GUI's **Help → Open Configuration Folder** opens the active file's parent directory. Caption history itself remains in memory and does not survive application restarts; timestamped caption logging is the opt-in persistent transcript path.
## Bazzite development with Distrobox
The helper scripts keep GTK development packages off the immutable host. They create and reuse `captioneer-dev` from the pinned `registry.fedoraproject.org/fedora-toolbox:44` image, install only missing packages, enter this mounted repository, and reuse the host's `models/` directory.
@@ -154,7 +193,7 @@ The helper scripts keep GTK development packages off the immutable host. They cr
Run the desktop command from source:
```bash
./scripts/distrobox/run.sh --mode=vad --output=overlay
./scripts/distrobox/run.sh
```
Build both binaries into the host-visible `build/` directory:
@@ -173,7 +212,7 @@ The scripts recognize these environment variables:
| `CAPTIONEER_DISTROBOX_IMAGE` | `registry.fedoraproject.org/fedora-toolbox:44` | Container image. Override it to test a newer Fedora release deliberately. |
| `GSK_RENDERER` | `cairo` in `run.sh` only | GTK renderer. The safe software default avoids host/container GPU DMABUF mismatches; an explicit value is preserved. |
Captioneer itself has no environment-based application configuration; use the command-line flags above. The Fedora image is pinned for reproducibility and will not advance automatically. The name and image overrides apply when creating or selecting the container; an existing named container is reused. Update the image intentionally when Fedora 44 is no longer supported. Distrobox provides the normal home, display, session bus, PipeWire/PulseAudio, and runtime-directory integration; the scripts add no broad socket or device mounts.
Captioneer uses XDG configuration/state locations rather than application-specific environment variables. Use the GUI, JSON file, or command-line flags above. The Fedora image is pinned for reproducibility and will not advance automatically. The name and image overrides apply when creating or selecting the container; an existing named container is reused. Update the image intentionally when Fedora 44 is no longer supported. Distrobox provides the normal home, display, session bus, PipeWire/PulseAudio, and runtime-directory integration; the scripts add no broad socket or device mounts.
## Troubleshooting
@@ -183,7 +222,10 @@ Captioneer itself has no environment-based application configuration; use the co
- **Music/noise produces captions:** use VAD mode and try `--vad-threshold=0.65`. Increase it in small steps if false detections continue. `--preview-threshold-dbfs=-40` can also reject quiet background audio, but an RMS gate cannot distinguish loud music from speech. Singing may still be treated as speech.
- **Processing overload warning:** inference fell behind and the oldest 100 ms packet was dropped. Try more `--threads` or a longer `--chunk-duration`.
- **Overlay is not always above other windows:** read the backend warning. On Wayland this usually means the compositor lacks Layer Shell; add a compositor rule for the `Captioneer Overlay` window if supported.
- **Overlay initialization fails:** `--output=overlay` exits with the error. `--output=both` warns and continues terminal-only.
- **Overlay initialization fails:** disable **Settings → Outputs → Caption overlay** and use GUI history or stdout; details remain available in the Console.
- **Always on top does not work for the main window:** this is a best-effort X11 feature. Generic Wayland intentionally leaves stacking policy to the compositor, and Captioneer reports that limitation in the Console.
- **Settings do not apply:** validation appears beside the Apply button. Recognition changes restart a running pipeline; an overlay backend change is saved but requires restarting the whole application.
- **Caption log is missing:** enable **Save timestamped caption log**, verify its directory, then look for the dated per-session file shown in the Console.
- **A locally built desktop binary misses shared libraries:** use `scripts/distrobox/run.sh` or install the matching GTK4 and GTK4 Layer Shell runtime packages. The desktop command in a published release carries these application libraries in its private `lib/` directory; keep the extracted directory together.
- **The model worker repeatedly restarts:** inspect the warning on stderr. Slow machines may need a larger `--model-timeout`; use `--model-auto-restart=false` when diagnosing a persistent crash so Captioneer exits on the first failure.
@@ -194,8 +236,13 @@ The transcription pipeline emits presentation-neutral events. `output/terminal`
```text
src/app/ capture/model/process lifecycle
src/captions/ caption events and fixed/VAD coordination
src/config/ defaults, CLI precedence, and XDG JSON persistence
src/diagnostics/ bounded structured application log
src/gui/ GTK4 main window, pages, settings, and styling
src/output/history/ bounded GUI caption history
src/output/terminal/ terminal renderer
src/output/overlay/ private GTK4/Layer Shell CGO adapter
src/output/transcript/ timestamped final-caption files
src/cmd/captioneer/ terminal command
src/cmd/captioneer-desktop/ GTK-tagged desktop command
```
@@ -240,4 +287,4 @@ The release workflow uses Gitea's built-in `GITEA_TOKEN`; no personal token is r
Captioneer is licensed under the [Apache License 2.0](LICENSE). Sherpa-Onnx is Apache-2.0 licensed, GTK4 is LGPL-2.1-or-later, and GTK4 Layer Shell is MIT licensed. Release archives dynamically link and redistribute the required shared libraries; `RUNTIME-LIBRARIES.tsv` records their source packages and `third-party-licenses/` contains the installed license notices supplied by those packages. Local source builds continue to link against the system libraries.
Version 1 intentionally has no animation, persisted caption history, runtime dragging, background-free mode, capture exclusion, Flatpak, desktop entry, or autostart integration.
This first GUI stage intentionally has no system tray, start-at-login behavior, global shortcuts, notifications, transcript database, cloud sync, plugin system, remote control, automatic updates, multiple themes, or full terminal emulator. Closing the main window exits the application; background availability belongs to the later tray phase.