docs: update README with VAD, new threshold option, and silence handling
Update README to clarify VAD mode behavior: provisional caption only after Silero detects speech, Parakeet idle during silence. Document new --vad-threshold option to control Silero speech confidence. Add troubleshooting tip for music/noise false detections.
This commit is contained in:
@@ -23,7 +23,7 @@ This creates `models/parakeet-tdt-v2/` and `models/silero_vad_v5.onnx`. They are
|
||||
|
||||
## Terminal captions
|
||||
|
||||
VAD mode is recommended for normal use. It redraws a provisional caption while speech is active and commits a timestamped caption after the utterance ends:
|
||||
VAD mode is recommended for normal use. It redraws a provisional caption only after Silero detects speech and commits a timestamped caption after the utterance ends:
|
||||
|
||||
```bash
|
||||
go run ./src/cmd/captioneer --mode=vad
|
||||
@@ -74,6 +74,8 @@ Use `--output=both` to keep terminal output as well:
|
||||
|
||||
In VAD mode, a 500 ms pause finalizes the current speech segment. Captioneer splits that transcript again at sentence punctuation and at word-safe boundaries of at most 64 characters, so continuous speech does not become one enormous paragraph. The live provisional view keeps the latest three digestible chunks and grows vertically without ellipsizing them. Completed chunks remain visible for at least three seconds, receive extra time based on their word count, and gradually fade from white to a restrained gray as they age. When several chunks finalize together, their removal is staggered in reading order instead of deleting the whole block at once. Up to six completed chunks remain above the provisional text; if a seventh arrives, the oldest is removed. Blank recognition results never erase completed text before its reading lifetime expires.
|
||||
|
||||
The expensive Parakeet recognizer stays idle during silence and audio that Silero does not classify as speech. A small rolling preview window also prevents long continuous audio from making each refresh progressively slower. Silero itself continues receiving the live stream because it needs the quiet frames to detect when speech ends and when it starts again.
|
||||
|
||||
Text is left-aligned inside the centered, fixed-width dark bubble and wraps at its configured width instead of extending off-screen. Pointer input passes through the window by default.
|
||||
|
||||
To build outside the helper script, install GTK4 and GTK4 Layer Shell development packages, then run:
|
||||
@@ -105,7 +107,8 @@ Layer Shell and X11 hints are requests, not universal guarantees. Same-layer win
|
||||
| `--chunk-duration=1s` | `1s` | Fixed chunk length or VAD preview refresh interval. Accepts Go durations such as `500ms` or `2s`. |
|
||||
| `--models-dir=models` | `models` | Directory containing `parakeet-tdt-v2/` and `silero_vad_v5.onnx`. |
|
||||
| `--threads=2` | `2` | CPU threads used by recognition and VAD. |
|
||||
| `--preview-threshold-dbfs=-45` | `-45` | RMS threshold that starts a provisional VAD caption. Raise it to reject background audio; lower it for quiet speech. |
|
||||
| `--preview-threshold-dbfs=-45` | `-45` | RMS gate below which recognition stays idle. Raise it to reject quiet background noise; lower it for quiet speech. In fixed mode, gated chunks are not sent to Parakeet. |
|
||||
| `--vad-threshold=0.5` | `0.5` | Silero speech confidence from `0` to `1`. Raise it to reject more music/noise; lower it if real speech is missed. Applies to VAD mode. |
|
||||
| `--model-auto-restart=true` | `true` | Restart the isolated model worker after a crash, capture failure, or timeout. Set `false` to report the failure and exit instead. |
|
||||
| `--model-timeout=30s` | `30s` | Maximum model startup, continuously busy processing time, or missing-heartbeat period before the worker is considered hung. `0s` disables hang detection. |
|
||||
| `--model-shutdown-timeout=2s` | `2s` | Time allowed for pending-audio flush and model cleanup after Ctrl+C or reload before the worker is force-killed. |
|
||||
@@ -138,6 +141,10 @@ Examples:
|
||||
# Fixed two-second captions on a named monitor
|
||||
./scripts/distrobox/run.sh --mode=fixed --chunk-duration=2s \
|
||||
--output=overlay --overlay-monitor=DP-1
|
||||
|
||||
# Be more conservative when instrumental music causes false speech detections
|
||||
./scripts/distrobox/run.sh --mode=vad --output=overlay \
|
||||
--vad-threshold=0.65 --preview-threshold-dbfs=-40
|
||||
```
|
||||
|
||||
## Bazzite development with Distrobox
|
||||
@@ -173,6 +180,7 @@ Captioneer itself has no environment-based application configuration; use the co
|
||||
- **Missing model files:** run `./scripts/download-models.sh` or correct `--models-dir`.
|
||||
- **`pactl` or `parec` missing:** install PulseAudio utilities, or use the Distrobox scripts.
|
||||
- **No captions:** confirm the current sink with `pactl get-default-sink`; Captioneer listens to `<sink>.monitor`.
|
||||
- **Music/noise produces captions:** use VAD mode and try `--vad-threshold=0.65`. Increase it in small steps if false detections continue. `--preview-threshold-dbfs=-40` can also reject quiet background audio, but an RMS gate cannot distinguish loud music from speech. Singing may still be treated as speech.
|
||||
- **Processing overload warning:** inference fell behind and the oldest 100 ms packet was dropped. Try more `--threads` or a longer `--chunk-duration`.
|
||||
- **Overlay is not always above other windows:** read the backend warning. On Wayland this usually means the compositor lacks Layer Shell; add a compositor rule for the `Captioneer Overlay` window if supported.
|
||||
- **Overlay initialization fails:** `--output=overlay` exits with the error. `--output=both` warns and continues terminal-only.
|
||||
|
||||
Reference in New Issue
Block a user