Bol: on-device dictation for macOS

Bol runs speech recognition (Parakeet TDT 0.6B v2 on the Neural Engine) and transcript cleanup (S1-mini by Superwhisper, 0.6B) entirely on the Mac. This report compares it with Wispr Flow, a cloud dictation service, on end-to-end latency, and compares its components with alternatives on recognition and cleanup accuracy.

Results

Speech recognition

Transcript cleanup

Evaluation details

Hardware. MacBook Pro, Apple M2 Pro, 16 GB, macOS 27.0.1, on mains power. Both apps installed on the same machine and used with the same built-in microphone.
Latency, Bol. bol-bench (in voice-macos) feeds each recording's samples straight into the same voicecore session the app uses: Silero VAD, the speech engine, the dictionary step and S1-mini cleanup through the bundled llama.cpp helper. The core's own timestamps give end of speech to final text. Two uncounted warm-up runs, then 3 rounds of all 13 recordings per engine; every run is reported. It runs headless, with a throwaway data folder.
Latency, secondary. Each app's own timer on live dictation of the benchmark sentence: Wispr Flow's e2eLatency, Bol's end_to_paste_ms.
Benchmark sentence.
Recordings. 13 recordings of the speaker's real voice (Wispr Flow's own recordings of real dictations, 16 kHz mono): 8 everyday dictations of 3 to 45 seconds and 5 of the benchmark sentence. Synthetic-speech, silent, empty and non-English recordings are excluded. Reference: Wispr Flow's Canto transcript of the same audio, so WER measures disagreement with Canto, not error against ground truth; fillers and number formatting are ignored.
Cleanup set. 410 items (320 en-GB and en-US dictations with fillers, repeats and casing; 90 items covering style, app context, dictionary replacements and category), scored as formatted WER (case- and punctuation-sensitive) against hand-written gold text.
Software. Bol with voicecore built from source; whisper.cpp (Metal) for Whisper; FluidAudio 0.18.2 for Parakeet; llama.cpp b11960 with prompt-lookup decoding for S1-mini. Wispr Flow 1.6. The report is regenerated from the logs by tools/eval/bol_eval.py.

Limitations

One speaker, one machine, and a small number of live runs per system. Differences under about 30 ms are within run-to-run noise at these counts.
Each app times itself; the two definitions of end of speech may differ by a few milliseconds.
The cleanup set contains no self-corrections ("Tuesday, no wait, Wednesday"), which S1-mini resolves in live use, so it understates that ability.
Wispr Flow's own transcription and cleanup accuracy are not scored here: its pipeline cannot be run on the evaluation files.