Running Whisper locally on a Mac
Notes from running local Whisper inside a Mac dictation workflow. Which model, how long you wait, and where the audio goes.
Whisper is the speech recognition model OpenAI released in 2022. Because it is public, you can run it on your own Mac. Yubi uses it as its second engine, and it turns speech into text without sending audio anywhere.
What you need
An Apple Silicon Mac and whisper.cpp, a lean C++ implementation that uses the Mac's neural engine. The current build of Yubi calls the whisper.cpp command line tool installed on your Mac and a model file you download once. Yubi does not bundle them yet. That is something I want to change.
Which model
Whisper comes in several sizes. Smaller is faster, larger is more accurate. On my Mac the small models dropped Japanese particles and sentence endings often enough that I settled on a larger one. Yubi uses a quantized build of large-v3-turbo, which keeps most of the large model's accuracy in a smaller, faster file.
How long you wait
Unlike SpeechAnalyzer, Whisper starts after you stop talking. For a short sentence you barely notice. For a long note it feels like one breath after you finish. If that pause bothers you, use SpeechAnalyzer instead, which is why Yubi supports both.
What worked better for me
Notes taken in a cafe or on a train platform hold together better with Whisper on my Mac. It also seems to cope well with long explanations full of restarts. These are observations from my own use, not a benchmark. For a quick one liner, I use SpeechAnalyzer and skip the wait.
Cost
The model is public and the work happens on your Mac, so there is no transcription service fee. Cloud services charge by the hour or by the month. Run Whisper locally and that bill does not exist.
Where the audio goes
When Yubi transcribes with Whisper, it writes the audio to a temporary file, processes it on the Mac, and deletes the file when the transcription is done. Nothing is uploaded. There is no setting called send to cloud, because there is nowhere to send it.
Why not the Mac App Store
The current build of Yubi calls a whisper-cli installed on the Mac, and the App Store sandbox does not allow that. So Yubi is a direct download. The major dictation apps I looked at are distributed the same way.