Cactus Compute has released Whistle, a speech recognition model small enough to fit in 16.9 megabytes, designed to run entirely on-device on phones, wearables, robots, smart home hubs, and even microcontrollers. What makes it notable isn’t just the size: Whistle shares the same C++ engine, container, and quantisation as the company’s earlier Needle text model, meaning a single binary can now take an audio clip and turn it directly into tool calls, without any server round-trip or extra runtime.

This matters because most speech-to-text systems that developers reach for today, like OpenAI’s Whisper, are built for cloud or desktop-class hardware. Whistle is aimed squarely at the opposite end of the spectrum: constrained, offline, battery-powered devices where every megabyte and millisecond counts.

How Whistle turns audio into text on-device

Whistle processes audio in three stages, all running locally on the CPU with no external dependencies. First, a front end converts 16 kHz mono audio into 80 log-mel frequency bins using 25 ms windows with a 10 ms hop, then a convolutional stem compresses the frame rate down so the model works at one frame per 80 milliseconds instead of processing every raw sample.

Next, an encoder built from eight “Simple Attention” blocks, the same architecture used in Needle, processes the entire clip non-causally, meaning a frame three seconds into the audio can attend to a frame twelve seconds later for better context. Finally, a decoder made of eight “Laddered Simple Attention” blocks generates the actual text, using grouped query attention and memory lookups borrowed directly from Needle’s design.

The one speech-specific addition is a gated cross-attention layer added to each decoder block, which lets the decoder read directly from the encoder’s output. Because the encoder’s key and value projections are computed once per clip and then reused, running five parallel decoding beams costs little more than running one, since the expensive audio-processing step isn’t repeated for each beam.

Decoding, language detection, and keyword biasing

Whistle decodes using five beams scored by length-normalised log probability, with a vocabulary of 8,192 text pieces plus seven language tokens, one for each supported language. Rather than returning the detected language as separate metadata, the model simply emits it as part of the output stream.

A distinctive feature is keyword biasing: developers can pass in specific phrases, and the engine walks an Aho-Corasick automaton (a classic string-matching algorithm) alongside the beam search to boost the probability of those words appearing in the transcript. This is useful for applications that need to reliably catch specific commands, names, or product terms. Transcripts are capped at 320 tokens, and a silence detector checks the clip’s loudness before decoding even starts, returning an empty result immediately for silent audio rather than wasting compute on the beam search.

A scalable model via the “ladder” design

One of Whistle’s more unusual design choices is what Cactus calls a “ladder” on the decoder. Every depth starting from two layers was trained as a fully independent, usable model, and a runtime flag (--audio-depth) selects which depth to load. This gives developers a built-in speed-versus-accuracy dial: shallower decoders run faster for constrained hardware, while deeper ones trade speed for quality. Notably, the encoder itself is never sliced this way; all eight encoder blocks run regardless of the chosen decoder depth.

How it compares to Whisper and Moonshine

Cactus benchmarked Whistle against OpenAI’s Whisper (base) and Moonshine using each model’s own official runtime at default settings. According to the published results, Whistle leads on LibriSpeech test-clean and test-other, SPGISpeech, Earnings-22, and the FLEURS average, while Whisper base comes out ahead on TED-LIUM, AMI, and the MLS average. The headline contrast, though, is size: Whisper base weighs in at 145.3 MB versus Whistle’s 16.9 MB.

Speed numbers tell a similar story. Whisper pads every input to 30 seconds internally, so its time-to-first-token stays flat no matter how short the clip is. Whistle’s time-to-first-token instead scales with the actual clip length: 5.9 ms for a 5-second clip, 11.1 ms for 10 seconds, and 36.3 ms for a full 30-second clip. Word error rates were measured using Whisper’s own normalisation method, with Whistle tested across 86,174 utterances and compared against the published figures for Whisper’s and Moonshine’s multilingual checkpoints. Cactus states it verified that none of the test audio overlapped with Whistle’s training data by comparing audio checksums and speaker IDs.

What this means for developers

For developers building voice features into apps, wearables, or embedded products, Whistle’s pitch is straightforward: a single small file and a single engine that can handle both speech recognition and text generation. Because it loads through the same needle_load function that reads Needle’s model files, a single binary can be configured to do speech, text, or both, which simplifies deployment on hardware where shipping multiple separate ML runtimes isn’t practical.

The practical trade-off, based on the benchmarks Cactus published, is that Whistle isn’t universally more accurate than Whisper base; it wins on several benchmarks and loses on others like TED-LIUM and AMI. Developers choosing between the two will need to weigh that against the roughly 8.6x difference in file size and the latency advantages on short clips, depending on what their specific use case and hardware constraints demand.

Frequently asked questions

What languages does Whistle support?
The model transcribes seven languages, each represented by its own language token that gets emitted as part of the output rather than returned separately.

Can Whistle run on microcontrollers?
Cactus lists microcontrollers among Whistle’s target platforms alongside mobiles, wearables, robots, smart home devices, and automotive systems, with no external dependencies required to run it.

How is Whistle different from using Whisper locally?
Whistle is roughly 8.6 times smaller than Whisper base (16.9 MB versus 145.3 MB), its first-token latency scales with actual clip length rather than always assuming 30 seconds, and it shares a runtime with Cactus’s Needle text model for combined speech-and-text pipelines.

Source: Whistle: Speech to Text in 16.9 MB