AnySpeechRecognizer Brings Pluggable Local Speech Models to Apple Platforms

Apple’s new SpeechAnalyzer API, introduced with iOS 26 and macOS 26, gives Swift developers a modern way to process recorded or live audio. Applications can assemble modules such as SpeechTranscriber and SpeechDetector, install model assets when needed, and consume results through asynchronous sequences. The limitation is that the recognition engine remains Apple’s own implementation.

AnySpeechRecognizer keeps much of that programming model while making the engine replaceable. Its package mirrors SpeechAnalyzer, SpeechTranscriber, DictationTranscriber, SpeechDetector, and AssetInventory. For an application already built around those types, moving to another recognizer mainly changes how the transcriber is constructed. Asset installation, analyzer setup, and result handling can remain familiar.

Key points

• Several local engines: The package supports NVIDIA’s Parakeet TDT 0.6B v3, Parakeet Ultra, Parakeet Unified, and Alibaba’s 0.6B and 1.7B Qwen3-ASR models.
• Different language profiles: Parakeet v3 recognizes 25 European languages and detects the language automatically. Qwen3-ASR covers 30 languages and 22 Chinese dialects, including Mandarin, Cantonese, Japanese, and Korean.
• Different streaming trade-offs: Parakeet supports live input, volatile results, word timings, and word confidence. Qwen3-ASR returns one final result for segments of up to about 40 seconds and does not provide those streaming-oriented attributes.
• Built-in diarization: Streaming Sortformer can run alongside Apple’s or Parakeet’s transcriber, track up to four speakers, and assign timed words to speaker turns.
• Core ML distribution: The converted models are available through the Hugging Face Hub and are downloaded only when an application requests them. They use the standard Hugging Face cache.

The supplied LibriSpeech measurements show Parakeet achieving lower word error rates than Apple’s transcriber in the tested file-transcription settings. Live recognition generally trades some accuracy for immediate feedback. The comparison was run on one M4 Max Mac Studio with a limited set of datasets, so it should not be treated as a universal benchmark. Overlapping speech in meetings remained difficult for every tested engine, with the source noting high error rates on AMI recordings.

AnyDecisionModel addresses a separate but related design problem. If an application needs a yes-or-no answer, a choice among alternatives, or a rating, generating a paragraph with a general-purpose language model is often unnecessary. AnyDecisionModel derives typed answers and probabilities from model outputs instead. It can use models running through MLX, Core AI, or llama.cpp, as well as decision models exposed through Jev or Ollama.

Together, the two packages point toward a more modular approach to on-device AI. Developers can choose a speech engine according to language coverage, latency, memory, and privacy requirements, then use a specialized decision model for downstream routing or classification. Core ML makes the deployment fit naturally into Apple hardware, but model size, download time, memory use, and performance on real-world audio still need product-level evaluation. AnySpeechRecognizer does not remove those constraints; it gives Swift applications a more open layer for managing them.

Source: Hugging Face Blog