Speech & Audio

Fish Audio’s open-source Drama 3 combines voice cloning with natural-language direction, multi-character dubbing, iterative revision, speech infilling, and streaming synthesis. Its goal is not merely to read text in a chosen voice, but to support an editable performance workflow.

Apple has disclosed an agreement to make job offers to certain Huxe employees and license the startup’s intellectual property on a non-exclusive basis. The arrangement could support future AI features in Apple Podcasts, although Apple has not confirmed any product plans.

A new approach lets an on-device cascaded voice agent predict tool calls from partial speech-recognition results and execute them while the user is still talking. The cached results help the language model respond sooner without removing the conventional fallback path.

Stability AI is shifting its center of gravity from image generation to tools for music professionals. The company has raised funding from Sony, Warner and Universal, alongside licenses to use their catalogs for training.

YuE2 combines symbolic composition and audio generation in one pipeline: it first writes a readable melody-and-chord score, then turns that plan into a full song with vocals and accompaniment. Its authors report stronger expert preferences when symbolic planning is enabled.

StepAudio 3 Realtime organizes an audio-language model around a continuous listen, converse, think, and act loop. Its reported design combines deep acoustic perception, seamless duplex dialogue, parallel reasoning, and asynchronous tool execution.

Google DeepMind has introduced Gemini 3.8 Live and Gemini 3.8 Live Extended Thinking, two audio models designed for more natural conversations, visual grounding, parallel reasoning, and background tool use. The models are rolling out across developer APIs, Workspace, Search, and Gemini experiences.

Google is broadening its multilingual AI strategy beyond text translation, aiming to capture tone, emotion, code-switching and local speech patterns. The effort combines native audio models, community-led data collection and on-device tools.

X-AuT presents a progressive approach to reducing the depth of audio encoders in speech language models. It combines behavioral probing with representation alignment and cross-scale distillation to limit deletion and premature end-of-sequence errors after pruning.

Xiaomi has open-sourced CocktailASR-1, an industrial-grade target-speaker automatic speech recognition model designed for overlapping conversations. Instead of relying only on stronger noise reduction, it conditions recognition on the identity of the speaker users want to hear.

Brain2Semantics2Text maps sentence-level MEG responses into a semantic embedding space before generating language. The approach prioritizes recovering meaning over reconstructing every phoneme or word.

Indian audio storytelling platform Pocket FM says its annualized revenue run rate has doubled to $500 million in a year. AI now contributes to 93% of its catalog and 99% of new content, while humans continue to shape core stories and intellectual property.

AuK presents an open-source foundation model that uses natural-language instructions and audio context to handle speech generation, editing, enhancement, and acoustic manipulation. Its distilled AuK-Flash variant targets much faster inference with only four steps.

OpenAI’s GPT-Live design separates latency-sensitive media and inference from tools, delegation, persistence, and other application logic. Stateful sessions, WebRTC improvements, and silent production testing are used to make the system scalable without interrupting conversations.

A new study examines why speech recognizers can produce fluent text that is unrelated to the audio. It identifies the final encoder stage as a key point where speech information becomes grounded and readable by the decoder, while showing that disrupting this stage alone does not create natural-sounding hallucinations.

Microsoft researchers introduce VibeVoice-ASR-Streaming, an end-to-end model that combines speech recognition and speaker attribution for low-latency audio processing. It continuously produces speaker-labeled transcripts as speech arrives.

Generative music tools are making electronic dance music easier to produce, but they are also creating a crisis of trust over who actually made a track. Some musicians are now investigating suspicious releases through sonic glitches, visual clues, and platform-specific patterns.

LibriBrain100 expands neural speech research with more than 100 hours of MEG recordings collected while 32 people listened to continuous natural speech. Its design combines unusually deep data from one participant with broader data for testing cross-subject adaptation.

Alibaba has launched CosyVoice Studio, a one-stop AI speech platform that combines transcription, synthesis, audio creation and real-time voice agents. The key shift is not only better speech input or output, but the addition of semantic understanding across the workflow.

Suno plans to add invisible watermarks to AI-generated audio and tighten download rules as pressure mounts from streaming abuse and copyright lawsuits. The move shows how AI music platforms are being pushed toward labeling, accountability, and tighter platform governance.

SwanTale aims to move speech generation beyond single-speaker text-to-speech by combining speaker design, acoustic scenes, and zero-shot voice reuse in one framework. The work targets creative workflows such as dubbing, audio drama, games, ads, podcasts, and short videos.