Meta Muse Voice Transcribe Tops Streaming Speech Benchmark but Its API Still Splits the Model

Meta Superintelligence Labs released Muse Voice Transcribe on September 1, its first audio model and the first speech-to-text product on the Meta Model API, and within hours the independent evaluator Artificial Analysis placed it at the top of its streaming speech-to-text leaderboard with a 3.1 percent word error rate delivered 0.16 seconds after a speaker stops talking. Meta priced the hosted model at $3 per 1,000 audio minutes, or $0.18 per hour, with speaker diarization and end-of-speech detection bundled rather than sold as add-ons.

The technical claim behind the launch is more interesting than the leaderboard position. Meta says a single autoregressive model handles transcription, speaker attribution for more than 20 participants, and turn-boundary detection as one continuous token stream, a design that would let voice products drop the three-service pipeline most of them run today. But the model's first week of independent scrutiny turned up a gap between that architecture and the product developers can actually call: in the current API, diarization and endpointing run as separate session modes, so one connection does not yet return both speaker labels and a turn-complete signal. The unification is real at training time. At serving time it is still a roadmap.

That distinction matters because of where Meta says the model is headed. The launch material, staged as an eight-person conversation the model transcribes live, frames reliable multi-speaker listening as the prerequisite for personal AI agents that follow real conversations through Meta's camera glasses rather than waiting for voice commands. That ambition lands while German regulators are actively weighing whether those glasses give bystanders adequate notice that they are being recorded.

A Learned Read-Write Policy Turns Streaming ASR Into Token Prediction

Muse Voice Transcribe belongs to the Muse Spark family of autoregressive multimodal models, which means it generates output one token at a time from a decoder rather than running a dedicated speech encoder feeding a separate text decoder. Meta's blog describes how it makes that architecture work for live audio. Incoming sound is sliced into 80-millisecond chunks, 12.5 per second, and each chunk is converted into a single "soft token," a continuous vector that sits in the model's input sequence alongside ordinary text tokens.

The streaming behavior comes from a decision the model makes after every chunk. It can either emit the next word of the transcript or predict a special <|next_audio|> token, which the serving system replaces with the actual next chunk of audio before the model continues. When the microphone stream ends, the system inserts an <|empty_audio|> token and the model flushes whatever text remains.

Researchers in simultaneous translation have called this kind of mechanism a read/write policy: at each step the system decides whether to read more input or write more output, and the quality of that decision governs both latency and accuracy. Earlier streaming recognizers typically fixed the trade-off in advance with a set chunk size or look-ahead window. Meta's approach makes the read decision itself a learned prediction, so the amount of audio the model hears before committing to a word can differ from one word to the next.

Meta calls the result "adaptive delay" and says it was trained with reinforcement learning using two rewards, one for word error rate and one for delay, combined multiplicatively rather than added together. The multiplicative form is a deliberate constraint: a policy cannot buy accuracy by waiting indefinitely or buy speed by guessing early, because collapse on either reward drags the product toward zero. In practice, Meta says, easy words come out quickly and acoustically or lexically ambiguous ones get more listening time. The company's own scatter plot places the model below the previous speed-accuracy Pareto frontier formed by Soniox, Cartesia and ElevenLabs systems, a claim the Artificial Analysis numbers are consistent with on the final-transcript metric.

Diarization and Endpointing Become Extra Vocabulary, Not Extra Models

Once transcription is a token-generation problem, Meta's blog explains, adding other audio-perception tasks means adding special tokens and training the model to emit them at the right moments.

For speaker attribution the model uses two token types. A <|start_of_turn|> token is predicted as soon as the model detects a speaker change. A <|speaker_A|> through <|speaker_Z|> tag is then predicted at the end of the current chunk to label who was talking. Meta notes that audio from the same speaker may be broken into several turn segments, but the label stays consistent across them. Two engineering consequences follow directly. The label space is 26 identifiers, which is why the company advertises "20+" speakers rather than an arbitrary count. And the tag arrives later than the turn marker, so an application that needs the speaker's identity immediately at the start of a turn will see a short lag between "someone new is talking" and "it is Speaker C."

For endpointing, a <|speech_onset|> token marks when someone begins speaking and a <|speech_endpoint|> token marks when they finish. That endpoint signal is what a voice agent uses to decide it is its turn to reply. Systems without it either add a separate voice-activity detector or infer turn completion from silence, which cuts off callers who pause mid-thought. Meta says all three tasks are trained together with streaming recognition, with additional rewards for diarization and endpointing layered on top of the base transcription reward.

Traditional diarization works differently. Most production systems extract a speaker embedding for each short segment and cluster those embeddings after the fact, which is why speaker labels in many meeting tools only stabilize once the recording ends. A law-firm intake integrator, Layer3Labs, wrote after testing the API that the recurring failure in its workflows had never been raw accuracy but the fact that attribution "arrived only after the call ended," blocking anything a live workflow wanted to do with speaker identity.

The Accuracy Lead Is Independently Verified, English-Only, and 0.3 Points Wide

The single most defensible number in the launch is the one Meta did not produce itself. Artificial Analysis, which runs a public streaming speech-to-text leaderboard with published methodology, measured Muse Voice Transcribe at a 3.06 percent final-transcript word error rate with 0.163 seconds between detected end of speech and final text, rounding to the 3.1 percent and 0.16 seconds Meta cites. On the same test, Cartesia Ink-2 in its semantic-endpoint configuration scored 3.4 percent at 0.43 seconds, ElevenLabs Scribe v2 Realtime 3.6 percent at 0.14 seconds, Alibaba's Qwen3 ASR Flash Realtime 3.7 percent, OpenAI's GPT Live Transcribe and xAI's Grok streaming model 3.9 percent, and Google's Gemini 3.5 Transcribe Live and AssemblyAI's Universal-3.5 Pro Realtime 4.0 percent.

Three caveats bound that result. First, the leaderboard is an English-weighted test. Meta trained the model on more than 70 languages and recommends 25 that it says were extensively validated, but none of the non-English claims are measured by the benchmark that produced the ranking. Second, the margin over the runner-up is three-tenths of a percentage point. Third, the leaderboard's own data shows how much the endpointing strategy alone moves the numbers: Cartesia's model appears twice, at 3.4 percent and 0.43 seconds when it detects utterance boundaries itself and at 4.0 percent and 0.07 seconds when an external system does, a swing on both axes larger than Meta's lead. ElevenLabs remains marginally faster than Meta at slightly lower accuracy.

The diarization figure is a different category of evidence. Meta reports a 17.5 percent average diarization error rate across three public benchmarks, AMI-IHM, AMI-SDM and VoxConverse, against 21.1 to 28.6 percent for five competing systems in its chart. Those benchmarks are public and the numbers are plausible, but Meta ran the comparison; Artificial Analysis did not independently reproduce it. A 17.5 percent error rate also means that on hard multi-party audio roughly one minute in six is attributed to the wrong person, which is why the strongest launch coverage described speaker separation as the weaker column across the entire field rather than a solved problem.

The "20+ speakers" capacity is likewise a stated capability rather than a demonstrated one. Meta's live demo uses eight speakers and its hour-long sample recording carries 11 labels. VentureBeat's survey of vendor documentation found that Speechmatics supports 50 speakers by default and 100 when configured, and Amazon Transcribe handles 30 in streaming mode, so the number is high but not a ceiling record. Speaker capacity and attribution accuracy are separate questions, and Meta's benchmark does not test rivals at their advertised maximums.

One independent hands-on test adds texture the leaderboard cannot. Kingy.ai ran a bounded 30-minute evaluation on public audio, paid for its own API access, and found Muse Voice Transcribe cut English word errors 22.2 percent relative to a locally run Whisper large-v3-turbo. The same test found a Hindi-English mixed-script transcript scored substantially worse than Whisper, and concluded that its diarization and endpointing proxies were not reliable enough for unattended high-stakes use.

The Shipped API Exposes Diarization and Endpointing as Separate Modes

The architecture section of Meta's blog describes one model producing transcript, speaker tags and endpoint tokens in a single sequence. The developer documentation describes something narrower. According to VentureBeat's review of the Meta Model API docs, diarization is a first-class operating mode alongside push-to-talk and endpointing modes, speaker labels are session-scoped letters rather than persistent identities, and timestamps are provided per turn rather than per word. Kingy.ai's test reached the same conclusion more bluntly: endpointing and diarization are separate session modes, and buyers should not assume one connection returns both simultaneously.

That is the largest gap between the launch narrative and the launch product. The pitch to enterprises is pipeline collapse, replacing a transcriber, a diarizer and a voice-activity detector with one call. For a dictation app or a single-speaker voice assistant, one mode suffices. For the use case Meta itself leads with, an agent following a multi-person conversation and deciding when to speak, an application needs speaker labels and turn-complete signals from the same stream. The model was trained to emit both; the API, as documented in its first week, asks developers to choose.

Other documented limits shape what can be built today. The API defaults to eight concurrent streams per tenant, caps real-time sessions at 60 minutes before an application must reconnect, and does not expose word-level confidence scores, sound-event detection or emotion tags. Meta's September 3 developer post adds that the endpoint supports both WebSocket streaming and one-shot transcription of pre-recorded files, includes voice-activity detection, accepts keyword biasing for names and domain terms, and is compatible with the OpenAI software development kit, so an existing transcription client typically needs only a new base URL and model name.

None of these are unusual constraints for a version 1.0 API, and the SDK compatibility means the cost of finding out how the limits affect a given workload is an afternoon of testing on a team's own worst audio. But they are the reason the model should be evaluated as a strong new transcription endpoint rather than as the finished unified perception layer its research description implies.

Muse Voice Transcribe Pricing Undercuts Most Rivals but Not All of Them

Meta charges $3 per 1,000 audio minutes for both streaming and batch transcription, bills only for audio actually processed rounded down to the second, and offers zero-data-retention processing at the same price. At $0.18 per hour, 1,000 hours of processed audio costs roughly $180.

VentureBeat normalized public streaming rates to a per-hour figure and found Meta near, but not at, the bottom of the market. Soniox's stt-rt-v5 publishes an equivalent of about $0.12 per hour with diarization for up to 15 speakers included. Above Meta sit xAI at roughly $0.20, Speechmatics at $0.24, Deepgram's Nova-3 at about $0.35 plus $0.12 for diarization, ElevenLabs Scribe v2 Realtime at $0.39 pay-as-you-go with no real-time diarization, AssemblyAI at $0.45 plus $0.12 for diarization capped at 10 speakers, Gemini 3.5 Transcribe Live at an estimated $0.54 blended token cost with no live diarization, Amazon Transcribe streaming at around $0.60, and OpenAI's GPT Live Transcribe at $1.02 with diarization not listed as a capability. Cartesia sells Ink-2 through monthly credit plans that work out to roughly $0.54 per hour if fully consumed on transcription, which is not directly comparable to a metered rate.

The competitive point is narrower than "cheapest" but sharper. Meta is the only vendor that sits at the top of the independent English accuracy table while bundling real-time diarization for more than 20 speakers at under $0.20 per hour. Against the two frontier labs whose transcription products compete most directly, the gap is stark: roughly a third of Google's estimated cost with better streaming accuracy and live speaker labels Google's live endpoint does not offer, and under a fifth of OpenAI's price.

This is the same playbook Meta has run across the Muse Spark text models, where each release has held pricing flat while climbing the Artificial Analysis Intelligence Index, and it has now been applied to the first non-text modality out of Meta Superintelligence Labs.

Read more: Meta Muse Spark 1.3 undercuts rivals on price

Meta Beats Gemini 3.5 Transcribe on the Axes That Matter for Multi-Party Audio

The launch arrived six days after Google moved its transcription capability into the core Gemini API with Gemini 3.5 Transcribe, and the two products are worth reading together because they make opposite bets.

Google's consolidation put diarization, word-level timestamps and function calling into one Gemini model billed through the same token system as the rest of the API, and its batch model ranked fifth on Artificial Analysis's offline leaderboard at 2.6 percent WER. But its batch path supports speaker attribution for at most three participants, its one-hour audio cap drops to 30 minutes when diarization or timestamps are enabled, and its live path does not support diarization at all.

Meta's product inverts those priorities. It gives up word-level timestamps and token-metered flexibility, and its offline accuracy has not been benchmarked at the top of the table, but it leads the streaming test, supports sessions beyond an hour, labels more than 20 speakers live, and costs a fixed fraction of Google's estimated rate. For a meeting-intelligence or contact-center product, where the hard problem is who said what while the conversation is still running, Meta's trade-offs are the more relevant ones. For a single-speaker dictation product that wants polished, filler-free text, Google's editing behavior and tighter offline accuracy may matter more.

Read more: Google folds speech-to-text into the Gemini API

OpenAI's position is the weakest of the three on this specific workload. Its GPT Live Transcribe model trails on the streaming leaderboard at 3.9 percent, does not list diarization, and costs more than five times Meta's rate, even after the transcription price cuts it made in July. The specialist vendors, ElevenLabs, Cartesia, AssemblyAI, Deepgram and Soniox, remain competitive on individual axes, and Soniox in particular undercuts Meta on price, but none combines Meta's accuracy ranking with bundled high-capacity live diarization.

Whisper deserves separate mention because it is the model most teams already run. OpenAI's open-weight Whisper large-v3-turbo is a batch model; self-hosted streaming wrappers typically land at one to five seconds of latency, and the base model provides no diarization, no punctuation by default, and no endpointing, so a production stack assembles those parts separately. Kingy.ai's test found Muse Voice Transcribe's argument against Whisper is not primarily accuracy, though its English result favored Meta. It is whether the pipeline should run as a batch job on the team's own hardware or stream through a hosted API, and Meta has made the hosted answer cheap enough to be tempting for teams that were self-hosting to save money rather than to keep audio on premises.

Closed Weights and Hosted-Only Delivery Draw the Line Meta's Open Strategy Will Not Cross

A Meta spokesperson confirmed to The New Stack that, unlike the company's Muse Glimmer agent model released under Apache 2.0 in August, Muse Voice Transcribe's weights will not be published. Meta has also not disclosed the model's parameter count, training data volume or audio data sources.

The pattern across Meta's 2026 releases is now consistent: open weights for distilled, smaller models that seed the developer ecosystem, closed weights and metered pricing for the frontier text models and, now, for the first specialty audio model. For buyers the consequence is concrete. Any organization that cannot move raw audio off its own infrastructure, whether for regulatory, contractual or sovereignty reasons, has no path to this model, and zero-data-retention processing addresses storage rather than transit. Teams in that position will continue assembling open pipelines from Whisper-class recognizers and open diarizers, accepting the latency and integration cost Meta's product is designed to eliminate.

The Glasses Roadmap Collides With Germany's Bystander-Consent Fight

Meta's scripted launch demo is unusually explicit about purpose. Company employees, transcribed live by the model, argue that for AI to be "personal" it has to "listen like a human on AI glasses in real conversations, not just voice commands," that real conversations are "messy" with overlaps and interruptions, and that this is why streaming endpointing and diarization matter. The framing traces back to Mark Zuckerberg's August essay on personal superintelligence, and the engineering choice it implies is that the model is being built to attribute speech among many people in a room, most of whom are not the user.

That is precisely the capability under regulatory examination in Germany. Hamburg's data protection commissioner, Thomas Fuchs, whose office is Meta's lead regulator in the country, tested the Ray-Ban Meta glasses this summer, concluded the recording indicator LED is too easy for bystanders to miss, and determined that filming people in public with the device breaches German data-protection law; his office has issued fines to individual users. On August 12 the advocacy group HateAid filed a criminal complaint against Meta, Ray-Ban, Oakley and four German retailers with the Central Office for Combating Internet Crime in Frankfurt, seeking a sales ban under the telecommunications and digital-services privacy statute that Germany previously used to remove a voice-recording children's doll from the market.

The Decoder's account of this thread, that Germany's Federal Network Agency "decided not to pursue" a ban, overstates the finality. What the Bundesnetzagentur has said, in a mid-August statement to the Frankfurter Allgemeine Zeitung, is that for the glasses examined so far it does not consider the conditions for a ban to be met and that no formal proceedings are under way. A subsequent report by WeSpeakIoT noted the agency had not physically inspected the current Wayfarer Gen 2 model and could not yet assess the device's AI functions, which Meta's own bystander-privacy documentation describes as operating with the capture LED off. The complaint remains on file, and the European Data Protection Board has been preparing a report on smart glasses that was expected by the end of summer.

Meta has not announced any integration of Muse Voice Transcribe into its glasses, and the model today ships only in the Mac app, Muse Code and the developer API. The point is not that the model is unlawful; transcription is transcription. The point is that a model whose stated design target is continuous attribution of speech among 20 or more people is the software half of a product whose hardware half is being challenged on exactly the question of whether those people know they are being processed. Meta's decision to demonstrate the model as an ambient listener rather than as a dictation engine tells regulators what the roadmap is before the roadmap exists.

Independent Evaluation Has Yet to Reproduce the Diarization and Language Claims

The near-term questions are measurable. Artificial Analysis or another independent evaluator has not yet reproduced Meta's diarization numbers or tested the model at 20 simultaneous speakers, and no per-language word error rates exist for the 25 validated languages, let alone the other 45 in the training set. Kingy.ai's Hindi-English regression is a single 30-minute data point, but it is the only independent code-switching measurement available, and it cuts against the launch narrative. The margin on English streaming accuracy is small enough that a single rival update, or a change in how Artificial Analysis handles endpointing configurations, could reorder the top of the table.

The product questions are Meta's to answer. Whether a future API version returns diarization and endpointing on one stream, adds word-level timestamps and confidence, and lifts the eight-stream default will determine whether the pipeline-collapse pitch becomes something a voice-agent developer can build on without a second vendor. The September 3 developer post reads as an integration guide rather than a roadmap, and no changelog commitments were made.

What Muse Voice Transcribe changes today is the reference price for accurate hosted streaming transcription with speaker labels included, and it changes it for the two frontier labs more than for the specialists. What it does not yet change is the shape of a multi-party voice pipeline, which still needs the developer to choose between knowing who is talking and knowing when they have finished. The next signal worth watching is not another leaderboard snapshot but the first API revision that lets one WebSocket carry both.