Nemotron 3.5 ASR vs Whisper: Which Speech-to-Text to Use

The short version

In June 2026 NVIDIA released Nemotron 3.5 ASR, an open-weight speech-to-text model that transcribes 40 language-locales while you are still talking. It is small, at 600 million parameters, and free to use commercially. It is also easy to over-read: the headline numbers come from NVIDIA’s own tests, the 40 languages are not equally good, and Cantonese isn’t one of them. Here is what it actually is, how it differs from OpenAI’s Whisper, and which one fits which job.

• Built for live audio. It produces text in chunks as short as 80 milliseconds. Whisper works on 30-second windows.
• 40 locales in three tiers. 19 are ready to use, 13 are rougher, and 8 need fine-tuning first.
• No Cantonese. Whisper large-v3 added a Cantonese language token; Nemotron 3.5 doesn’t list it.
• Officially NVIDIA GPUs on Linux. Community ports for Apple silicon exist and look promising, but they aren’t NVIDIA’s.

What NVIDIA released

The model, published on Hugging Face on 4 June 2026, is a 600M-parameter FastConformer encoder with 24 layers and an RNN-T decoder. The encoder is “cache-aware”: instead of re-reading the whole recording every time new audio arrives, it keeps what it already worked out about earlier chunks and only processes the new slice. That is what makes it cheap enough to stream.

You pick the chunk size when you run it, without retraining: 80, 160, 320, 560 or 1,120 milliseconds. Smaller chunks mean text appears sooner; larger chunks give the model more context and fewer mistakes. You can tell it which language to expect or let it detect the language itself, in which case it tags each sentence, for example <de-DE>. Punctuation and capitalisation are built in.

The licence is NVIDIA’s OpenMDW-1.1, and NVIDIA describes the model as ready for commercial use. The weights are open; the full training data is not, so “open-weight” is more accurate than “open source”.

The 40 languages, honestly

NVIDIA splits the list into three tiers, and the tier matters more than the headline count.

• Transcription-ready (19): English (US, UK), Spanish (US, Spain), French (France, Canada), Portuguese (Brazil, Portugal), Italian, Dutch, German, Turkish, Russian, Arabic, Hindi, Japanese, Korean, Vietnamese and Ukrainian.
• Broad coverage (13): Polish, Swedish, Czech, Norwegian Bokmål, Danish, Bulgarian, Finnish, Croatian, Slovak, Mandarin, Hungarian, Romanian and Estonian.
• Adaptation-ready (8): Greek, Lithuanian, Latvian, Maltese, Slovenian, Hebrew, Thai and Norwegian Nynorsk. NVIDIA says these are not tuned for production and need fine-tuning on your own data.

Here are some of NVIDIA’s own scores on the FLEURS test set, with the language specified and the fastest 80 ms setting. Lower is better. Japanese, Korean and Mandarin are scored per character (CER), the others per word (WER), so compare within a row type, not across.

Averaged across the transcription-ready tier, word error rate falls from 10.38% at 80 ms to 8.84% at the slowest 1.12-second setting. The broad-coverage tier averages 25.86% at 80 ms, which is a lot of corrections for a long transcript. NVIDIA also notes that text normalisation isn’t perfect across all 40 locales, which can inflate the reported errors. Either way, these are the vendor’s numbers on clean read speech, not on meetings or phone calls.

The gap that matters for many readers in Hong Kong and the wider Cantonese-speaking world: Cantonese is not in any tier, and Mandarin sits in the weaker one. Whisper large-v3, by contrast, added a dedicated Cantonese language token and covers 99 languages, though how well it handles any one of them is a separate question you have to test.

Streaming is the real difference

Whisper reads audio in 30-second windows. For a recording you already have, that is fine: long files are cut into overlapping 30-second chunks and stitched back together. For live speech it is awkward, because the model was not designed to emit words as they arrive, and streaming setups built on Whisper rely on workarounds around those windows.

Nemotron 3.5 is built the other way round. It is meant for live captions, voice assistants and call transcription, where a second of delay is noticeable. The team at Daily, which builds voice-agent infrastructure, called it the lowest-latency speech-to-text model they had tested, and reported final transcripts staying under about 250 ms even under heavy load on a single NVIDIA L40S GPU.

If your use is transcribing finished recordings, that advantage shrinks. A voice memo doesn’t need words to appear while it plays. Accuracy, language coverage and where the model can run matter more.

Why “Nemotron beats Whisper” claims need checking

We looked for a clean, independent comparison between Nemotron 3.5 and Whisper and didn’t find one. A few things to watch for when you read the coverage:

• NVIDIA’s speed comparison isn’t with Whisper. Its throughput figures (240 streams on one H100 at 80 ms) are against its own older Parakeet model.
• Some benchmarks test a different model. One GPU benchmark that often comes up compares Whisper large-v3-turbo with NVIDIA’s earlier English-only streaming model, not with 3.5.
• Punctuation skews scores. That same benchmark notes NVIDIA’s models output punctuated text, which can add 2 to 3 points of word error rate against plain-text references. Scores across model families aren’t directly comparable.
• Some features in articles aren’t in the model card. We saw reports crediting the model with speaker identification and a head-to-head win over Whisper. The official model card claims neither, so we haven’t repeated those figures.

The honest test is your own audio: a few minutes of the kind of recording you actually make, run through both, and compared line by line.

Can it run on a Mac or iPhone?

Officially, no. NVIDIA lists its own GPU families, from Volta to Blackwell plus Jetson, on Linux. A GPU-backed server is the supported path, and NVIDIA’s packaged NIM container asks for at least 16 GB of video memory.

Unofficially, the community moved fast. One developer published a Core ML version with an 8-bit encoder that takes 612 MB on disk. On an M5 Pro Mac it transcribed 60 seconds of English in roughly four seconds (a real-time factor of 0.068), used about 1 to 1.2 GB of memory, and lost very little accuracy against the original in a 50-sample check. There are also MLX ports for Apple silicon. Treat these as experiments: they are maintained by individuals, they were measured on one high-end Mac, and we found no published iPhone measurements, where memory limits and heat are tighter.

On-device is the case we pay most attention to, because our own transcription app, OffgridScribe, runs Whisper on the iPhone through WhisperKit. The Core ML port is the first sign that Nemotron could become a realistic option there too, but it hasn’t been shown on a phone yet.

The other easy route is a hosted API. DeepInfra, Together and OpenRouter all offer the model. That is convenient, but it means your audio goes to that provider, which is a different privacy choice from running it yourself. Our guide on what transcription apps do with your recordings covers the questions worth asking.

Which should you use?

The last row comes from NVIDIA itself: for English-only work, its model card points to the separate English streaming model rather than the multilingual one. If you mainly need a transcript of something already recorded, our walkthrough on transcribing and labelling speakers on iPhone covers the practical side.

What we don’t know

• How it handles real recordings: accents, crosstalk, phone-quality audio, names and jargon. The published scores are on read speech.
• Whether it supports word timestamps or speaker labels. The model card doesn’t say it does.
• How the Apple ports perform on an iPhone, and how long community maintainers will keep them updated.
• Whether NVIDIA will add Cantonese or move Mandarin up a tier. Fine-tuning is possible, but it needs your own labelled audio.

Sources

• NVIDIA: Nemotron 3.5 ASR model card (Hugging Face)
• Daily: open-source multilingual transcription with Nemotron 3.5 ASR
• Wavect: a production review of Nemotron 3.5 ASR
• Community Core ML INT8 port, with M5 Pro measurements
• OpenAI: Whisper large-v3 model card
• E2E Networks: Whisper, Parakeet and Nemotron (English) on an L4

For disclosure: we make OffgridScribe, an on-device transcription app built on Whisper, so we have an interest in this comparison. We haven’t run Nemotron 3.5 ourselves; the figures here come from NVIDIA and the sources above. We use AI tools to help research and draft, and we check facts and sources before publishing.

More from Offgrid Studio

Otter.ai vs Google vs Rev: What Happens to Your Recordings?
Google’s Gemini Transcribe Is Impressive. Here’s the Catch.
How Much Memory Does an AI Model Need?