Which Whisper model on which iPhone, iPad or Mac
On the iPhone 17 Pro (iOS 26.0), Whisper tiny.en is the fastest Whisper model in these runs at 97.6x real time. Whisper large-v3-v20240930_626MB has the lowest error rate on LibriSpeech, 6.8%, and runs at 9x. On a Mac with Apple M4 (macOS 26.1), Whisper tiny.en reaches 108.5x real time, and Whisper large-v3 has the lowest LibriSpeech error rate, 6.2%, at 3.2x.
The tests ran between 2025-10-07 and 2025-10-17; the data ends there, so newer devices and newer models are not in it.
Pick a device
Tested on iOS 26.0, 2025-10-08.
| Model | Size (MB) | Speed (x real time) | Error rate, LibriSpeech (%) | Error rate, Earnings22 (%) |
|---|---|---|---|---|
| tiny | 73 | 93.7 | 19.6 | 17.1 |
| base | 139 | 60.6 | 16.5 | 12.4 |
| base.en | 139 | 63.3 | 11.7 | 11 |
| tiny.en | 145 | 97.6 | 16.5 | 16.4 |
| small | 463 | 20.6 | 9.9 | 10 |
| small.en | 464 | 21.6 | 9.9 | 8.1 |
| large-v3-v20240930_626MB | 626 | 9 | 6.8 | 8.6 |
| distil-large-v3 | 1,458 | 1.6 | 7.6 | 8 |
| large-v2 | 2,946 | 0.5Slower than real time | 8.5 | 6.8 |
| large-v3 | 2,947 | 0.6Slower than real time | 6.3 | 16.6 |
Fastest and most accurate per device
Among models that ran at least as fast as real time. Speed is how many seconds of audio are transcribed per second of processing.
| Device | OS | Fastest | Lowest error rate (LibriSpeech) |
|---|---|---|---|
| Apple M2Mac14,3 | macOS 26.1 | Whisper tiny.en · 89.3x | Whisper large-v3 · 6.2% · 2.6x |
| Apple M4Mac16,10 | macOS 26.1 | Whisper tiny.en · 108.5x | Whisper large-v3 · 6.2% · 3.2x |
| Apple M4 ProMac16,8 | macOS 26.0.1 | Whisper tiny.en · 112.9x | Whisper large-v3 · 6.2% · 3.6x |
| Apple M1Macmini9,1 | macOS 26.1 | Whisper tiny.en · 72.7x | Whisper large-v3-v20240930_626MB · 6.9% · 6.7x |
| iPad Air (5th generation)iPad13,16 | iPadOS 26.0 | Whisper tiny.en · 65.5x | Whisper small.en · 9.1% · 15.7x |
| iPad (10th generation)iPad13,18 | iPadOS 18.6.2 | Whisper tiny.en · 49.8x | Whisper small · 9.7% · 9.8x |
| iPad Air 11-inch (M3)iPad15,3 | iPadOS 26.0 | Whisper tiny.en · 96.8x | Whisper small.en · 9.1% · 19x |
| iPad mini (A17 Pro)iPad16,1 | iPadOS 18.7 | Whisper tiny.en · 83.4x | Whisper small · 9.7% · 15.4x |
| iPad Pro 11-inch (M4)iPad16,3 | iPadOS 26.0 | Whisper tiny.en · 105.6x | Whisper distil-large-v3 · 7.7% · 2.7x |
| iPhone 12 miniiPhone13,1 | iOS 18.7 | Whisper tiny.en · 50.2x | Whisper small · 9.7% · 5.3x |
| iPhone 13iPhone14,5 | iOS 18.7 | Whisper tiny.en · 63.2x | Whisper large-v3-v20240930_626MB · 6.9% · 4x |
| iPhone 14iPhone14,7 | iOS 18.7 | Whisper tiny.en · 62.4x | Whisper large-v3-v20240930_626MB · 6.9% · 4.5x |
| iPhone 15iPhone15,4 | iOS 26.0 | Whisper tiny.en · 74x | Whisper large-v3-v20240930_626MB · 6.8% · 3.5x |
| iPhone 16 ProiPhone17,1 | iOS 26.1 | Whisper tiny.en · 95.4x | Whisper large-v3-v20240930_626MB · 6.8% · 6.9x |
| iPhone 16iPhone17,3 | iOS 26.0.1 | Whisper tiny.en · 93.2x | Whisper large-v3-v20240930_626MB · 6.8% · 7.2x |
| iPhone 16eiPhone17,5 | iOS 18.6.2 | Whisper tiny.en · 90.9x | Whisper large-v3-v20240930_626MB · 6.9% · 5.3x |
| iPhone 17 ProiPhone18,1 | iOS 26.0 | Whisper tiny.en · 97.6x | Whisper large-v3-v20240930_626MB · 6.8% · 9x |
| iPhone 17iPhone18,3 | iOS 26.0 | Whisper tiny.en · 92.7x | Whisper large-v3-v20240930_626MB · 6.8% · 8x |
What the numbers mean
- Speed is the seconds of audio divided by the seconds the whole WhisperKit pipeline took (the dataset's timings fields inputAudioSeconds and fullPipeline), summed over every clip. 10x means ten seconds of audio per second.
- Error rate is the dataset's wer field, the Word Error Rate of each transcription against its reference text, averaged over the clips. LibriSpeech is clean read speech; Earnings22 is recorded company earnings calls, which are harder. Lower is better.
- Argmax's own dashboard computes the error rate over the whole set after normalizing the text, so its numbers differ from the ones here. Compare models with each other on this page, not with their dashboard.
- Size is the dataset's modelSizeMB field, as recorded in each test file. Model names are as the dataset writes them; a name ending in 626MB is a compressed variant.
- Each dataset is about ten minutes of audio. LibriSpeech is made of 101 short clips; Earnings22 is a single recording, so its error rate is one sample and can swing widely between models. One test run per device and model is not a statistical sample.
Source
Data: Argmax, WhisperKit evals dataset (MIT license). Device names come from the dashboard that Argmax publishes with the dataset. We reduced the files to one row per device and model; we ran none of these tests ourselves.
WhisperKit evals dataset on Hugging Face