Fastest AI in 2026: Speed Benchmarks Across Top Models

Cerebras serves Llama 4 Scout at 2,600 output tokens per second, 19 times faster than the next-fast GPU cloud provider, according to Artificial Analysis benchmarks. That single number reveals something most "best AI" lists miss: the model you pick matters far less for speed than the hardware running it.

Most AI leaderboards rank models by intelligence scores. They treat speed as a footnote. But for developers building real-time chatbots, coding assistants, or agentic pipelines, inference speed determines whether the product feels instant or sluggish. A model that scores 5% higher on MMLU but runs 10 times slower is the wrong choice for a live customer-facing app.

Speed also breaks into two distinct metrics that serve different needs:

• Output tokens per second (tok/s): How fast the model generates text after it starts. This matters for long responses, code generation, and batch processing.
• Time to first token (TTFT): How long you wait before the first word appears. This matters for chatbots, interactive UIs, and any application where perceived responsiveness counts.

A model can be fast on one metric and slow on the other. Mercury 2 generates over 800 tokens per second but has a TTFT above 2 seconds. Groq's TTFT sits under 100ms but its throughput trails Cerebras. Picking the right model means knowing which metric your application actually needs.

This ranking uses independently verified benchmarks from Artificial Analysis, which measures each provider's API multiple times daily under consistent conditions. All figures reflect real API performance, not theoretical hardware limits.

1. Llama 4 Scout on Cerebras

Output speed: 2,600+ tok/s | TTFT: 0.33s

Cerebras runs Meta's Llama 4 Scout (17B active parameters, 109B total) on its wafer-scale engine, a single chip the size of a dinner plate that eliminates the memory bottleneck of GPU clusters. The result is throughput that no GPU provider has matched. Scout uses a mixture-of-experts architecture, activating only 17B parameters per token while drawing on a 109B knowledge base.

Best for: High-volume batch processing, agentic pipelines that chain multiple LLM calls, any workload where tokens per second directly reduces cost.

2. Mercury 2 by Inception Labs

Output speed: 800-1,000+ tok/s | TTFT: ~2.9s

Mercury 2 is a diffusion-based language model. Instead of generating tokens one at a time left to right, it refines multiple text blocks simultaneously. Inception Labs reports 1,009 tok/s on Nvidia Blackwell GPUs. Independent benchmarks from Artificial Analysis put it between 789 and 886 tok/s depending on the measurement window.

The tradeoff is latency: Mercury 2's TTFT averages 2.9 seconds, far higher than autoregressive models. But once generation starts, raw throughput is 5 to 10 times faster than comparably-sized models.

Best for: Batch text generation, document drafting, any workflow where total completion time matters more than perceived responsiveness.

3. Kimi K2.6 on Cerebras

Output speed: 981 tok/s | TTFT: sub-1s

Moonshot AI's Kimi K2.6 is a 1-trillion-parameter mixture-of-experts model. Cerebras serves it at 981 tokens per second, 6.7 times faster than the next-best GPU cloud provider. The model scores competitively with GPT-4o on reasoning benchmarks while running at nearly 10 times the speed on Cerebras hardware.

Best for: Large-context reasoning tasks where you need both quality and speed on open-weight infrastructure.

4. Groq (Llama and Gemma models)

Output speed: 800 tok/s (Llama 3 70B) | TTFT: under 80ms

Groq's custom Language Processing Unit (LPU) takes a different approach than Cerebras. Where Cerebras optimizes throughput, Groq optimizes latency. The median TTFT of 80ms versus 280ms on H100 GPUs creates a 3.5x perceived speed difference for users. Groq reports that switching customer support chatbots from H100 inference to Groq produced a 34% increase in user satisfaction scores, attributed entirely to faster response onset.

Best for: Interactive chatbots, customer-facing applications, any use case where the user is watching the cursor blink.

Store and query AI outputs from any model in one workspace

Fast.io gives agents and humans 50GB of free storage with built-in RAG search. Upload results from Cerebras, Groq, or any provider, then query them with Intelligence Mode. No credit card, no vector database setup.

Sign Up Free

The models below don't hold speed records, but they hit a practical sweet spot: fast enough for real-time applications, smart enough for production workloads.

5. Gemini 3.5 Flash (Google)

Output speed: 284 tok/s | TTFT: sub-0.3s

The fast model scoring above 70 on Artificial Analysis's quality index. Gemini 3.5 Flash runs nearly 2.5 times faster than GPT-4o while maintaining competitive quality. Google's Flash-Lite variant pushes even faster at 393 tok/s, though with quality tradeoffs.

Best for: Production applications that need both speed and intelligence without specialized hardware.

6. SambaNova (DeepSeek R1)

Output speed: 255 tok/s (DeepSeek R1 671B) | TTFT: sub-0.5s

SambaNova's custom RDU chips deliver the fast independently benchmarked inference for DeepSeek R1, a 671B-parameter reasoning model. Running a model this large at 255 tok/s is significant because most GPU providers serve it at under 50 tok/s. SambaNova also hits 580 tok/s on Llama 3.1 70B and over 600 tok/s on OpenAI's gpt-oss-120B.

Best for: Enterprises running large open-weight models who need GPU-class speed without GPU-class complexity.

7. GPT-4o (OpenAI)

Output speed: 117 tok/s | TTFT: sub-0.5s

Not the fast model on this list, but the fast model in OpenAI's lineup. GPT-4o runs about 2 times faster than GPT-4 Turbo and maintains a strong quality-to-speed ratio. For teams already integrated with OpenAI's API, the speed is adequate for most real-time applications.

Best for: Teams in the OpenAI ecosystem who need reliable, well-documented speed without switching providers.

8. Claude Opus Fast Mode (Anthropic)

Output speed: ~2.5x standard Opus throughput | TTFT: standard

Anthropic takes a different approach to speed. Instead of offering a separate "fast" model, Claude Opus 4.8 includes a fast mode that doubles output throughput without downgrading model quality. The speed improvement applies to output tokens per second only, not time-to-first-token. At $10/$50 per million tokens (versus $5/$25 standard), it costs twice as much per token but finishes tasks in roughly half the time.

Best for: Developers and agentic workflows where Opus-level reasoning matters but wall-clock time is the constraint.

Raw tokens per second is a misleading metric when taken alone. Here's a framework for matching speed characteristics to real workloads.

Chatbots and interactive UIs: TTFT dominates. Users perceive a response as "fast" based on when the first word appears, not when the last word finishes. Groq's sub-80ms TTFT feels instant. A model running at 2,000 tok/s with a 3-second TTFT feels slower to the user even though it finishes sooner on long responses.

Batch processing and document generation: Total throughput matters. If you're generating 500 product descriptions overnight, you want maximum tok/s regardless of TTFT. Cerebras or Mercury 2 will finish the job in a fraction of the time.

Agentic pipelines: Both metrics matter, but throughput usually wins. An agent that makes 20 LLM calls per task accumulates latency at each step. Cutting generation time by 10x on each call compounds into minutes saved per task. Teams running agents on Fast.io workspaces often pair high-throughput inference with persistent file storage so agents can read and write results between calls without re-downloading context.

Cost per token vs. cost per second: Faster inference can be cheaper overall. A model running at 1,000 tok/s on Cerebras uses the API for 1/10th the time of a model running at 100 tok/s. Even if the per-token price is higher, the reduced API time can lower total cost. Always calculate cost per completed task, not cost per token.

Speed without quality is just fast nonsense. The reason this ranking matters is that several models have cracked the speed barrier without sacrificing quality.

Llama 4 Scout on Cerebras scores within range of GPT-4o on standard benchmarks while running 20 times faster. Mercury 2 matches Claude 4.5 Haiku and GPT-5 Mini on quality while delivering 10 times the throughput. These are not toy models trading accuracy for speed.

The real question is whether your task needs frontier reasoning. For code generation, complex analysis, and multi-step reasoning, slower frontier models like Claude Opus or GPT-4o still outperform on accuracy. For summarization, translation, extraction, and classification, the speed-optimized models produce equivalent quality at a fraction of the latency.

One emerging pattern: developers route requests to different models based on complexity. Simple queries go to a fast model like Gemini Flash or Groq-hosted Llama. Complex queries go to Opus or GPT-4o. This routing layer, sometimes called a "model router," lets teams optimize both speed and quality without choosing one over the other.

For teams storing and managing the outputs of multiple AI models, a shared workspace like Fast.io gives both agents and humans access to the same files, version history, and Intelligence Mode search. Upload outputs from any model, then query them using built-in RAG without setting up a separate vector database. The free agent plan includes 50GB storage and 5,000 monthly credits with no credit card required.

Three trends are accelerating inference speed heading into the second half of 2026.

Diffusion-based language models are maturing. Mercury 2 proved that the diffusion approach works for text. Expect more labs to ship diffusion LLMs that generate multiple tokens simultaneously instead of one at a time. The TTFT penalty will shrink as the architecture matures.

Wafer-scale and custom silicon keep pulling ahead. Cerebras, Groq, and SambaNova have each built custom chips that bypass the memory bandwidth bottleneck of standard GPUs. As these platforms support more models, the speed gap between custom hardware and GPU clouds will widen further. NVIDIA's Blackwell B200 narrows the gap but doesn't close it.

"Fast mode" is becoming table stakes. Anthropic's fast mode for Claude Opus, Google's Flash model family, and OpenAI's speed-optimized variants all signal that speed is no longer a niche selling point. Every major provider now offers a speed-optimized tier alongside their flagship models.

For developers building on these models, the infrastructure question shifts from "which model" to "which provider runs my model fastest." Open-weight models like Llama 4 and DeepSeek can run on any hardware, so the choice of inference provider becomes the speed decision.