Etched AI Review 2026: Sohu Chip Specs, Benchmarks, and What Actually Ships

Inference now accounts for roughly 60% of total AI compute spending, up from under 40% in 2023, according to Epoch AI's 2026 compute trends report. That shift is the reason Etched exists. Training a frontier model is a one-time cost spread across months. Serving that model to millions of users is an ongoing expense that scales with every API call, every chatbot session, every agent loop. If you can cut the cost-per-token by an order of magnitude, the economics of the entire AI stack change.

Etched AI, founded in 2022 by Harvard dropouts Gavin Uberti (CEO) and Chris Zhu (CTO), is betting that specialization wins this game. Their Sohu chip is an application-specific integrated circuit (ASIC) designed to do exactly one thing: run transformer inference. No training. No convolutions. No diffusion. Just the forward pass of attention-based models, etched directly into silicon.

The company has raised close to $1 billion in total funding, including a $500 million round in January 2026 led by Stripes that valued the company at $5 billion. Notable backers include Peter Thiel, Ribbit Capital, and Positive Sum. GitHub CEO Thomas Dohmke and former Coinbase executive Balaji Srinivasan are among the angel investors.

That war chest buys credibility. But credibility isn't the same as shipping silicon. The question for anyone evaluating Etched in 2026 is whether the performance claims hold up, and whether the architectural bet on pure transformers is the right one given how fast the model landscape is moving.

Helpful references: Fast.io Workspaces, Fast.io Collaboration, and Fast.io AI.

Sohu is fabricated on TSMC's 4nm process at reticle-limit die size, roughly 800mm squared. It pairs the compute die with 144 GB of HBM3E memory, delivering approximately 4,800 GB/s of memory bandwidth. That's roughly 1.8x the bandwidth of an NVIDIA H100.

The architectural distinction is in the compute fabric itself. Where a GPU executes transformer operations as a sequence of general-purpose matrix multiplications, Sohu implements each stage of the transformer forward pass as fixed-function hardware. The control path is simpler, which means the same transistor budget can host more arithmetic units doing actual work.

Etched claims this design achieves over 90% floating-point utilization on transformer workloads. For comparison, GPUs typically achieve 30% to 40% FLOP utilization on the same tasks. The gap isn't because GPUs are poorly designed. It's because they're designed for everything: training, inference, vision models, diffusion, scientific computing. That generality comes with overhead. Sohu trades it away.

Key specs at a glance:

• Process: TSMC 4nm
• Memory: 144 GB HBM3E
• Bandwidth: ~4,800 GB/s
• FLOP utilization: 90%+ (claimed)
• Architecture: fixed-function transformer pipeline
• Training support: none (inference only)
• MoE support: none
• Power consumption: not disclosed

Etched's headline number: an 8-chip Sohu server generates over 500,000 tokens per second on Llama 70B. An 8-GPU H100 system produces roughly 23,000 tokens per second on the same model. That's a 20x gap, or equivalently, one 8-chip Sohu server replaces about 160 H100 GPUs.

Against NVIDIA's newer Blackwell architecture, which delivers roughly 4x the H100's inference throughput, Sohu still claims a 5x to 10x advantage on dense transformer models.

These are extraordinary claims. And as of June 2026, none of them have been independently verified.

No third-party benchmarks exist. Etched has not published batch-32 or batch-256 throughput figures, which matter for real production workloads where you're serving many concurrent users. The 500,000 tok/s number comes entirely from Etched's own marketing materials and demo videos. The company has discussed putting early reference units in the hands of select partners, but hasn't named those partners or published any co-authored benchmark results.

This doesn't mean the claims are false. Fixed-function ASICs genuinely can outperform general-purpose processors on their target workload by wide margins. Bitcoin mining ASICs proved this years ago. But the magnitude of Etched's claims, combined with the total absence of independent validation, means buyers should treat these numbers as projections rather than proven performance.

Store and serve your AI model outputs in one workspace

Fast.io gives AI teams a shared workspace with built-in semantic search, versioning, and an MCP server for agent access. Start a 14-day free trial.

Start 14-Day Trial

Sohu's fixed-function design is both its strength and its most serious vulnerability. The chip can only run dense transformer models with standard attention patterns. It cannot run:

• Mixture-of-Experts (MoE) models with dynamic expert routing
• Vision or multimodal models
• Diffusion models
• State-space models (SSMs) like Mamba
• Any architecture outside the attention pattern

In 2024, when Etched announced Sohu, this seemed like a reasonable tradeoff. Dense transformers dominated the landscape. By mid-2026, the picture has changed.

DeepSeek V4, a 671B MoE model, is the most downloaded model on Hugging Face. Qwen3-235B-A22B, another MoE architecture, represents the frontier of open-weight capabilities. Both are incompatible with Sohu. The MoE approach activates only a fraction of total parameters per token, which requires irregular memory access patterns that fixed-function transformer circuits cannot accommodate.

This isn't a future risk. It's the current state of the open-weight ecosystem. If the trend toward MoE continues, and every major lab except Anthropic has shipped MoE models in 2025-2026, then Sohu's addressable market narrows with each new model release.

Etched's counter-argument is that dense transformers still power most deployed commercial models, including the GPT-4 family and Claude. That's true today. But an ASIC is a multi-year investment. The question isn't whether dense transformers matter now, it's whether they'll still dominate inference workloads in 2028 and beyond.

Etched isn't the only company challenging NVIDIA's dominance in inference. Here's how the major players compare:

NVIDIA (H100 / Blackwell)

NVIDIA's strength is universality. H100 and Blackwell GPUs run every model architecture, every framework, every workload. The software ecosystem (CUDA, TensorRT, Triton) is decades deep. Blackwell delivers roughly 4x H100 performance on large LLM inference. The tradeoff is cost: NVIDIA hardware commands premium pricing, and utilization rates on general-purpose silicon are inherently lower than what a specialized ASIC can achieve.

Groq LPU

Groq's Language Processing Unit takes a different path: a dataflow processor with 500 MB of on-chip SRAM per chip, achieving 150 TB/s bandwidth. The LPU delivers roughly 275 to 284 tokens per second on Llama 70B with consistent latency regardless of context length. In December 2025, NVIDIA struck a $20 billion licensing deal that brought Groq's technology and leadership into NVIDIA's portfolio, though Groq continues to operate GroqCloud independently. That deal effectively validated the inference-specific hardware thesis while consolidating it under NVIDIA's umbrella.

Cerebras WSE

Cerebras went the opposite direction from Etched: instead of a small specialized chip, they built the largest chip ever made. The Wafer-Scale Engine is a single silicon die the size of an entire wafer. Cerebras went public in May 2026, closing its first day at a roughly $56 billion fully diluted valuation, the largest U.S. tech IPO since Snowflake. Unlike Sohu, Cerebras supports training and a broader range of model architectures.

Where Sohu fits

If Etched's claims hold up, Sohu would be the performance leader for dense transformer inference by a wide margin. But it's also the least flexible option, the only one without shipping customers, and the only one that can't run MoE models. The risk-reward profile is asymmetric: extraordinary upside if transformers stay dominant and Sohu ships on time, but limited fallback if either assumption breaks.

As of June 2026, Sohu has not shipped to external customers. The original timeline targeted late 2024 to early 2025 for initial shipments. That window passed without public delivery. A Manifold prediction market asking "Will the Sohu AI chip ship to customers within a year?" resolved "no" in July 2025.

Etched has stated it is prioritizing volume readiness over early demonstrations, citing Groq's slow-and-steady deployment approach as a precedent. The company has discussed reservation commitments with major AI labs and cloud providers but hasn't named partners or announced deployment dates.

For anyone evaluating Etched for production use, here's what to watch:

• Independent benchmarks: The first third-party benchmark on real Sohu hardware will be the single most important data point. Until it exists, all performance numbers are vendor claims.
• Batch throughput figures: Production inference serves many concurrent requests. Single-stream tok/s is a marketing metric. Batch-32 and batch-256 numbers at realistic context lengths tell you whether Sohu works in a real deployment.
• Named customers: When a cloud provider or AI lab publicly deploys Sohu, the risk profile changes dramatically. Until then, the chip exists only in controlled demos.
• MoE roadmap: Etched hasn't publicly addressed whether future chip revisions could support MoE architectures. Any signal here changes the long-term thesis.
• NVIDIA's response: Blackwell is already 4x faster than H100. The next generation, Rubin, is on the roadmap. NVIDIA's pace of improvement narrows the window for ASIC challengers.

The delay isn't unusual for a semiconductor startup. Chip development timelines regularly slip, and first-generation products from new companies almost always face yield and validation hurdles. But each quarter without shipping customers is a quarter where NVIDIA gets faster and MoE models get more popular.