Intel-backed AI chip startup SambaNova breathes new life into aging Nvidia GPUs in… is attracting attention across the tech world. Analysts, enthusiasts, and industry observers are watching closely to see how this story develops.
This update adds another signal to a fast-moving sector where product decisions, platform changes, and competition can quickly shape the market.
Third-party testing shows heterogeneous compute platform combining H200s and SN50 RDUs churning out 763 tok/s in MiniMax M2.7
Intel's big bet on SambaNova appears to be paying off in a big way. This week, the AI chip startup shared benchmark results showing its latest generation of AI acceleration, which combines Nvidia GPUs and the company's accelerators, beating GPU-only inference platforms by a wide margin.
The testing, conducted by the AI benchmarking gurus at Artificial Analysis, showed SambaNova's SN50-series accelerators, announced in February, churning out 763 tokens a second in MiniMax M2.7 at short context lengths (10,000 input tokens) — several times faster than competing inference providers running on GPUs alone.
Meanwhile, for longer context lengths, the company says that its platform is able to sustain more than 450 tokens a second.
This feat was accomplished by combining Nvidia GPUs with SambaNova Reconfigurable Dataflow Units (RDUs) to form a heterogeneous inference platform.
Specifically, the computationally intensive prefill phase of the inference pipeline, during which prompts are processed and key value caches are generated, was handled by four Nvidia H200 GPUs.
Meanwhile, memory-bandwidth-bound decode operations, where output tokens are generated, were done on a single SambaNova rack containing 16 SN50 accelerators.
Disaggregating prefill from decode has become a key lever for reducing token costs for long-running AI agents, like code assistants. Nvidia initially demonstrated this with its NVL72 rack platforms, by varying the ratio of GPUs used for prefill versus decode. The company further disaggregated this with its Groq-based LPX racks revealed at GTC this spring. Since then, just about everyone from AMD to AWS and Cerebras has announced some kind of disaggregated or heterogeneous inference platform using one or more accelerators.
With SambaNova's latest performance figures, the startup hopes to demonstrate how customers can breathe new life into their aging GPU fleets by using its platforms as decode accelerators. And because its platforms are air-cooled, they can be deployed in existing datacenters — something that can't be said of Nvidia's latest generation of Rubin GPUs, which absolutely need liquid cooling.
SambaNova plans to show off even more powerful inference configs, with 128 and eventually 256 accelerators to demonstrate its ability to maintain high token generation rates at high throughput. As we’ve previously explored, this is something that GPUs alone have historically struggled with and one of the key drivers behind Nvidia’s Groq acquihire late last year.
The results come just a month after SambaNova and Intel announced Vector Core Compute would be among the first to deploy the combined GPU + RDU offering with TogetherAI as their first large-scale customer.
Ramping production of any chip isn’t a cheap prospect, but for its fifth-gen part, capital shouldn’t be an issue. On Wednesday, SambaNova completed the first close of a $1 billion Series F funding round led by General Atlantic, giving the AI chip startup an $11 billion valuation. ®
Why This Matters
This development may influence user expectations, future product strategy, and the competitive balance inside the broader technology industry.
Companies in adjacent segments often react quickly to similar moves, which is why stories like this tend to matter beyond a single announcement.
Looking Ahead
The full impact will become clearer over time, but the story already highlights how quickly the modern tech landscape can evolve.
Observers will continue tracking the next steps and how they affect products, users, and the wider market.
