DeepSeek V4 Flash Runs on a Single AMD MI300X GPU

A developer runs the 304B-parameter DeepSeek V4 Flash frontier model on one AMD MI300X accelerator at 168 tokens per second, no quantization required.

Developer Ryan Zhou has published a working configuration that runs DeepSeek's V4 Flash — a 304-billion-parameter frontier language model — on a single AMD MI300X accelerator, achieving 168.6 tokens per second in single-stream decode and validated at 256K context length.

The GitHub repository includes a Docker Compose stack with pinned versions, kernel patches, and tuning tables. The entire 156.67 GiB model fits in the GPU's 192 GB of HBM3 memory without weight quantization or offloading — something impossible on an 80 GB NVIDIA H100.

The MI300X has 2.4× the HBM capacity of an H100 at roughly half the list price, but running modern AI workloads on AMD hardware has been notoriously difficult due to software immaturity. Zhou's work builds on Fergus Finn's detailed worklog at Doubleword, which catalogued the sharp edges: AMD's fnuz FP8 variant (incompatible with OCP-standard kernels), MoE routing bugs at high concurrency, and missing AITER fast-path kernels for the gfx942 architecture.

The repository ships patches for each of these issues, validated in production. At 8 concurrent streams, aggregate throughput reaches 542 tok/s; burst tests at 64 streams produce 830 tok/s with no out-of-memory errors. A hybrid KV cache strategy pairs 20 GB of GPU cache with 96 GiB of CPU offload for evicted prefix entries.

Commenters on Hacker News, where the story reached the front page with 362 points, noted that the MI300X is an OAM module — the upcoming MI350P will offer PCIe form factor with 144 GB, still enough for DeepSeek V4 Flash's native MXFP4 quantization. Several flagged that throughput remains below DeepSeek's own H800 benchmarks (15K tok/s/GPU), suggesting further optimizations are possible.

What makes this significant: it opens the door to running frontier-scale AI models on non-NVIDIA hardware at substantially lower cost. One card, no quantization tricks, production-ready.