The recent achievement by Wafer in serving the **GLM5.2** model on **AMD Instinct MI355X** GPUs demonstrates a significant shift in the economics of AI inference. By optimizing the software stack, Wafer achieved an aggregate throughput of **2626 tokens/s/node** at a cost over **2x lower** than comparable NVIDIA Blackwell (B200) deployments. ### What Changed * **Quantization Optimization:** Wafer moved from the official FP8 quantization to **MXFP4** using AMD Quark, achieving lossless performance on benchmarks like GPQA-Diamond and GSM8K. * **Framework Selection & Tuning:** They utilized **sglang** as the inference engine, as it offered the best native support for their quantization path. * **Kernel-Level Fixes:** Wafer overcame "day-0" friction by manually patching the sglang ROCm stack. Key fixes included: * Correcting a quantization lookup mismatch for the MTP (Multi-Token Prediction) head, which enabled speculative decoding. * Adding missing ROCm guards to fused multi-step metadata kernels. * Manually tuning MoE (Mixture of Experts) kernel selection for GLM5.2's specific FP4 shapes, moving away from slow default heuristics. * **Deployment Strategy:** By switching from TP8 (Tensor Parallelism 8) to a **TP4×DP2** (Tensor Parallelism 4 × Data Parallelism 2) configuration, they significantly improved prefill-bound performance. ### Why It Matters * **Performance-per-Dollar:** The primary takeaway is that AMD hardware, when paired with specialized engineering, can provide a highly competitive alternative to NVIDIA’s Blackwell series. With AMD GPUs costing ~2.75x less, the ability to close the performance gap through software optimization makes them a viable, cost-effective solution for high-demand inference. * **Closing the Software Gap:** Historically, NVIDIA’s "day-0" software support has been a major barrier for AMD. Wafer’s work proves that as agentic tools and kernel optimization techniques mature, the "friction" of using AMD hardware is rapidly decreasing, allowing providers to bypass the premium pricing of NVIDIA ecosystems. ### Source * **Primary Source:** [Wafer Blog: Performance per dollar is getting faster and cheaper](https://www.wafer.ai/blog/glm52-amd) (July 3, 2026). This post provides the detailed technical breakdown of the quantization, kernel patches, and performance metrics.