Microsoft Scales Up Its Own AI Silicon
Microsoft has announced a major expansion of its in-house Azure Maia AI inference systems. By 2027, the company expects to reach a striking multi-gigawatt deployment scale.
The move makes Maia the main inference engine for future Microsoft products, including Copilot. It also signals a deliberate plan to reduce heavy reliance on outside GPUs, especially those from NVIDIA.
Maia 200 Is Live, With 1.7x Better Efficiency
Microsoft confirmed that its Azure Maia 200 system now runs in data centers across the United States, Europe, and Australia.
Maia 200 already carries real inference workloads for Microsoft Teams and Copilot. It supports OpenAI’s flagship models, Microsoft’s own MAI models, and popular open-source models.
The Power of Co-Design
Its energy efficiency stands out most. Microsoft co-designed the hardware around the needs of its vast software services. As a result, Maia 200 delivers 1.7 times the tokens per watt of leading GPU infrastructure.
Before the end of this year, GitHub Copilot and the Microsoft Foundry model platform will also move to this in-house architecture.
Maia 300: Built for the Agentic AI Era
If Maia 200 was a first test, the Microsoft Maia 300 is the true giant. It is due in 2027 and built for the age of agentic AI.
AI is shifting from simple chatbots to agents that handle long-context reasoning, multi-step planning, and code execution. Consequently, the bottleneck has moved from raw compute to memory bandwidth and data movement.
Key Design Choices
To meet that challenge, Maia 300 makes several bold design choices:
- TSMC N2P and chiplets: Instead of one giant die, Maia 300 uses multiple chiplets on TSMC’s advanced N2P process. It splits compute, memory controllers, and I/O to improve cost and scalability.
- Extreme memory bandwidth: At a 950W TDP, it carries eight stacks of the latest HBM4 memory for up to 18.0 TB/s of bandwidth. Software-managed SRAM scratchpads ease the heavy KV-cache load of large models.
- Inference-first dataflow: It supports the advanced low-precision MXFP4 format. A software-defined dataflow design keeps data local on the chip and cuts needless data movement. Microsoft estimates 40% more tokens per megawatt than leading GPUs.
- Dense, power-hungry racks: Each rack holds 96 accelerators with 1.7 PB/s of total HBM bandwidth. Rack power reaches a formidable 167 kilowatts, and rack-level token output climbs to seven times that of Maia 200.
Buying From NVIDIA While Building a Way Out
The timing is delicate. NVIDIA had just promoted its open Vera Rubin AI factory design at the OCP Summit, with racks of up to 250 kW. It also announced a partnership with Microsoft there.
Soon after, Microsoft unveiled its own 167 kW Maia 300 rack built for inference. The contrast shows that the fight over data center space and power has grown fierce. It also shows that Microsoft still needs NVIDIA GPUs to drive AI forward, yet wants its own accelerators to lessen that dependence.
Why Inference Is the Real Prize
In the generative AI supply chain, model training remains NVIDIA’s home turf. However, Microsoft knows that the largest recurring costs will come from billions of daily Copilot inference requests.
If every inference service keeps paying a toll to NVIDIA for costly GPUs, AI profit margins can never reach their peak.
A Two-Track Strategy
Microsoft’s plan is shrewd. It will keep spending heavily on NVIDIA GPUs to train frontier OpenAI models. At the same time, it will build Maia 300 on TSMC N2P and move all internal inference work onto its own hardware.
Through this tightly integrated, heterogeneous platform strategy, Microsoft can sharply cut Copilot’s operating costs. It also gains strong momentum ahead of Microsoft Ignite 2026 next month.
Support Our Threat Intelligence
If you find our technology report and cybersecurity news helpful, consider supporting our work.
Buy Me a Coffee
PayPal
