GPUs and AI accelerators
The compute engines that run AI training and inference. GPUs (graphics processing units) dominate because their massively parallel design maps well onto the matrix math behind neural networks; dedicated accelerators (TPUs, custom ASICs) chase the same workloads with more specialized silicon.
Key facts
- NVIDIA holds the large majority of the AI training market, helped as much by its CUDA software ecosystem as by the chips themselves.
- Alternatives exist: AMD's Instinct GPUs, Google's TPUs, Amazon Trainium/Inferentia, Microsoft Maia, and startups like Cerebras and Groq.
- A modern AI accelerator is defined less by raw FLOPS than by memory bandwidth, interconnect, and how many can be linked into one coherent system.
- Each generation packs more transistors, faster memory, and tighter chip-to-chip links, which is why frontier training keeps scaling.
Where it bottlenecks
Demand far outstrips supply for the newest accelerators; allocation is gated by advanced packaging and memory rather than raw wafer output.
Who dominates it
Companies in this layer
Amazon
United StatesDesigner of Trainium and Inferentia accelerators and the largest cloud operator.
AMD
United StatesDesigner of Instinct data-center GPUs, the main merchant alternative to NVIDIA for AI compute.
Designer of TPU accelerators and a hyperscale AI data-center operator.
NVIDIA
United StatesDominant designer of AI training and inference GPUs, and of the CUDA software stack and NVLink/networking around them.