GPU Clusters & Compute

The story, event by event

Every point below is traced to a real source — nothing on this page is invented.

1. 23 Jun 2007impact 72

   CUDA — GPUs become programmable for compute

   CUDA turned NVIDIA's GPUs from fixed-function graphics chips into general-purpose parallel computers.

   CUDA 1.0 exposed the GPU's hundreds of parallel cores through a C-like programming model, letting developers write general numerical code instead of disguising it as graphics operations.

   Wikipedia

   By making massively parallel hardware programmable outside of graphics pipelines, CUDA created the substrate every subsequent deep-learning accelerator workload — from AlexNet to frontier training clusters — would be built on.

   WikipediaNVIDIA

   It made the GPU a general compute device — the foundational primitive under the entire deep-learning compute branch.

2. 30 Sept 2012impact 74

   GPU training becomes the default

   AlexNet proved that GPUs, not CPUs, were the right substrate for training deep neural networks.

   AlexNet trained a deep convolutional network on two GTX 580 GPUs (3GB each) for about six days, using custom CUDA kernels to make the workload fit consumer hardware.

   NeurIPS

   The result beat the next-best ImageNet entry by a wide margin, and the field read the lesson as being about hardware as much as architecture — accelerators, not CPU clusters, were now the default substrate for training.

   NeurIPS

   It began the compute arms race — every deep-learning advance since has been paced by accelerator supply.

3. 1 Jan 2023impact 76

   Frontier runs hit datacenter scale

   Training a frontier model became a datacenter-scale capital project rather than a lab experiment.

   GPT-3's 175B-parameter run was reported as trained on a Microsoft-built supercomputer described at launch as comprising 10,000 GPUs — a scale of infrastructure investment far beyond a university lab.

   arXivMicrosoft

   Once a single training run required tens of thousands of accelerators plus the datacenters, power, and networking to feed them, frontier-scale training concentrated in the handful of organizations that could finance clusters at that scale.

   arXiv

   Compute at this scale became a gatekeeper — frontier capability concentrated in a few well-capitalized players.

4. 18 Mar 2024impact 70

   Hopper → Blackwell datacenter GPUs

   NVIDIA's Hopper and Blackwell generations became the default silicon for frontier AI training.

   H100, built on the Hopper architecture, was announced at GTC in March 2022 and became the standard GPU for large-scale model training across the industry.

   NVIDIA

   NVIDIA followed with Blackwell — the B200 and dual-die GB200 — announced at GTC in March 2024 as the next-generation datacenter GPU, extending the same architectural lineage into the newest frontier training clusters.

   NVIDIA

   It cemented NVIDIA's effective monopoly over the silicon that frontier training depends on.

5. 1 Apr 2026impact 65

   Mistral AI invests €4B in European AI cloud and data centers

   Mistral AI is making substantial investments in its own AI cloud infrastructure and data centers across Europe.

   Mistral acquired infrastructure startup Koyeb to build a "true AI cloud."

   The company announced a €4 billion investment strategy to build data centers in France and Sweden.

   Mistral plans to launch Mistral Compute, a European AI platform powered by Nvidia, in 2026.

   This move signifies a trend among major AI developers to vertically integrate and control their compute infrastructure, reducing reliance on external cloud providers and ensuring sovereign AI capabilities.

6. 29 Apr 2026impact 60

   Stargate Initiative Progress

   time frame: 90 days; added infrastructure: 3GW

7. 6 Jul 2026impact 60

   SK Hynix IPO Announcement

   ipo amount: 28000000000; ipo shares: 17800000; stock increase percentage: 260; year over year revenue increase percentage: 200

Lineage