Archive - SemiAnalysis

Google Gemini Eats The World – Gemini Smashes GPT-4 By 5X, The GPU-Poors

Compute Resources That Make Everyone Look GPU-Poor

Aug 28, 2023 •

and

Daniel Nishball

How Nvidia’s CUDA Monopoly In Machine Learning Is Breaking - OpenAI Triton And PyTorch 2.0

Over the last decade, the landscape of machine learning software development has undergone significant changes. Many frameworks have come and gone, but…

Jan 16, 2023 •

GPT-4 Architecture, Infrastructure, Training Dataset, Costs, Vision, MoE

Demystifying GPT-4: The engineering tradeoffs that led OpenAI to their architecture.

Jul 10, 2023 •

and

Nvidia's Blackwell Reworked - Shipment Delays & GB200A Reworked Platforms

MGX GB200A NVL36, B102, B20, CoWoS-L, CoWoS-S, GB200A NVL64, ConnectX-8, Liquid Cooling vs Air Cooling, NVLink Backplane, PCB, CCL, Substrate, BMC…

Aug 4, 2024 •

,

,

Daniel Nishball

,

, and

Multi-Datacenter Training: OpenAI's Ambitious Plan To Beat Google's Infrastructure

Gigawatt Clusters, Telecom Networking, Long Haul Fiber, Hierarchical & Asynchronous SGD, Distributed Infrastructure WinnersGigawatt Clusters, Telecom…

Sep 4, 2024 •

,

Daniel Nishball

, and

Jeremie Eliahou Ontiveros

100,000 H100 Clusters: Power, Network Topology, Ethernet vs InfiniBand, Reliability, Failures, Checkpointing

Frontier Model Scaling Challenges and Requirements, Fault Recovery through Memory Reconstruction, Rack Layouts

Jun 17, 2024 •

and

Daniel Nishball

AI Datacenter Energy Dilemma - Race for AI Datacenter Space

Gigawatt Dreams and Matroyshka Brains Limited By Datacenters Not Chips

Mar 13, 2024 •

,

Daniel Nishball

, and

Jeremie Eliahou Ontiveros

GB200 Hardware Architecture - Component Supply Chain & BOM

Hyperscale customization, NVLink Backplane, NVL36, NVL72, NVL576, PCIe Retimers, Switches, Optics, DSP, PCB, InfiniBand/Ethernet, Substrate, CCL, CDU…

Jul 17, 2024 •

,

,

,

,

Jeremie Eliahou Ontiveros

, and

Daniel Nishball

Datacenter Anatomy Part 1: Electrical Systems

Meta Datacenter Scrapped, Vertiv, Schneider Electric, Eaton, Datacenter Bill Of Materials By Component, Transformers, Switchgear, Redundancy, UPS, ATS…

Oct 14, 2024 •

,

Jeremie Eliahou Ontiveros

, and

Daniel Nishball

The Memory Wall: Past, Present, and Future of DRAM

Winners & Losers in the 3D DRAM Revolution

Sep 3, 2024 •

,

,

,

, and

AI Neocloud Playbook and Anatomy

H100 Rental Price Cuts, AI Neocloud Giants and Emerging Neoclouds, H100 Cluster Bill of Materials and Cluster Deployment, Day to Day Operations, Cost…

Oct 3, 2024 •

and

Daniel Nishball

Nvidia’s Plans To Crush Competition – B100, “X100”, H200, 224G SerDes, OCS, CPO, PCIe 7.0, HBM3E

Roadmap, Supply, Anti-competitive: AMD, Broadcom, Google, Amazon, and Microsoft Have Their Work Cutout For Them

Oct 10, 2023 •

and

#nojs-banner { position: fixed; bottom: 0; left: 0; padding: 16px 16px 16px 32px; width: 100%; box-sizing: border-box; background: red; color: white; font-family: -apple-system, "Segoe UI", Roboto, Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol"; font-size: 13px; line-height: 13px; } #nojs-banner a { color: inherit; text-decoration: underline; } This site requires JavaScript to run correctly. Please turn on JavaScript or unblock scripts