The One Thing You Need to Know About Data this Week: Giant Bets on Tiny Circuits

OpenAI and GPUs

The one thing you need to know about data this week is why Sam Altman and OpenAI are proposing to construct $7 trillion worth of new manufacturing assets focused on making GPUs. 

What GPUs are and why do they seem so scarce? And why would anyone invest more than the GDP of India and Japan put together, to make them?

1. Because GPUs do math fast for AI

2. Because GPU production is controlled by two companies, ASML and TSMC

3. Because “vertical integration” can be clever

Let’s begin.

1. The GPU

GPUs are graphics processing units. They are built to do massive amounts of math. That’s useful when you’re processing a terabyte of data to train a large language model, or LLM.

GPUs have been around since the video game company Atari created the ANTIC, or the Alphanumeric Television Interface Controller to make the joystick game Asteroids faster and funner.

The challenge, back in 1979, was mixing video games and math. If you were going to move a spaceship object (gaming insiders called them “sprites”) around on a screen… and maybe collide them with other sprites, like asteroids… and the asteroids could collide with other asteroids… well that was a lot of coordinates to calculate.

Atari invented ANTIC to do all that math, so your spaceship could be smashed with pinpoint accuracy when the coordinates of ship x asteroid collided.

If you look at the structure of a GPU compared to the more familiar CPU, or central processing unit, it’s pretty different.

The CPU has a compact design. It makes decisions for the computer quickly. It sits in the center of the computer doing a variety of tasks swiftly. Perfect for a PC.

The GPU has more capacity to do math. It is heavied up with Arithmetic Logic Units (ALUs)… units that do calculations… distributed across multiple units… hundreds… so that many, many calculations can be performed simultaneously. Perfect for a video game.

And beyond the guts of a single computer, there is an overall transition afoot. The installed base of CPU-centric general-purpose computing is giving way to accelerated computing centered on GPUs. (We’ve seen estimates that CPUs are 85% of the current compute power installed base). A massive market for Nvidia and other GPU providers… including now OpenAI?… to rip out CPUs and replace them with GPUs—which are more productive and energy-efficient, to boot.

2.The two companies that control GPU production

The trick is… and here we get to the second key point… that GPUs have such a crazy supply chain that Sam Altman thinks he needs to make his own. 

In his book Chip War, Chris Miller describes exactly how ridiculously hard it is to make a silicon chip in the 21st century. By his telling, it all comes down to essentially two companies.

The first company is the one that produces machines for stamping circuits on chips.

A silicon chip is basically a tiny city of flowing circuits. So tiny that the “wiring pathways” are as small as a few nanometers. What’s a nanometer? A billionth of a meter, or 0.000000001 meters.

And when you’re etching a 13.5-nanometer pathway onto a tiny wafer of silicon, it’s so small your etching tool is basically… light. Like, photons. Like…. a single photon. Using a technique called EUV, or extreme ultraviolet lithography.

As you can imagine, the equipment to accomplish this task (which was once, by the way, considered impossible) is highly, highly specialized: “A machine,” Miller tells us, “with hundreds of thousands of components” and which will usually “cost over $100 million each.” 

And these EUV lithography machines are all made by a single company, ASML, or Advanced Semiconductor Materials Lithography. Based, of all places, in Veldhoven, the Netherlands. A tidy green suburb an hour from Rotterdam, with very little to do on a Saturday night.

Except, I guess, sit around and count your $370 billion valuation.

ASML then sells its EUV lithography machines to a little outfit in Taiwan called TSMC, or the Taiwan Semiconductor Manufacturing Company. 

It’s a long story--obviously I recommend Miller’s book--but he claims that now “TSMC builds almost all the world’s most advanced processor chips.” CNN puts their market share at 90%. For one product, for one client, in one fabrication plant alone--according to Miller--TSMC produced 1,000,000,000,000,000,000 transistors.

Now-legendary AI / GPU player Nvidia gets all of its chips from TSMC. And this is the strength of its position: Miller again: “TSMC could pull ahead of rivals technologically because it was a neutral player around which other companies would design their products.” 

In other words, Nvidia designs the chips they want. But TSMC makes them.

This includes Sam Altman and his proposed $7 trillion chip project. In Altman’s vision, chip “fabs” would be established all over the world… with his investments… but created and operated…

…by TSMC.

Would Altman own this new infrastructure, if TSMC is still running it? If he doesn’t—if he controls but does not own—will it change the duopoly dynamics?

3. Vertical integration

Terms like “vertical integration” are incredibly dull and take me back to the weird thrill of business school and learning to talk like an MBA.

For the uninitiated, vertical integration refers to the process of a company acquiring assets “down” in its supply chain--buying its suppliers, in other words. This gives you control over your materials.

Altman’s seven trillion dollars would allow OpenAI to vertically integrate into the chips that power its models. 

There are some examples of vertical integration--like AT&T (a distributor of entertainment through broadband) and Warner Brothers (a creator of entertainment), since uncoupled--that don’t make the idea seem too appealing. 

But others--especially in aerospace--seem visionary. 

Because when your suppliers can’t make what you need… well then, you need to make your own inputs. 

One great example is the satellite data company Planet Labs. 

Planet has launched its own fleet of satellites to take pictures of Earth which they sell as data. In the process they have completely revolutionized satellite tech with a “flock” of “Dove” satellites that flutter around the planet by the dozen, on little plastic wings that double as solar panels.

• Old tech: the satellite is the size of a Chevy Silverado

• Planet tech: the satellite is the size of a clarinet case

• Old tech: the satellite circles the Earth every 16 days

• Planet tech: the satellite circles the Earth every day

• Old tech: the satellite detects images 100 feet wide

• Planet tech: the satellite detects images 10 feet wide

And when Planet writes about how it supplies its innovations, it boils it down to a single, damning sentence:

“It is unfortunate, but the satellites developed by Planet Labs contain no component directly sourced from the space industry.”

Ouch. In other words: “if I had to buy what those guys were selling, we’d still be using 1990s tech.” In other words: if you want to do something radical, you have to make your own gear. You have to vertically integrate.

What radical accomplishments does Sam Altman hope to attain through vertical integration? Independence. The ability to create a diverse hand of GPU fabs that won’t be subject to China-Taiwan tensions. Energy efficiency.

And a claim to the future of computing—which we all need to monitor.

Even if our financing is slightly more modest.