For years, Elon Musk has talked about Dojo — the AI supercomputer that would be the cornerstone of Tesla’s AI ambitions. It’s essential sufficient to Musk that he lately mentioned the corporate’s AI staff goes to “double down” on Dojo as Tesla gears as much as reveal its robotaxi in October.
However what precisely is Dojo? And why is it so essential to Tesla’s long-term technique?
Briefly: Dojo is Tesla’s custom-built supercomputer that’s designed to coach its “Full Self-Driving” neural networks. Beefing up Dojo goes hand-in-hand with Tesla’s aim to succeed in full self-driving and convey a robotaxi to market. FSD, which is on about 2 million Tesla autos immediately, can carry out some automated driving duties, however nonetheless requires a human to be attentive behind the wheel.
Tesla delayed the reveal of its robotaxi, which was slated for August, to October, however each Musk’s public rhetoric and knowledge from sources inside Tesla inform us that the aim of autonomy isn’t going away.
And Tesla seems poised to spend massive on AI and Dojo to succeed in that feat.
Tesla’s Dojo backstory
Musk doesn’t need Tesla to be simply an automaker, or perhaps a purveyor of photo voltaic panels and vitality storage techniques. As an alternative, he desires Tesla to be an AI firm, one which has cracked the code to self-driving vehicles by mimicking human notion.
Most different corporations constructing autonomous automobile know-how depend on a mix of sensors to understand the world – like lidar, radar and cameras – in addition to high-definition maps to localize the automobile. Tesla believes it could obtain absolutely autonomous driving by counting on cameras alone to seize visible information after which use superior neural networks to course of that information and make fast choices about how the automobile ought to behave.
As Tesla’s former head of AI, Andrej Karpathy, mentioned on the automaker’s first AI Day in 2021, the corporate is mainly making an attempt to construct “an artificial animal from the bottom up.” (Musk had been teasing Dojo since 2019, however Tesla formally introduced it at AI Day.)
Corporations like Alphabet’s Waymo have commercialized Stage 4 autonomous autos – which the SAE defines as a system that may drive itself with out the necessity for human intervention below sure situations — via a extra conventional sensor and machine studying strategy. Tesla has nonetheless but to provide an autonomous system that doesn’t require a human behind the wheel.
About 1.8 million folks have paid the hefty subscription worth for Tesla’s FSD, which at present prices $8,000 and has been priced as excessive as $15,000. The pitch is that Dojo-trained AI software program will ultimately be pushed out to Tesla prospects by way of over-the-air updates. The size of FSD additionally means Tesla has been capable of rake in tens of millions of miles value of video footage that it makes use of to coach FSD. The concept there’s that the extra information Tesla can accumulate, the nearer the automaker can get to really reaching full self-driving.
Nevertheless, some business specialists say there is perhaps a restrict to the brute power strategy of throwing extra information at a mannequin and anticipating it to get smarter.
“To start with, there’s an financial constraint, and shortly it would simply get too costly to try this,” Anand Raghunathan, Purdue College’s Silicon Valley professor {of electrical} and pc engineering, instructed TechCrunch. “Some folks declare that we’d truly run out of significant information to coach the fashions on. Extra information doesn’t essentially imply extra info, so it depends upon whether or not that information has info that’s helpful to create a greater mannequin, and if the coaching course of is ready to truly distill that info into a greater mannequin.”
Raghunathan says regardless of these doubts, the pattern of extra information seems to be right here for the short-term a minimum of. And extra information means extra compute energy wanted to retailer and course of all of it to coach Tesla’s AI fashions. That’s the place Dojo, the supercomputer, is available in.
What’s a supercomputer?
Dojo is Tesla’s supercomputer system that’s designed to perform as a coaching floor for AI, particularly FSD. The identify is a nod to the house the place martial arts are practiced.
A supercomputer is made up of 1000’s of smaller computer systems referred to as nodes. Every of these nodes has its personal CPU (central processing unit) and GPU (graphics processing unit). The previous handles general administration of the node, and the latter does the complicated stuff, like splitting duties into a number of elements and dealing on them concurrently. GPUs are important for machine studying operations like people who energy FSD coaching in simulation. In addition they energy massive language fashions, which is why the rise of generative AI has made Nvidia probably the most useful firm on the planet.
Even Tesla buys Nvidia GPUs to coach its AI (extra on that later).
Why does Tesla want a supercomputer?
Tesla’s vision-only strategy is the primary purpose. The neural networks behind FSD are skilled on huge quantities of driving information to acknowledge and classify objects across the automobile after which make driving choices. Which means, when FSD is engaged, the neural nets have to gather and course of visible information constantly at speeds that match the depth and velocity recognition capabilities of a human.
In different phrases, Tesla means to create a digital duplicate of the human visible cortex and mind perform.
To get there, Tesla must retailer and course of all of the video information collected from its vehicles all over the world and run tens of millions of simulations to coach its mannequin on the info.
To present you a way of scale, Tesla mentioned that as of Might 2024, Tesla autos with FSD model 12 have pushed 300 billion miles already.
Tesla seems to depend on Nvidia to energy its present Dojo coaching pc, however it doesn’t wish to have all its eggs in a single basket — not least as a result of Nvidia chips are costly. Tesla additionally hopes to make one thing higher that will increase bandwidth and reduces latencies. That’s why the automaker’s AI division determined to provide you with its personal {custom} {hardware} program that goals to coach AI fashions extra effectively than conventional techniques.
At that program’s core is Tesla’s proprietary D1 chips, which the corporate says are optimized for AI workloads.
Inform me extra about these chips
Tesla is of the same opinion to Apple, in that it believes {hardware} and software program ought to be designed to work collectively. That’s why Tesla is working to maneuver away from the usual GPU {hardware} and design its personal chips to energy Dojo.
Tesla unveiled its D1 chip, a silicon sq. the scale of a palm, on AI Day in 2021. The D1 chip entered into manufacturing as of a minimum of Might this 12 months. The Taiwan Semiconductor Manufacturing Firm (TSMC) is manufacturing the chips utilizing 7 nanometer semiconductor nodes. The D1 has 50 billion transistors and a big die dimension of 645 millimeters squared, in accordance with Tesla. That is all to say that the D1 guarantees to be extraordinarily highly effective and environment friendly, and deal with complicated duties rapidly.
“We will do compute and information transfers concurrently, and our {custom} ISA, which is the instruction set structure, is absolutely optimized for machine studying workloads,” mentioned Ganesh Venkataramanan, former senior director of Autopilot {hardware}, at Tesla’s 2021 AI Day. “This can be a pure machine studying machine.”
The D1 continues to be not as highly effective as Nvidia’s A100 chip, although, which can be manufactured by TSMC utilizing a 7 nanometer course of. The A100 comprises 54 billion transistors and has a die dimension of 826 sq. millimeters, so it performs barely higher than Tesla’s D1.
To get the next bandwidth and better compute energy, Tesla’s AI staff fused 25 D1 chips collectively into one tile to perform as a unified pc system. Every tile has a compute energy of 9 petaflops and 36 terabytes per second of bandwidth, and comprises all of the {hardware} obligatory for energy, cooling and information switch. You possibly can consider the tile as a self-sufficient pc made up of 25 smaller computer systems. Six of these tiles make up one rack, and two racks make up a cupboard. Ten cupboards make up an ExaPOD. At AI Day 2022, Tesla mentioned Dojo would scale by deploying a number of ExaPODs. All of this collectively makes up the supercomputer.
Tesla can be engaged on a next-gen D2 chip that goals to resolve info circulation bottlenecks. As an alternative of connecting the person chips, the D2 would put the whole Dojo tile onto a single wafer of silicon.
Tesla hasn’t confirmed what number of D1 chips it has ordered or expects to obtain. The corporate additionally hasn’t offered a timeline for a way lengthy it would take to get Dojo supercomputers working on D1 chips.
In response to a June publish on X that mentioned: “Elon is constructing an enormous GPU cooler in Texas,” Musk replied that Tesla was aiming for “half Tesla AI {hardware}, half Nvidia/different” over the following 18 months or so. The “different” could possibly be AMD chips, per Musk’s remark in January.
What does Dojo imply for Tesla?
Taking management of its personal chip manufacturing signifies that Tesla may at some point have the ability to rapidly add massive quantities of compute energy to AI coaching packages at a low price. Notably as Tesla and TSMC scale up chip manufacturing, making the chips extra inexpensive.
It additionally signifies that Tesla might not must depend on Nvidia’s chips sooner or later, that are more and more costly and exhausting to safe.
Throughout Tesla’s second-quarter earnings name, Musk mentioned that demand for Nvidia {hardware} is “so excessive that it’s typically troublesome to get the GPUs.” He mentioned he was “fairly involved about truly with the ability to get regular GPUs after we need them, and I feel this due to this fact requires that we put much more effort on Dojo as a way to make sure that we’ve bought the coaching functionality that we want.”
That mentioned, Tesla continues to be shopping for Nvidia chips immediately to coach its AI. In June, Musk posted on X:
“Of the roughly $10B in AI-related expenditures I mentioned Tesla would make this 12 months, about half is inside, primarily the Tesla-designed AI inference pc and sensors current in all of our vehicles, plus Dojo. For constructing the AI coaching superclusters, Nvidia {hardware} is about 2/3 of the associated fee. My present finest guess for Nvidia purchases by Tesla are $3B to $4B this 12 months.”
Inference compute refers back to the AI computations carried out by Tesla vehicles in actual time, and is separate from the coaching compute that Dojo is liable for.
Dojo is a dangerous wager, one which Musk has hedged a number of instances by saying that Tesla won’t succeed.
In the long term, Tesla might theoretically create a brand new enterprise mannequin based mostly on its AI division. Musk has mentioned that the primary model of Dojo will probably be tailor-made for Tesla pc imaginative and prescient labeling and coaching, which is nice for FSD and coaching Optimus, Tesla’s humanoid robotic. However it wouldn’t be helpful for a lot else.
Musk has mentioned that future variations of Dojo will probably be extra tailor-made to normal function AI coaching. One potential drawback with that’s that the majority AI software program out there was written to work with GPUs. Utilizing Dojo to coach normal function AI fashions would require rewriting the software program.
That’s, until Tesla rents out its compute, just like how AWS and Azure lease out cloud computing capabilities. Musk additionally famous throughout Q2 earnings that he sees “a path to being aggressive with Nvidia with Dojo.”
A September 2023 report from Morgan Stanley predicted that Dojo might add $500 billion to Tesla’s market worth by unlocking new income streams within the type of robotaxis and software program providers.
Briefly, Dojo’s chips are an insurance coverage coverage for the automaker, however one that would pay dividends.
How far alongside is Dojo?
Reuters reported final 12 months that Tesla started manufacturing on Dojo in July 2023, however a June 2023 publish from Musk steered that Dojo had been “on-line and working helpful duties for just a few months.”
Across the identical time, Tesla mentioned it anticipated Dojo to be one of many prime 5 strongest supercomputers by February 2024 — a feat that has but to be publicly disclosed, leaving us uncertain that it has occurred. The corporate additionally mentioned it expects Dojo’s complete compute to succeed in 100 exaflops in October 2024.
(1 exaflop is the same as 1 quintillion pc operations per second. To succeed in 100 exaflops and assuming that one D1 can obtain 362 teraflops, Tesla would want greater than 276,000 D1s, or round 320,500 Nvidia A100 GPUs.)
Tesla additionally pledged in January 2024 to spend $500 million to construct a Dojo supercomputer at its gigafactory in Buffalo, New York.
In Might 2024, Musk famous that the rear portion of Tesla’s Austin gigafactory will probably be reserved for a “tremendous dense, water-cooled supercomputer cluster.”
Simply after Tesla’s second-quarter earnings name, Musk posted on X that the automaker’s AI staff is utilizing Tesla HW4 AI pc (renamed AI4), which is the {hardware} that lives on Tesla autos, within the coaching loop with Nvidia GPUs. He famous that the breakdown is roughly 90,000 Nvidia H100s plus 40,000 AI4 computer systems.
“And Dojo 1 can have roughly 8k H100-equivalent of coaching on-line by finish of 12 months,” he continued. “Not large, however not trivial both.”