2 A Civilizing Mission
Greg Brockman became the first to commit to building OpenAI. To be Brockman’s cofounder, Altman handpicked Ilya Sutskever, then an AI researcher at Google whom Altman cold-emailed to come to the Rosewood dinner in the summer of 2015; Sutskever enthusiastically accepted upon learning that Musk would be in attendance.
Brockman and Sutskever made an interesting duo. Tall and stocky, with an amiable demeanor, Brockman was an engineer and a startup guy like Altman. He had grown up on a hobby farm in North Dakota. In between milking cows, he fell in love with math and then science. In 2008, he enrolled in Harvard and transferred to MIT two years later. After another semester, he dropped out of college entirely, unable to swallow any more school when he could be out in the real world building products. He moved to the Bay Area and joined Stripe as a budding startup with only three other people; he impressed the founders so much with his coding genius that he became chief technology officer. Over five years, he prototyped many of Stripe’s early products, helping it grow into a powerhouse fintech company that provides digital payments infrastructure to the likes of Amazon and Shopify. The run left him with significant wealth and the rarefied Valley status of having helped build a multibillion-dollar company.
Sutskever was the scientist. Lean and wiry, he was born in the Soviet Union and raised in Israel, where he blossomed as a math prodigy. After struggling to find teachers who could keep up with his advancement, his parents enrolled him as an eighth grader in courses at the Open University of Israel. At sixteen, he moved to Toronto and attended high school for just a month before being admitted to the University of Toronto in 2003 as a third-year undergraduate student. It was there that Sutskever met Geoffrey Hinton, a British Canadian professor who had done seminal work in AI research. Hinton became the only person whom Sutskever would call a mentor and who’d subsequently have a profound influence on his work and life. In 2012, together with another one of Hinton’s grad students, Alex Krizhevsky, they shocked the AI world by sweeping the floor at an academic contest called ImageNet to build software for automatically identifying objects in photos. Where every other team struggled to get their software’s error rate below 25 percent, Hinton, Sutskever, and Krizhevsky drove theirs down to 15 percent. Early the following year, Google announced that it had acquired their newly formed company, DNNresearch, in a heated auction for $44 million. The move minted the academics into multimillionaires and unleashed the first major rush to commercialize artificial intelligence. “We thought we were in a movie,” Hinton says.
Brockman and Sutskever met for the first time during the Rosewood dinner. Much like themselves, the others in the room were either entrepreneurs or scientists. The discussion ping-ponged back and forth between academic deliberations about different approaches to AI research and, Musk’s particular fixation, whether there was still time to beat out DeepMind and Google, essential, they believed, to correcting the course of AI development. The critical bottleneck, everyone agreed, was talent: Most of the top AI researchers were employed, like Sutskever, if not by Google, then by other tech giants, enjoying extravagant salaries, benefits, and job security.
AGI was also central to the discussion, which at the time was highly unusual. Most serious scientists considered the idea of digitally replicating true human-level intelligence to be science fiction, or at the very least decades or more away from attainability. Bold declarations that it was within reach enough to invest in it presently was viewed largely as pseudoscience and quackery. But Hassabis had embraced that term to describe the ambitions of DeepMind, despite a belief among his own research staff that this was distasteful, shameless marketing. The Rosewood group equally felt that the same goal, AGI, would best describe their own aspirations if they intended to form a competitor to go toe to toe with Hassabis’s organization.
Even to Sutskever, who secretly believed AGI was possible and would come to full-throatedly endorse some of the most aggressive predictions about the speed of its creation, the brazen talk at first made him a little squeamish. If other researchers found out that he was openly discussing the pursuit of this objective, he worried, he risked losing his credibility within the scientific community. Those concerns did not hold back Brockman, an outsider to the field and sincere in his belief that AGI, with enough effort and focus, could be just around the corner. That Sutskever and the other researchers were willing to at least privately entertain the feasibility of a new lab could have only strengthened Brockman’s confidence. As Altman drove him back from the hotel to San Francisco that night, Brockman told him that he was ready to commit himself to the project.
In Silicon Valley, there is a common saying: Becoming cofounders is like entering a marriage. As with any committed partnership, Brockman and Sutskever courted each other after Altman had diplomatically told Brockman he needed to be paired with someone who understood AI research. A few weeks after the Rosewood dinner, the two grabbed another meal alone in Mountain View. It was a perfect match. “I knew it was going to work though we’d just met…. Ilya and I had an extremely high-bandwidth interaction,” Brockman later wrote on his blog. “Our ideas enhanced and complemented one another.”
Over the next few months, as Sutskever thought through the proposal, Brockman took on the task of convincing others to join the new moonshot venture. He called leading figures in the field to get their recommendations for a list of top AI talent. He dined with professors at universities to ask about their best students. He heavily researched each candidate before any conversation to more persuasively recruit them. Altman would later extol those early efforts in a blog post simply titled “Greg.” “A lot of people ask me what the ideal cofounder looks like,” Altman wrote. “I now have an answer: Greg Brockman.”
Altman and Musk also had their fair share of recruiting conversations, slowly loosening up researchers resistant to the idea of AGI. “AGI might be far away, but what if it’s not?” Pieter Abbeel, a professor at the University of California, Berkeley, remembers Musk urging him. “What if it’s even just a 1 percent or 0.1 percent chance that it’s happening in the next five to ten years? Shouldn’t we think about it very carefully?” Abbeel would join OpenAI as a research adviser and later full time with several of his PhD students.
At first, many of the people Brockman approached were willing to sign on only if others were as well. Undeterred, he invited his ten most-wanted engineers and researchers to discuss their hesitations and rally their excitement over wine in Napa Valley. He hired a bus to drive everyone there and back so he could continue pitching them on the more than hourlong ride each way. Three weeks later, by Brockman’s deadline, nearly all had accepted his offer.
As Musk, Altman, and Brockman discussed how best to position OpenAI at launch, all were keenly aware of the importance of its public perception. They agreed with Altman’s proposal to make it a nonprofit and to play up the openness for which it was named. OpenAI, the anti-Google, would conduct its research for everyone, open source the science, and be the paragon of transparency.
“I hope for us to enter the field as a neutral group, looking to collaborate widely and shift the dialog towards being about humanity winning rather than any particular group or company,” Brockman wrote to Musk and Altman in November 2015. “(I think that’s the best way to bootstrap ourselves into being a leading research institution.)”
“There is a lot of value to having the public root for us to succeed,” Musk replied later that month, with a suggestion to rewrite the announcement of OpenAI’s formation to have broader appeal. “We need to go with a much bigger number than $100M to avoid sounding hopeless relative to what Google or Facebook are spending,” he added. “I think we should say that we are starting with a $1B funding commitment. This is real. I will cover whatever anyone else doesn’t provide.”
In later correspondence, the group acknowledged that they could walk back their commitments to openness once the narrative had served its purpose and as the need arose, such as to avoid bad actors getting their hands on the technology. “As we get closer to building AI, it will make sense to start being less open,” Sutskever raised to the trio in January 2016, shortly after OpenAI launched. “The Open in openAI means that everyone should benefit from the fruits of AI after its [sic] built, but it’s totally OK to not share the science.” “Yup,” Musk responded.
In December 2015, the announcement went out on a Friday night, to coincide with Neural Information Processing Systems, the largest annual AI research conference, where Hinton and Sutskever had auctioned off DNNresearch three years earlier. The blog post, “Introducing OpenAI,” listed each of the nine founding members, including Brockman, who would serve as CTO, and Sutskever, who would direct research. Musk and Altman would be cochairs. Altman and Brockman had also joined Musk in his pledge to see that the lab would have $1 billion in funding. So had Jessica Livingston, Peter Thiel, and LinkedIn cofounder Reid Hoffman. Hoffman had worked with Musk and Thiel at PayPal and often invested with them in startups as the “PayPal mafia.” “We expect to only spend a tiny fraction of this in the next few years,” the post said of the funding.
In the final countdown before the announcement, Sutskever had almost stayed at Google. To all of the other founding members, OpenAI had offered a base salary of $175,000 and YC or SpaceX stock. To Sutskever, the lab had instead offered him nearly $2 million, a whopping sum for a nonprofit. Even then, Google had offered him more, and then more again, reaching two or three times that amount, in a bid to keep him. Musk and Altman delayed the company announcement repeatedly as Sutskever agonized over the decision, calling his parents and fielding pleas from Musk and Brockman. In the end, Google’s dizzying offer underscored to Sutskever why a nonprofit like OpenAI was needed.
That day Musk marked the occasion with an email to the founding team solemnly pledging his commitment to making OpenAI victorious. “Our most important consideration is recruitment of the best people,” he wrote, which he promised to support, along with whatever else for which he could be helpful. “We are outmanned and outgunned by a ridiculous margin by organizations you know well,” he added, “but we have right on our side and that counts for a lot. I like the odds.” To preempt any other counteroffers from luring away members of their founding team, OpenAI immediately increased everyone’s base salary by another $100,000.
Musk would later recount facing the fury of Larry Page for personally poaching Sutskever. The two didn’t speak much again as their views continued to clash on AI development. But OpenAI’s recruiting suddenly became easier. Moonshots and associations with billionaires were a powerful draw in the Valley. Within a few months, the number of employees doubled.
The lack of clarity, the big check, and the billionaire worship were Silicon Valley at the heady peak of its unchecked power. But OpenAI had been clever with its positioning: It straddled the border between the techno-chauvinist version of Silicon Valley and a more conscientious strand that was emerging. Over the following year, Donald Trump’s spectacular rise and win in the 2016 US presidential election would shock the left-leaning workforce of the tech industry into self-reflection. As upheaval ripped through companies like Meta and Google and techlash sentiment gripped the public, AI researchers, too, began to question whether the field had moved too quickly to yoke its technologies to corporate bottom lines.
An accounting of the societal impacts of commercializing AI research returned an unsettling scorecard: Automated software being sold to the police, mortgage brokers, and credit lenders were entrenching racial, gender, and class discrimination. Algorithms running Facebook’s News Feed and YouTube’s recommendation systems had likely polarized the public, fueled misinformation and extremism, enabled election interference, and, most horrifying in the case of Facebook, precipitated ethnic cleansing in Myanmar.
But the main funding alternative, taking money from the government, had its own ethical land mines. In 2018, thousands of Google employees would protest a secret company contract with the Pentagon for its program known as Project Maven to develop AI-powered surveillance drones. The capabilities, employees said, could lay the groundwork for autonomous weapons; the Pentagon, which said this had not been its intention, would move away from that position with the Ukraine war.
It became a cynical refrain among AI researchers: sell out to Big Tech or to the military industrial complex, or leave AI research. Between these binary extremes, OpenAI seemed like a third way, corrupted by neither profit nor state power. “It was a beacon of hope,” said Chip Huyen, a machine learning engineer and popular tech blogger observing from the sidelines.
Not everyone was impressed. On the night of OpenAI’s launch in December 2015, Timnit Gebru, the AI researcher who’d questioned Musk about prioritizing the threats of AI over climate change, couldn’t believe the announcement.
All week the Stanford University graduate student, an Ethiopia-born Eritrean refugee who moved to the US as a teen, had been reminded of the high cost of being a Black woman in an environment dominated by white men. It was her first time joining the throngs of AI researchers at the weeklong Neural Information Processing Systems, then called NIPS and later rebranded to NeurIPS for short. She was one of the only Black people there. The following year at the 2016 conference, she would put an actual tally to it, counting only six other Black researchers among the 8,500 attendees. At Stanford, she joked about the lack of Black researchers on campus by saying she could found a Black in AI group and have meetings alone. She imagined starting a YouTube channel to lampoon the situation, changing her hairstyle and acting out different personalities to dramatize her speaking with herself. But as much as she tried to make light of her isolation, this conference in 2015 was reminding her how quickly things could turn hostile. At a party one night, she was getting some water when a group of drunk guys wearing Google Research T-shirts locked eyes on her and decided to make her the object of their fun. They surrounded her. One man forced her into a hug; another foisted a kiss on her cheek as he snapped a humiliating photo. At the same conference, a friend of hers was harassed by a professor.
Now here was a group of people—nine out of eleven of whom were white men—being showered in previously unheard-of amounts of money, speaking about the theoretical prospect of a bad superintelligence taking over the world, and proposing to counteract it by building a better superintelligence.
That night, Gebru drafted a scathing critique of what she’d observed in an anonymous open letter: the spectacle, the cultlike exaltation of AI celebrities, and, most of all, the overwhelming homogeneity of the people building and shaping such a consequential technology. This homogeneous culture was not only pushing away talented researchers but also leading to a dangerously narrow conception of AI and of who could benefit from the technology.
“We don’t have to project into the future to see AI’s potential adverse effects,” Gebru wrote. “It is already happening.”
On her flight back home from the conference, she thought twice about posting the letter anonymously. Instead she posted a shorter, more sanitized version of her critique, using her name on Facebook.
Several weeks later, she typed up an email with the subject line “Hello from Timnit.”
“When I go to computer vision conferences, I am often the only black person there,” she wrote. “But now I have seen 5 of you:) and thought that it would be cool if we started a black in AI group or at least know of each other.”
One by one she added the researchers’ emails. And then she pressed send.
In the early days of OpenAI, Altman and Musk were barely around as cochairmen. Busy with their full plate of other endeavors, the two left Brockman and Sutskever to build up the organization. As Sutskever rallied researchers to give him their best ideas, Brockman threw himself into the work of developing the right organizational culture.
Some years later, Brockman would recount to me his thinking. To prepare, he read every book he could find on ambitious science and technology undertakings in US history: the transcontinental railroad, Thomas Edison’s light bulb, the early network of computers that would lay the groundwork for the modern Web. He absorbed them like religious texts, searching for hints and guidance on how to design his own endeavor.
One story he held dear was the likely apocryphal tale of John F. Kennedy approaching a janitor holding a broom at the NASA space center. “Kennedy asks him, ‘Sir, what are you doing?’ And he says, ‘Oh, I’m helping put a man on the moon,’ ” Brockman recounted, clearly delighted. “Everyone having this sense of mission and purpose—I think that’s something really amazing and something I don’t see as reflected in what happens generally today.”
He later added: “I really feel like we as Americans have stopped daring to dream.”
To succeed, he believed, OpenAI needed that same level of alignment; every person at every level of the company needed to be like that janitor. He pointed out to me that, in fact, during the first few months of OpenAI, when everyone worked out of his apartment, he embodied that spirit literally and spent a lot of time cleaning people’s glassware. He created a company policy requiring all employees to work out of the San Francisco office, a policy that OpenAI would hold onto until the pandemic. This, of course, came with some trade-offs; not everyone wanted to live in the Bay Area, he acknowledged. I would learn through my other interviews that this was particularly true for women and people of color who, like Gebru, felt alienated by the white and male culture of the dominant tech industry. But to Brockman cohesion was more important, and being physically together helped with the serendipitous exchange of ideas.
Brockman decided, too, that he would call all OpenAI employees “members of technical staff,” inspired by Xerox PARC, the storied research and development lab in Palo Alto, which had done so, after a tradition at the equally famed Bell Labs in New Jersey, to create a more democratic work environment.
When considering the criticisms leveled at OpenAI for its pursuit of AGI, he drew parallels with Edison’s light bulb. “A committee of distinguished experts said ‘It’s never going to work,’ and one year later he shipped,” Brockman said. “How could that be?” It was, as science writer Arthur C. Clarke in the book Profiles of the Future called it, “a failure of imagination.”
Among the attendees at the Rosewood dinner had been Dario Amodei. Amodei, a computational neuroscientist turned AI researcher, was then working in the Silicon Valley–based AI lab of Chinese company Baidu before doing a brief stint at Google. His sister Daniela Amodei had worked with Brockman at Stripe, and when Brockman first started to engage seriously in AI developments, he had turned to Dario for learning resources. Dario didn’t join OpenAI immediately but was intrigued by the premise. OpenAI, under Musk’s influence, seemed to stand out from other AI labs as the most willing to focus on so-called AI safety.
In 2016, while still at Google, Amodei cowrote a foundational paper to the discipline, articulating a central problem in AI safety as addressing “the problem of accidents in machine learning systems, defined as unintended and harmful behavior that may emerge from poor design of real-world AI systems.” This was distinct from other AI-related challenges, he and his coauthors wrote, including privacy, security, fairness, and economic impact. AI “safety” in this framework, in other words, was about preventing rogue, misaligned AI—the root from which, as described by Nick Bostrom, superintelligence could become an existential threat.
To Amodei, there was no matter more important to work on: the prevention of superhuman AI causing catastrophic outcomes, even human extinction. Both Amodei siblings were sympathetic to the effective altruism, or EA, movement, a controversial ideology that had been spawned among philosophers at Oxford University, where Bostrom was based, and taken hold in Silicon Valley. Over time the movement, which preaches dedicating oneself to doing maximal good in the world by using extreme rationality and counterintuitive logic to guide decisions, had, in no small part due to Bostrom’s influence, identified the existential threat of rogue AI as a leading issue area for its adherents to pursue. Two years earlier, Daniela’s husband, Holden Karnofsky, had founded a nonprofit called Open Philanthropy to donate money in part based on EA principles. Open Phil, as it was called, would fast become the primary funder of catastrophic and existentially related AI safety research. (By November 2024, it had awarded more than three hundred AI-safety-related grants worth $440 million.)
But this existential brand of AI safety, built on philosophical thought experiments, would soon come under fire as the AI research community awakened to the less apocalyptic and immediate real-world harms of AI. Around the same time Amodei published his paper, ProPublica published a groundbreaking investigation called “Machine Bias” that revealed algorithms were being used across the US criminal justice system in misguided attempts to predict future criminals, and those algorithms were classifying Black people as higher risk than white ones who had more extensive criminal records. The piece, and an overall souring on Big Tech post-2016 over the harms of social media, sparked a new wave of research reckoning with the harmful societal impacts of AI.
Deborah Raji, an AI accountability researcher at the University of California, Berkeley, would come to champion the reexamination of the overwhelming focus of AI safety research on theoretical rogue AI and its possible existential risks to the detriment and de-prioritization of other real, evidence-based problems, coauthoring a 2020 paper in response to Amodei’s. She argued that truly “safe” AI systems could not be built by isolating the behaviors of the technical systems themselves without placing them in full context of their impacts on the very things—privacy, fairness, and economics— that Amodei had set apart. Where Amodei had raised the idea of AI creating “negative side effects” as it relentlessly pursued an objective, using an example akin to the paper clip thought experiment of a cleaning robot knocking over a vase or damaging the walls on its path to tidying up, Raji pointed out that this was already happening. In its relentless pursuit of commercial products and AGI, the AI industry had produced expansive negative side effects, including the wide-scale infringement of privacy to train facial recognition and the spiraling environmental costs of the data centers required to support the technology’s development.
“It is not just the actions of an AI agent that can produce side effects,” she and her coauthor wrote. “In real life, basic design choices involved in model creation and deployment processes also have consequences that reach far beyond the impact that a single model’s decision can have. In reality, for AI systems to even be built, there is very often a hidden human cost.”
Within OpenAI, various researchers, some of them among the small handful of women of color at the company, would press executives to expand their “AI safety” definition and include research on areas such as the discriminatory impacts of deep learning models. Executives were dismissive. “That’s not our role,” one said.
In May 2016, Amodei, still at Google, stopped by OpenAI’s office to see how things were going. OpenAI had just moved out of Brockman’s apartment to a space above a chocolate factory in San Francisco’s Mission District, the city’s oldest neighborhood and a Latino stronghold. Researchers padded around in socks.
“There are twenty to thirty people in the field, including Nick Bostrom and the Wikipedia article, who are saying that the goal of OpenAI is to build a friendly AI and then release its source code into the world,” Amodei told Altman and Brockman, according to an account in The New Yorker.
“We don’t plan to release all of our source code,” Altman said. “But let’s please not try to correct that. That usually only makes it worse.”
“But what is the goal?” Amodei asked. “Our goal right now…is to do the best thing there is to do,” Brockman replied. “It’s a little vague.”
Amodei joined two months later to lead AI safety research. Thereafter, Open Phil would donate $30 million to OpenAI to secure a three-year board seat for Holden Karnofsky. In 2018, at Brockman’s invitation, Daniela, who had been the first recruiter at Stripe, would also move over to OpenAI to build up its team as an engineering manager and its VP of people. “We have a long, cute history of knowing each other,” Daniela would joke to me of her and Brockman a year later. “That’s right,” Brockman would say, chuckling. “When we started OpenAI, and I started doing the initial recruiting here, I was like, ‘I really wish I had Daniela.’ ”
By the end of 2020, the Amodei siblings would become so disturbed by what they viewed as Altman’s and OpenAI’s break from its original premise that they would cleave off to form another AI lab, Anthropic, taking critical staff with them and creating a rivalry that would play a pivotal role in the frenzied release of ChatGPT. Karnofsky would step down from OpenAI’s board, having served his term and due to the new conflict of interest. On the list of candidates he nominated for his replacement, he would include one of his former employees: Helen Toner.
The problem was that OpenAI had no idea what it was doing. A year in, it had poached, begged, and borrowed its way to a stellar team in the aggressive fight for talent within the industry, keeping up the excitement internally just from the sheer density of top people. Still, it struggled to find a coherent strategy. And the momentum and shine were beginning to wear off.
Its list of projects sprawled every which way in a kitchen-sink reflection of the field. It was using robots and video games and simulated virtual worlds for training agents—all as ways of trying to reach more advanced AI capabilities. Little was working, and what did work felt derivative of something someone else had already done. Whatever AGI was, it wasn’t that. “The bigger projects that they had, it didn’t seem like they were doing anything super innovative,” says Nikhil Mishra, an AI researcher who interned at OpenAI in 2017.
Brockman’s and Sutskever’s leadership abilities were also being pushed to their limits. While Brockman spent most of his days coding, Sutskever stalked around the office repeatedly asking each researcher, “What’s your next big thing?” It made for a rudderless, high-stress environment. There was no real management structure or clear set of priorities. Sometimes people would get fired on the weekends, and the rest of the team would only find out the following Monday when they didn’t show up. And the lab was burning cash, most of it to hold down the salaries of the team it had assembled. In 2016, OpenAI spent more than $7 million out of its $11 million in expenses on compensation and benefits.
Musk was getting impatient. It didn’t help that DeepMind was suddenly garnering worldwide adulation. In March 2016, its program AlphaGo beat Lee Sedol, one of the world’s best human players in the ancient Chinese game of Go. (“Deepmind is causing me extreme mental stress,” Musk wrote to OpenAI leadership shortly before the five-game match. “If they win, it will be really bad news with their one mind to rule the world philosophy.”) The games were live streamed from South Korea to over two hundred million viewers. A year later Netflix released a blockbuster documentary about the company’s journey.
Musk came into the office periodically to demand more progress, at times setting completely unrealistic deadlines that were characteristic of his management philosophy. Many employees chafed at the expectations, believing they made no sense for the winding, unpredictable nature of research. During one all-hands meeting, Wojciech Zaremba, the robotics lead who had been part of the founding group, presented his plans for the kinds of robotics advancements he wanted his team to pursue. Musk had only one question: “When? When are you going to do those things?”
“I don’t know,” Zaremba said. Musk pushed back. “Well, then you don’t really have a plan.” So in March 2017, Brockman and Sutskever began in earnest to develop a more focused research road map. Their central question: What would it really take for OpenAI to reach AGI—and be the first to do so?
Sutskever intuitively believed it would have to do with one key dimension above all else: the amount of “compute,” a term of art for computational resources, that OpenAI would need to achieve major breakthroughs in AI capabilities. The ImageNet competition and subsequent advancements that he had been a part of had all involved a material increase in the amount of compute that had been used to train an AI model. The advancements had involved other things, too: significantly more data and more sophisticated algorithms. But compute, Sutskever felt, was king. And if it were possible to scale compute enough to train an AI model at human brain scale, he believed, something radical would surely happen: AGI.
The amount of compute is based on three things: the processing power of an individual computer chip, or how many calculations it can crunch per second; the total number of computer chips available; and how long they are left running to perform their calculations. The first is dictated by the computer chipmaking industry, which has for decades doubled the horsepower of a single chip every two years through intensive research and development. This rate of progress is known as Moore’s Law, based on a prediction that legendary Intel cofounder Gordon Moore first made in the 1960s, then revised a decade later, about how quickly his industry could innovate. Moore’s Law turned into a self-fulfilling prophecy. It became the target for how quickly chipmaking firms believed they needed to innovate in order to keep up with competition and stay relevant.
Brockman and Sutskever performed a simple calculation: Based on the pace of Moore’s Law, how long would it take to reach the level of compute OpenAI needed for brain-scale AI? The answer was bad news: It would take far too long.
Around the same time, Amodei and another researcher, Danny Hernandez, had begun to look at the same idea from a different direction. On a simple chart, with time as the x-axis, they plotted the amount of compute that every major breakthrough in AI research had actually used since 2012, beginning with Sutskever’s grad school breakthrough, the start of the AI revolution. They discovered that compute use was in fact growing faster than Moore’s Law. Much faster. In the last six years, it had doubled every 3.4 months, or, put another way, increased 30 million percent.
Brockman began to call this new doubling curve OpenAI’s Law. Not only did OpenAI need massively more amounts of compute to reach its end goal, he and the other leadership believed it also needed to scale its compute at a pace that at the very least matched this new law. Chipmaking firms had imposed Moore’s Law on their companies with existential fervor; the leadership now saw OpenAI’s Law in the same light.
If they couldn’t wait for Moore’s Law, they needed to grow their compute the other way: They needed a whole hell of a lot more chips.
The kinds of chips that OpenAI needed were expensive. Known as graphics processing units, or GPUs, they had originally been designed to quickly render graphics on computers, such as for giving video games a low-latency, glossy finish. But the same form factor excelled at training the AI models OpenAI wanted to develop, since they shared with graphics-rendering a common requirement: the need for crunching massive amounts of numbers in parallel.
The vast majority of the industry bought these GPUs from only one company: the Santa Clara–headquartered chipmaker Nvidia. Nvidia not only made the best GPUs in the world but also had developed a companion software platform called CUDA, short for Compute Unified Device Architecture, that had a powerful grip on AI developers.
In 2017, a custom Nvidia server with eight of their best GPUs cost $150,000 —a price that would rise roughly with inflation to nearly $195,000 by 2023. In the coming years, OpenAI’s Law was projecting that OpenAI would need thousands, if not tens of thousands, of GPUs to train just a single model. The cost of electricity to power that training would also explode. OpenAI needed more money—not just $1 billion, but billions of dollars to sustain itself in the coming years.
The realization would lead the organization to lose its financial footing. To Brockman and Sutskever, it challenged the very premise of OpenAI’s structure. How could a nonprofit raise that much annually to keep up with the pace required to stay number one? They briefly considered merging with a chip startup, but, in the summer of 2017, they began serious discussions with Altman and Musk about whether OpenAI needed to transform into a for-profit. That was their best hope to entice investors with a chance at generating a financial return. After several weeks of negotiations, the deliberations ended abruptly without resolution. If OpenAI were to become a for-profit, Altman, who was in the middle of considering his run for California governor and getting a lackluster reception in focus groups, wanted to be the company’s chief executive. So did Musk; he wanted full control of the lab and to have majority equity.
Caught in the middle, Sutskever and Brockman nearly went with the latter. The two preferred Musk’s leadership. But Altman appealed to Brockman directly with their personal relationship and concerns about Musk’s unreliability. Musk faced many external pressures and was prone to erratic and unstable behavior. Should OpenAI succeed, wouldn’t it be dangerous to give Musk full control of AGI? Convinced, Brockman appealed to Sutskever, who remained uncertain. In September 2017, he emailed Musk and Altman, on behalf of him and Brockman, in a last-ditch attempt to resolve the situation.
“Elon: We really want to work with you,” Sutskever wrote. “We believe that if we join forces, our chance of success in the mission is the greatest.” But Musk’s desire for total control felt antithetical to OpenAI’s original spirit, he said. “You are concerned that Demis could create an AGI dictatorship. So [are] we. So it is a bad idea to create a structure where you could become a dictator if you chose to.
“Sam: When Greg and I are stuck, you’ve always had an answer that turned out to be deep and correct,” Sutskever continued. That said, Altman’s behaviors had often left the two confused about his true beliefs and intentions. “We don’t understand why the CEO title is so important to you,” he wrote. “Your stated reasons have changed, and it’s hard to really understand what’s driving it. Is AGI truly your primary motivation? How does it connect to your political goals? How has your thought process changed over time?
“There’s enough baggage here that we think it’s very important for us to meet and talk it out,” his email concluded. “If all of us say the truth, and resolve the issues, the company that we’ll create will be much more likely to withstand the very strong forces it’ll experience.”
Within ten minutes, Musk had responded. “Guys, I’ve had enough. This is the final straw,” he wrote. If Sutskever and Brockman still wanted to pursue a for-profit, they would need to strike out on their own. Otherwise, OpenAI would continue as a nonprofit. “I will no longer fund OpenAI until you have made a firm commitment to stay or I’m just being a fool who is essentially providing free funding to a startup,” Musk said. Fifty minutes later, he followed up again. “To be clear, this is not an ultimatum to accept what was discussed before. That is no longer on the table.”
Altman piped up in the thread the following morning: “i remain enthusiastic about the non-profit structure!” He sent further assurance to Musk via one of Musk’s trusted deputies, Shivon Zilis, who worked at Tesla and Neuralink, his brain-machine interface company. “Great with keeping non-profit and continuing to support it,” Zilis wrote to Musk with notes of what Altman told her. “Admitted that he lost a lot of trust with Greg and Ilya through this process. Felt their messaging was inconsistent and felt childish at times.” Altman had also been bothered by how much Greg and Ilya kept sharing with the rest of OpenAI throughout the negotiations. “Felt like it distracted the team,” Zilis said.
But the reality was that keeping OpenAI a nonprofit wouldn’t solve its money problem. As Brockman and Sutskever continued to meet with potential nonprofit investors, they struggled to get anywhere near the kind of capital that they believed OpenAI would need. Musk’s capricious wavering on his funding commitment also threatened to throw OpenAI into a state of crisis. Behind the scenes, Altman began searching for funding alternatives and to wean off OpenAI’s dependency on Musk. He called Reid Hoffman, who offered to step in and hold down employee salaries and operational costs. He considered launching a new cryptocurrency. He investigated an array of different corporate structures, including a public benefit corporation, which Musk had been keen on and would allow OpenAI to become a for-profit while still legally binding it to its mission.
Compounding the urgency was an ever-present worry that OpenAI could lose its best researchers at any moment. Previously, with Musk’s firm backing, OpenAI had aggressively cranked up its nonprofit salaries to ward off counteroffers. Now the talent war had only grown more heated, and Musk himself had poached away one of OpenAI’s key founding scientists, Andrej Karpathy, in June 2017, to direct Tesla’s AI division. On compensation, OpenAI had a major disadvantage: It couldn’t offer equity into the organization, which many Bay Area tech workers viewed as necessary to afford the steep cost of living.
Musk soon arrived at his own conclusion for how to solve OpenAI’s money problem. In January 2018, Andrej Karpathy emailed Musk with new data showing how much Google was dominating top AI research publications. “Working at the cutting edge of AI is unfortunately expensive,” Karpathy wrote. “It seems to me that OpenAI today is burning cash and that the funding model cannot reach the scale to seriously compete with Google (an 800B company).”
While turning OpenAI into its own for-profit could help raise capital, it would require the lab to develop an AI product from scratch, a significant distraction from its fundamental AI research. “The most promising option I can think of, as I mentioned earlier, would be for OpenAI to attach to Tesla as its cash cow,” Karpathy said. Tesla had already done most of the heavy lifting to develop an AI product—namely, its self-driving function, Autopilot, he continued. If OpenAI could help speed up Tesla’s efforts to mature Autopilot into a full-fledged self-driving solution, that alone could possibly boost Tesla’s revenue enough to foot OpenAI’s costly compute bill.
Musk forwarded Karpathy’s email to Brockman and Sutskever. “Andrej is exactly right,” Musk wrote. “Tesla is the only path that could even hope to hold a candle to Google. Even then, the probability of being a counterweight to Google is small. It just isn’t zero.”
But by then, Altman had abandoned his political plans and succeeded in his efforts to persuade Brockman, and, through Brockman, Sutskever, that he would be the better leader. With the group’s decision, Musk no longer wanted to be publicly affiliated with the organization. “I will not be in a situation where the perception of my influence and time doesn’t match the reality,” he’d previously written. A few weeks later, Musk stepped down as OpenAI cochair. Altman became president of the nonprofit.
To the public, OpenAI framed the departure as Musk having a conflict of interest and stayed mum about its new financial reality: Of the $1 billion commitment, it ultimately received only around $130 million, less than $45 million of which had come from Musk. OpenAI’s future now rested on Altman’s singular fundraising abilities to recover those losses and continue to fulfill its accelerating need for even more capital.
Musk announced his decision to leave in person at an OpenAI all-hands meeting. To many employees, unaware of any of the drama at the leadership level, Musk’s departure brought a release of pressure but also significant uncertainty about the future of the organization. Until then, Musk had been a big driver of the lab’s public profile. During the meeting, he didn’t hold back: The need to make safe AGI first was imperative, and it was clear now that OpenAI would fail to do this as a nonprofit, he told employees; he would instead pursue the same goal at Tesla, which had far higher chances of succeeding with the deep coffers of a well-resourced company.
An intern questioned Musk’s intentions. Was this really the best solution? Had Musk really exhausted all alternatives? Advancing an OpenAI competitor at Tesla seemed like it would only serve to create for-profit race dynamics and could risk undermining safe AGI development. “Isn’t this going back to what you said you didn’t want to do?” the intern asked.
Musk blew up. “You’re a jackass! I’ve thought about this so much. I’ve tried everything. You can’t imagine how much time I’ve spent thinking about this,” he said. “I’m truly scared about this issue.”
The intern was later commemorated for his heroism with a “jackass” trophy. The day after Christmas that year, Musk wrote again to Altman, Brockman, and Sutskever:
SUBJECT LINE: I feel I should reiterate.
My probability assessment of OpenAI being relevant to DeepMind/Google without a dramatic change in execution and resources is 0%. Not 1%. I wish it were otherwise.
Even raising several hundred million won’t be enough. This needs billions per year immediately or forget it.
Altman needed to fundraise, fast.
OpenAI cranked up its publicity, focusing on demonstration projects that could highlight the lab’s capabilities to a lay audience. It leaned into one project in particular: an effort to build an AI agent that could beat the world’s best human players at the complex battle strategy video game Dota 2. OpenAI had already created an agent that could beat the best human player one on one. Now it would try to build a team of five agents to face off against the world’s best team of five human players.
Consciously or not, it was a page out of DeepMind’s book. Dota 2 had a worldwide championship that would be live streamed and spotlight OpenAI’s research in clear and dramatic win-or-lose terms. DeepMind had moved on to a similar project attempting to beat top human players in the strategy game StarCraft II, which could create an arbitrary yet natural comparison among potential OpenAI investors. The Dota 2 project was also compute heavy, a good way to test out and showcase the lab’s long-term scaling strategy. Brockman, who led the initial phase of the Dota 2 project, expanded his team and got to work.
Now all that was missing was a documentary. That task fell to a member of OpenAI’s robotics team. He bought expensive camera equipment and began following the Dota team around in the office. He wrote his own script and rough cut the footage into a three-hour-long saga. For all his efforts, people at OpenAI who reviewed the draft agreed that it was terrible. Professionals were hired, and Brockman began bankrolling them in part with his own money.
All the while, Altman fleshed out the plan for raising money. After considering a variety of for-profit structures, he landed on an unusual proposal to balance the need for capital with a continued commitment to OpenAI’s mission. While benefit corporations had a built-in mechanism for maintaining this balance, they also came with too many other rules. Instead, Altman would create a limited partnership, or LP, to act as a for-profit arm for receiving investment and commercializing OpenAI’s technologies. That arm would place a ceiling on investors’ returns and be governed by OpenAI’s nonprofit. The advantage was that the operating agreement for LPs could be written based on whatever the creator wanted. OpenAI could specify that the mission took precedence over investors. LPs also limited the power shareholders could exercise so they never gained majority control.
Altman framed the proposal to employees carefully: OpenAI’s initial commitment to avoid profit motives was made in the spirit of preventing the lab from compromising on its mission. But given that the lab’s success required capital the nonprofit couldn’t raise, clinging onto the original structure now held a greater risk of endangering the mission. In the end, most people agreed, though some reluctantly, that the LP was the best way forward.
In April 2018, OpenAI released a charter to pave the way for the transition. Without publicly revealing anything about the change to come, the document reiterated the lab’s purpose, now with new wording: “OpenAI’s mission is to ensure that artificial general intelligence (AGI)…benefits all of humanity.” Such a mission, the document added, would need OpenAI to be “on the cutting edge of AI capabilities” and require “substantial resources”; it could mean walking back the commitment to release the lab’s research due to “safety and security concerns.” For the first time, OpenAI also spelled out its AGI definition: “highly autonomous systems that outperform humans at most economically valuable work.”
That summer, as the Dota team began winning amateur matches and trumpeting its results across tech media (“OpenAI’s Dota 2 AI Steamrolls World Champion E-sports Team with Back-to-Back Victories,” lauded one headline), Altman bumped into Microsoft CEO Satya Nadella at the Allen & Company conference in Sun Valley, Idaho. The annual event, known as the “summer camp for billionaires,” had been the backdrop for many a major corporate deal. Altman was ready to strike his own.
He pitched Nadella on an OpenAI investment, enough to pique the chief executive’s interest. But Nadella questioned whether he should invest in an external organization when his company had its own long-standing AI research division within Microsoft Research. When he returned to Microsoft, he posed the question to his senior advisers.
“Microsoft Research and OpenAI are both organizations pushing the frontier,” Xuedong Huang, then the chief technology officer of Azure AI, reasoned. Why not invest in both?
Within half a year, OpenAI and Microsoft were discussing a deal in earnest. Altman laid the legal groundwork, hurrying along the creation of the limited partnership and appointing himself as its CEO. Internally, the project was code-named Oregon Trail. To keep the deal secret from prying eyes, the for-profit entity was also incorporated under the alias SummerSafe LP. The name was a reference to an episode of the cartoon show Rick and Morty where the titular characters, mad scientist Rick and his grandson Morty, leave behind Morty’s older sister Summer for another universe and instruct their car to “keep Summer safe.” The car takes the objective seriously, resorting to extreme and harmful mechanisms of defense, including murdering, paralyzing, and torturing people who approach the vehicle. It was a nod to the potential pitfalls of AI.
In early 2019, senior Microsoft leadership began coming through the OpenAI office. First came Kevin Scott, the tech giant’s excitable chief technology officer, who had followed OpenAI and grown particularly fond of the startup; then came Craig Mundie, a senior adviser to Nadella who had served on Microsoft leadership, including as its chief research and strategy officer, for over twenty years. Bill Gates also turned up, reserved and tight-lipped as usual, as he watched a series of demos. Most employees were left in the dark about Microsoft’s engagement. Altman told the small team working on the deal to keep knowledge of a possible investment limited.
Around the same time, Altman began to face trouble at YC. After five years as head of the organization, frustration with Altman had reached critical levels over an issue strikingly similar to one that had arisen at Loopt: his seeming prioritization of his own projects and aspirations over the organization’s— sometimes even at its expense. The amount of time he was spending on OpenAI negotiations and away from advising YC startups wasn’t helping. Some saw Altman as reaping significant personal benefit, gaining massive returns by investing in YC companies with his own personal fund Hydrazine, while doing limited work. Upon learning of his absenteeism, a concerned Jessica Livingston urged Altman to step down from the YC presidency, according to The Washington Post. Altman agreed. In early 2019, Paul Graham flew from the UK, where he had retired, to San Francisco to finalize the decision.
Altman tried to smooth over the change publicly. On March 8, 2019, the day he hosted Senator Schumer, he published a blog post on YC’s website announcing that he would transition from YC president to chairman to make more time for OpenAI. Days later, on March 11, Brockman and Sutskever publicly unveiled OpenAI LP, and Altman revealed his role as its chief executive. The timing was artful. The media widely reported Altman’s move as a well-choreographed step in his career and his new role as YC chairman. Except that he didn’t actually hold the title. He had proposed the idea to YC’s partnership but then publicized it as if it were a foregone conclusion, without their agreement, The Wall Street Journal reported. The blog post was later edited to remove mention of Altman completely.
At OpenAI, Altman’s new title merely formalized the role he had been playing since Musk’s departure. When Altman took the reins, many employees were relieved. His calm and collected demeanor was a welcome alternative to Musk’s intensity and unpredictable mood swings. Altman also helped alleviate mounting gripes with Brockman’s and Sutskever’s management. He brought in an executive coach and provided training to the managers. He installed more senior leaders, bringing in Brad Lightcap, an investor at YC, to be chief financial officer; promoting Bob McGrew, who had formerly led engineering and product management at the Thiel-founded Palantir, from the robotics team to a VP of research; and hiring Mira Murati, who had led product and engineering at the virtual reality startup Leap Motion and for Tesla’s Model X, to oversee hardware strategy and a core line of research.
With the formation of OpenAI LP, most employees resigned from the nonprofit and signed new contracts, now with equity, under the for-profit. (The exceptions included international employees on visas tied to the nonprofit.) A payband structure tied compensation not just to “engineering expertise” and “research direction,” but also to charter alignment. Level three employees needed to “understand and internalize the OpenAI charter.” Level fives needed to “ensure all projects you and your team-mates work on are consistent with the charter.” Level sevens were “responsible for upholding and improving the charter, and holding others in the organization accountable for doing the same.” Executives also wrote up an FAQ doc to manage residual nerves. “Can I trust OpenAI?” one question asked. The answer began with “Yes.”
In the broader tech world, OpenAI’s transition set off a wave of accusations that the lab was walking back its original promise. The initial terms of the limited partnership stated that the first round of investors would have their returns capped at 100x of what they put in. OpenAI termed the invented structure a “capped-profit” company. In a post on Hacker News, a popular news aggregation website run by YC, a user asked how this cap was at all meaningful. “So someone who invests $10 million has their investment ‘capped’ at $1 billion. Lol. Basically unlimited unless the company grew to a FAANG-scale market value,” they wrote, using the acronym for Facebook, Apple, Amazon, Netflix, and Google.
Brockman responded under his username, gdb: “We believe that if we do create AGI, we’ll create orders of magnitude more value than any existing company.”
Another user followed up. “Early investors in Google have received a roughly 20x return on their capital. Google is currently valued at $750 billion. Your bet is that you’ll have a corporate structure which returns orders of magnitude more than Google…but you don’t want to ‘unduly concentrate power’?” they wrote, quoting from the charter. “What exactly is power, if not the concentration of resources?”
Initial investments poured in to the LP, including more than $60 million rolled over from OpenAI’s nonprofit, $10 million from YC, and $50 million each from Khosla Ventures and Hoffman’s charitable foundation. Hoffman was initially reluctant to invest more in OpenAI when it had no product or market plan, he later recounted. But he ultimately agreed to colead the round after Altman told him it would help legitimize the seriousness of OpenAI’s intention to develop a profitable business.
Microsoft, meanwhile, continued to deliberate. Nadella, Scott, and other Microsoft executives were already on board with an initial investment. The one holdout was Bill Gates.
For Gates, Dota 2 wasn’t all that exciting. Nor was he moved by robotics. The robotics team had created a demo of a robotic hand that had learned to solve a Rubik’s Cube through its own trial and error, which had received universally favorable coverage. Gates didn’t find it useful. He wanted an AI model that could digest books, grasp scientific concepts, and answer questions based on the material—to be an assistant for conducting research.
OpenAI had only one project that approached fitting the bill: a large language model called GPT-2 that was capable of generating passages of text that closely resembled human writing. In February that year, OpenAI had taken the unusual step of proclaiming to the press that this model, once advanced a little further, could become an exceedingly dangerous technology. Authoritarian governments or terrorist organizations could weaponize the model to mass-produce disinformation. Users could overwhelm the internet with so much trash content that it would be difficult to find high-quality information. OpenAI would take the ethical high road, it said, and withhold the full version of the model, which had 1.5 billion parameters, or variables, an approximate measure of a model’s size and complexity. Instead, to give the public just a taste of the kind of capabilities that society needed to prepare for, it would publish only a diminished version, less than one-tenth of the size, that had a limited ability to generate a few sentences at a time but was prone to non sequiturs and repetition.
GPT-2 wasn’t even close to grasping scientific concepts, but the model could do some basic summarization of documents and sort of answer questions. Perhaps, some of OpenAI’s researchers wondered, if they trained a larger model on more data and to perform tasks that at least looked more like what Gates wanted, they could sway him from being a detractor to being, at minimum, neutral. In April 2019, a small group of those researchers flew to Seattle to give what they called the Gates Demo of a souped-up GPT-2. By the end of it, Gates was indeed swayed just enough for the deal to go through.
In a subsequent all-hands, Altman delivered the news, championing Microsoft as the right investor and partner. The tech giant had the money and the compute that OpenAI needed, and its leadership was deeply value aligned with the mission to ensure beneficial AGI. OpenAI had also made very loose commitments around what to deliver to Microsoft for commercialization. The lab hadn’t needed to compromise on much of anything, Altman said. It was a very good deal.
Within Microsoft, the investment was framed practically. Whether OpenAI did or didn’t reach AGI wasn’t really their concern. But OpenAI was clearly on the cutting edge, and investing early could finally turn Microsoft into an AI leader—both in software and in hardware—on par with Google. “The thing that’s interesting about what Open AI and Deep Mind and Google Brain are doing is the scale of their ambition,” wrote Scott to Nadella and Gates in mid-June, referring to Google’s AI research division, “and how that ambition is driving everything from datacenter design to compute silicon to networks and distributed systems architectures to numerical optimizers, compiler, programming frameworks, and the high level abstractions that model developers have at their disposal.” Microsoft was desperately behind on multiple fronts, he said: It had struggled to replicate Google’s best language models, and its Azure cloud-computing platform had large gaps compared with Google’s equivalent infrastructure. It could take years for Microsoft to catch up by itself. He was “very, very worried.”
Nadella responded the same day, removing Gates and adding Microsoft’s CFO Amy Hood. “Very good email that explains, why I want us to do this…and also why we will then ensure our infra folks execute,” he said, using the abbreviation for infrastructure.
A month later, on July 22, 2019, Microsoft announced its $1 billion investment. Under the terms of the deal, its returns would be capped at 20x.