Even as OpenAI’s approach stirred increasing controversy, the company’s resolve in scaling only strengthened. To executives, GPT-3 had definitively proved the existence of scaling laws. Now, at the start of 2021, they were ready to exploit this winning formula. The Anthropic team’s departure had also diluted the internal stronghold of resistance against commercialization. With new consensus, the remaining leadership put together a research road map laying out the narrowed focus of the company’s research and how it would feed into productization in a self-reinforcing loop.

“Our primary 2021 goal is to build an aligned system that is vastly more capable than anything that existed before,” the road map began. This system would at a baseline be a language model, but could also be trained to develop multimodal capabilities. “The goal is challenging, but we can see a path to achieving it in 2021,” it said.

That path involved three things: First was scaling GPT-3 by another 10x using a new supercomputer from Microsoft arriving in the third quarter with eighteen thousand Nvidia A100s, the newest, most powerful GPUs then in existence. Second was doing more research to increase by 25x OpenAI’s compute efficiency, or how much processing power it could milk out of its available chips. Third was improving the quantity and quality of training data, in part by tapping into user data and shifting the model toward the best parts of the data distribution with reinforcement learning from human feedback.

Beneath a section titled “How to accomplish it,” the road map elaborated further. As an initial step, OpenAI would bring various deep learning models up to “a large scale,” including a language model, a code-generation model, an image-to-text model for describing images, and a text-to-image model for generating images from a text prompt. It would also start a project to develop a digital “agent”—an AI model that would not just generate humanlike outputs but could be given a goal, such as to send an email, and operate autonomously to achieve it. As the next step, the company would then select one of these models to scale “to the limit afforded by our 18k A100 cluster.” The language and code models would also be turned into products and released to gather real-world data from the people using them: “New in 2021: we emphasize deploying models as products and learning from user interaction, as it can be a data flywheel that can lead to vast capability improvements.”

Under another section, titled “Details,” the document rationalized why this approach made both scientific and business sense. OpenAI’s previous experience had demonstrated that more scale was its “most reliable way of achieving new capabilities.” Scientifically, that meant that scaling was its best hope of attaining a breakthrough—some kind of capability that previously seemed impossible. And scaling language and code models in particular was “tantalizing due to the mere possibility” that they could reach breakthroughs in human-level meta-learning, or learning to learn, and reasoning. Scaling multimodal models, meanwhile, could potentially quicken the pace of improvements even further. With enough breakthroughs, the document said with remarkable definitiveness, “we will actually reach AGI.”

Strategically as a business, scaling each of these models would also “develop capabilities that we wish to utilize for some end.” Better language and code models could make OpenAI itself more productive and accelerate its advancement. Along with better text-to-image models, they would also “lead to amazing products.”

In parallel, OpenAI needed to invest in compute efficiency. While scaling had worked wonders to keep the company in first position, the strategy was beginning to taper. “Over the past 2 years, we’ve made astounding progress by scaling, simply because there has been a large hardware overhang,” the document said, referring to the fact that AI researchers had previously failed to use the maximum number of computer chips available to train AI models. As such, “we were able to massively outperform the rest of the ML world by using all available compute to train models of then-unprecedented size and capability.”

Now OpenAI was “approaching the limit” of the amount of compute it could possibly acquire at any given moment. It was also facing new competition from “other labs” that had adopted the same scaling strategy, the document acknowledged, without explicitly naming Anthropic. “Our capacity ramp is such that in the next two years, we will be able to train one model that uses 100x the compute of GPT-3.” While scaling alone would produce “very formidable” progress, it would not match the leap from GPT-2 to GPT-3, which had been driven by a 500x compute increase. “All additional progress must come from better methods,” the road map concluded.

The road map listed several areas of exploration for identifying those methods. Some of these it called “2x” and “10x” methods—those that might be able to achieve 2x or 10x gains in compute efficiency. The suggested methods included distillation, reverse engineering smaller models from larger ones; data filtering, finding the data that would produce the biggest leaps in performance; and sparsity, developing so-called sparse, or lighter weight, AI models. The last one referred to a feature of neural networks: In a traditional deep learning model, a neural network is “densely” connected, with every node in a layer wired to every node in another. Sparse models are trained with only a small subset of the nodes connected, with the aim of significantly reducing the computational costs in exchange for slightly less accurate models that are still good enough for most purposes.

On top of the 2x and 10x work, the Research division needed to look for other methods that would “steepen the slope of the scaling law”—those that would produce greater leaps in model performance without increasing its data, parameters, and compute. OpenAI’s best hypothesis so far for the most promising new methods, the document said, were reasoning and active learning —a technique that involved an AI model iteratively identifying which parts of a dataset to prioritize for human workers to annotate. (Shortly thereafter, a group of OpenAI researchers would discover a small error in the original scaling laws that meant that the company needed to train its models slightly longer than previously understood to get better performance. With a hint of smugness, the group would tout the findings as a unique competitive advantage: “This is something we have now that Anthropic doesn’t.” A year later, Google would release a paper that made public the same result.)

Finally, the road map added, OpenAI needed to start searching for “the breakthrough system of the future”—a breakthrough as meaningful as GPT-1 to give the company a new path of development to exploit. Perhaps this would emerge from its scaling and computational efficiency work, but it would continue to conduct exploratory research at the cutting edge of the field, including developing algorithms for solving math problems, experimenting with multiagent systems, and pursuing other new ideas. As part of this work, researchers would continue to “study the science of deep learning to better understand how our tools really work.” In other words, OpenAI needed to better understand what exactly it was that the company was building.

---

As the Research division proceeded along its road map, the Applied division slowly built up its forces, hiring a go-to-market lead, a sales team, and more engineers. With the GPT-3 API serving a growing base of developers, it became the testing ground for working out the kinks of productization and monetization, including adapting the company’s back-end infrastructure to support a service with users and coming up with a pricing strategy.

Serving users also brought up questions of what kinds of behaviors OpenAI would and would not allow with its products, as well as new responsibilities to enforce those rules. The company didn’t yet have an official team for trust and safety, an established discipline within the tech industry—not to be confused with the existentially related concerns of OpenAI’s Safety clan—for handling such questions and for anticipating and preventing a broad range of internet abuses, such as money laundering, cyberbullying, and misinformation. Instead, OpenAI hired a small group of staff and contractors to review the applications developers were submitting to get access to the API, and to reject or approve them based on ad hoc rules the team drafted along the way. Many of the lines they drew were arbitrary. They decided to accept companion bots but not sex bots; to allow apps for generating social media copy but not ones that posted directly to social media platforms or impersonated public figures. “It was all vibes basically,” said a person who was involved with making the guidelines. “There was a lot of figuring it out as we were reviewing.”

Another team, led by Ari Herbert-Voss, a research scientist with a background in security who had joined in 2019, sought to discover and patch GPT-3’s undesirable behaviors and error-prone outputs by attempting to break and exploit the model in various ways and then designing mechanisms to make it more resistant to failure and misuse. One mechanism included developing an early version of the content-moderation filter for which OpenAI would later contract workers in Kenya. Researchers trained the filter on whatever examples they could find or think to write and generate from AI models themselves. But when it was shipped, the filter worked poorly, blocking broad swaths of benign content, such as basic references to Black or trans people. Developers and other API users complained. Many people on the API team had already been reticent to apply any filtering at all, worried about it degrading customer experience. They made the filter optional.

OpenAI called this process of stress testing and refining the model “red teaming,” a term borrowed from the cybersecurity industry that refers to a systematic and thorough process of verifying the security of an organization and its capabilities to respond to an attack. OpenAI’s version of red teaming was and still is not the same thing, says Heidy Khlaaf, a safety engineer, cybersecurity expert, and AI researcher. It is patchy and ad hoc, and does not establish any guarantees on the safety and security of the model. Khlaaf, who worked with OpenAI during its early days of trying to establish its stress-testing protocols, subsequently grew alarmed at how the AI industry co-opted long-established phrasing from her field to create a false veneer of rigor. “In software engineering, we do significantly more testing for a calculator,” she adds. (Yes, a calculator.) “It’s not an accident they are using terminology that carries a lot of credibility.”

One early customer that triggered significant internal discussion was Luka, a San Francisco–based company designing an AI-powered virtual companion app called Replika. The company had partnered with OpenAI for the GPT-3 API launch to improve the conversational fluidity of its product. Despite Replika’s companion bot branding, OpenAI quickly discovered that the app’s users often engaged in sexually explicit conversations. OpenAI employees debated whether this fell within the line of acceptability. In the end, the company decided to ban Replika from using its model. In addition to concerns about sexual content, the GPT-3-powered app sometimes generated emotionally manipulative responses that were convincing users that their Replika, much like a human, could get hurt if they didn’t check in regularly. OpenAI staff also grew increasingly uncomfortable that they could read the conversations.

In another instance, Brockman gave API access to a Utah-based startup where his brother worked called Latitude, building AI-powered virtual worlds. Latitude had already been using an earlier OpenAI model to power a choose-your-own adventure game, inspired by Dungeons and Dragons, which allowed users to choose any action they wanted by typing it into a dialog box. With the new API, Latitude upgraded its game to run on GPT-3. Several months later, some users began using it to generate text-based scenarios involving sexual abuse of children. After discovering this issue through OpenAI’s relatively new monitoring system, Ari Herbert-Voss raised it to the rest of his team.

“I found some stuff last night. There’s a lot of sexual content being generated,” Herbert-Voss said in a meeting.

At first people chuckled. “Okay, that’s just how the internet works, isn’t it?” “No, this is CSAM-level stuff,” he said, using the acronym for child sexual abuse material.

Now there was panic. “Oh shit, how do we stop this?” The incident led to a long back-and-forth between OpenAI and Latitude about how to handle the situation. Brockman worried about taking punitive measures that could heavily affect his brother’s company. In the end, Latitude hastily implemented a filter to block text-based child sex abuse content. OpenAI released a public statement to distance itself from the lack of content moderation and to subtly place the blame on Latitude. To some OpenAI employees, the blame was clearly on the company’s own technology and lack of process, and the incident weighed heavily. Latitude had already banned some users for generating text-based sexual content involving children with OpenAI’s previous model; that it would happen again and at scale with GPT-3 was foreseeable. “It was sad to me that we deployed this API with our mission of benefiting humanity, and everyone had such positive impressions about how we had users saving time on customer service or whatever,” one former OpenAI employee says, “but in reality, a lot of our traffic was going to AI Dungeon child sexual content and a creepy AI girlfriend product.”

---

Back within the Research division, the code-generation team was making the fastest progress.

The Divorce had resulted in the duplicate code-gen efforts becoming one, and Wojciech Zaremba had become its main point person. A Polish computer scientist who had grown up winning math, coding, chemistry, and physics competitions, Zaremba was known for his incredible technical aptitude and attentiveness to team-building as well as his passion for the healing power of friendship, the wilderness, sex, and drugs. He sometimes loudly regaled people around the office about upcoming plans for weeks-long retreats. “We are going to hike for eight miles. And then we are going to have sex. And then we are going to hike for another eight miles,” he once boasted to another OpenAI leader, as others within earshot listened awkwardly.

Still in the depths of the pandemic, Zaremba had asked his team of initially roughly ten researchers to come into the office even as other teams stayed remote, believing that in-person work was necessary to crack the challenge of the model’s development. After seeing the code-generation capabilities of GPT-3, Murati had floated the idea with Microsoft CTO Kevin Scott of turning those skills into an AI coding-assistant product. In 2018, Microsoft had acquired GitHub, the most popular platform for software developers to store and share their code. OpenAI had already been scraping GitHub of its own volition. Microsoft executives directed GitHub to hand OpenAI all of the code in its public repositories to save all of the trouble. As the code-gen team got better and better results, Altman made regular appearances at meetings, encouraging the researchers to keep going and deliver their best to Microsoft. By spring, after exciting the tech giant’s executives with several demos, it was clear that the model would be OpenAI’s second commercial project, following GPT-3.

Some of Scott’s own staff had reservations about the model’s development. While giving OpenAI free access to the code in GitHub’s public repositories was not illegal, it still felt like a violation of the user community’s trust. Much of that code had been shared in the spirit of fostering open-source software development, which was grounded in helping independent developers and small startups have a chance at being competitive, not in helping the big players entrench their monopoly. In a memo, they laid out key critiques to the GitHub project, suggesting Scott reconsider the premise of hoovering up developer data published under a Creative Commons license without consent or compensation, a former staffer remembers. Microsoft, the memo said, should consider canceling the product, or at the very least take a percentage of the product’s profits and give it back to the open-source community. While Scott was receptive, creating the tool and being first to market was his central focus, the staffer says. In the end, Microsoft donated some money to an existing program for supporting open-source developers called GitHub Sponsors and left the product vision unaltered.

Within OpenAI, employees justified the project through different arguments. Some agreed with Altman that working on a product to make Microsoft happy and thus continue to secure money and compute resources seemed essential to fulfilling OpenAI’s mission. To other employees, a code-generation model seemed highly economically valuable, aligning well with the company’s definition of AGI as “highly autonomous systems that outperform humans at most economically valuable work.” In this respect, Altman was also keen on code generation as a way to accelerate OpenAI’s own economically valuable work, a belief that would later feed into the start of an effort called AI Scientist, about advancing OpenAI’s models to autonomously perform AI research.

To many researchers, there was also a third argument: The effort was an important stepping stone to developing the next GPT model, which they hoped would be able to perform some degree of reasoning, still a key missing ingredient. In the broader field, as debates raged between the Hinton and Marcus camps over whether deep learning alone could produce a model with such a capability, OpenAI researchers hypothesized that if it could, training a model on code would likely help. Coding data was one of the most obvious and largest sources of data that encoded structured patterns of logic. The argument flowed back to the same origin: If code generation helped advance AI models toward AGI, what better way to achieve OpenAI’s mission?

Despite the billions of lines of code available from GitHub, the volume of data still paled in comparison to what had been used to train GPT-3. The team believed the code-generation model would need to be trained on both GitHub and the GPT-3 dataset to get the best results. They also found new sources of data, including scrapes of Stack Overflow, an online Quora-like forum for developers to post coding questions to a community; coding instruction manuals; and programming textbooks in different languages. The question was how best to combine all this data: Was it better to fine-tune the existing GPT-3 model on GitHub and other material, or better to train a fresh model from scratch with everything new mixed in with the old? The team stuck to fine-tuning to save money; the experiments they were running were already costing as much as a hundred thousand dollars apiece, based on Microsoft’s pricing for its cloud services. Training a new model could cost tens of millions of dollars. When it later came to developing what OpenAI would call GPT-3.5, it switched the approach, mixing the data together at the outset. By OpenAI’s internal measures, the results did indeed suggest that the addition of coding data improved the model’s ability to perform logic-based tasks, not just in code, but in English—a phenomenon known as transfer learning.

In the summer of 2021, OpenAI delivered an initial rough version of its code-gen model, called Codex, to GitHub and Microsoft. The model was too big and too slow, making it both costly to serve at scale and a bad user experience. Tensions emerged as all three organizations dealt with the growing pains of their first collaboration. Confusion abounded over whose responsibility it was— OpenAI’s or GitHub’s—to optimize the model into a deployable product. There was also a lack of clarity among OpenAI employees around how much IP they should be sharing with their GitHub counterparts, while GitHub employees struggled with how much to trust OpenAI. Disagreements compounded as the companies clashed over how and when to release the product and who would get the credit.

Murati eventually brokered a compromise—a skill that would gain her increasing respect among people who worked with her across companies. Microsoft would get its moment by releasing its consumer-facing product, GitHub Copilot, in June 2021. OpenAI would then release its version of Codex directly in the company’s API in August.

The arrangement would give Microsoft a new user base and a modest financial bump: In two years, GitHub Copilot would grow to one million paid subscribers, bringing in over $100 million in annual recurring revenue. But for OpenAI, the deal deepened an emerging sense at the company that it would be better served to work on its own consumer products. OpenAI’s researchers had worked hard on the model and were ceding all of the brand recognition to GitHub and Microsoft; watching those two companies enjoy the credit for OpenAI’s work in public was a tough pill to swallow. Microsoft was also a challenging partner; many felt it had far too much bureaucracy and required too much hand-holding to make the most of OpenAI’s models. By relying on the tech giant to deliver its technologies to the public, OpenAI was also losing visibility into and data from its users and, most importantly, control over its vision.

---

As OpenAI concentrated its bets, Altman was applying the same strategy to his other projects and investments. Over the years, as he’d shifted toward more hard-tech innovation, he had developed a belief in betting big and long on the most important projects.

“If you could wave a wand, change anything about the tech, startup, entrepreneurship ecosystem, what would you change?” his brother Jack had asked him at an event as he’d stepped down from YC.

“It would be to get everyone in the ecosystem to take a much longer time horizon,” Altman had said. “This world where people start a company and plan to run it for four or five years, join a company and only plan to stay for one or two—that’s not how important shit gets done.”

Along those lines, 2021 seemed to mark a major shift in Altman’s personal investment strategy away from taking a large number of small bets toward taking a small number of really large ones. That year a startup he’d cofounded in 2019 called Tools for Humanity that had remained largely quiet saw an influx of funding and media coverage and a concerted ramp-up in its operations, as Altman directed more external attention to the company. The venture was a dedicated effort to develop a working mechanism for universal basic income, or UBI, a popular Silicon Valley idea to give everyone a regular minimum distribution of income. Altman often passionately discussed UBI as the possible antidote to a future world where AI could create mass economic fallout. At YC he had started the largest pilot in the US to study the concept, spinning out a nonprofit in the process called OpenResearch that administered the program.

Over three years, OpenResearch gave a $1,000 monthly stipend to a randomly selected group of one thousand out of three thousand low-income people, with the rest getting fifty dollars a month as a control. In July 2024, OpenResearch would release its findings, showing that the unconditional cash helped people meet their basic needs, assist others, and have more economic leeway.

Tools for Humanity’s main product, Worldcoin, was a self-described “collectively owned” cryptocurrency that would allow everyone to eventually get a share of its value. As part of the scheme, the company was developing a dramatic-looking chrome-colored orb—roughly the size of a bowling ball and partly a reflection of Altman’s design tastes—to scan people’s irises and verify their identity before giving them their cut. The iris scanning would be a necessity, the founders argued, once AI also made it increasingly hard to decipher fake media from reality. An extensive investigation from Eileen Guo and Adi Renaldi at MIT Technology Review would later find that these iris-scanning efforts were mired in data privacy infringements, deceptive marketing practices, and potential legal violations. In July 2023, Worldcoin would officially launch to massive controversy, as people began lining up by the thousands, particularly in Global South countries, to give over their biometric data with little understanding of what they were doing it for other than the vague promise of free money.

Also in 2021, Altman made his two largest ever investments: $180 million into an antiaging company called Retro Biosciences, working to extend human lifespans through cellular rejuvenation, and $375 million into Helion Energy, working to commercialize nuclear fusion. “I basically just took all my liquid net worth and put it into these two companies,” Altman told MIT Technology Review’s Antonio Regalado. Altman described both technologies in language that mirrored OpenAI’s research road map—they seemed impossible currently but, if scaled up aggressively, could be around the corner.

The Retro Biosciences bet reflected Altman’s fixation on longevity. He was an avid follower of “young blood” research—a line of scientific inquiry that studied how to reverse aging with transfusions of healthier, younger blood. Notably, it was an area in which Thiel was also interested, spawning a plethora of articles and memes about his desire to inject himself with the blood of teenagers. While at YC, Altman had also signed up with a $10,000 deposit to be on the wait list of a controversial startup called Nectome, which had been in one of the accelerator’s batches. Ripped straight out of science fiction, Nectome was pitching a service that would cryogenically freeze customers’ brains to one day —potentially hundreds of years into the future—upload to a computer after scientists had cracked the technology to do so. The catch was that Nectome needed the person’s brain to be fresh for the preservation to work. To Antonio Regalado, cofounder Robert McIntyre called his product “100 percent fatal.”

Helion reflected Altman’s obsession with finding ways to generate clean, cheap, and abundant energy. He often remarked that the cost and availability of energy was highly correlated with quality of life and with economic growth. But without carbon-free alternatives, rising energy consumption would “destroy the planet,” he said. In 2023, he would describe Helion as “more than an investment” and “the other thing besides OpenAI I spend a lot of time on.” Microsoft would subsequently sign a deal to purchase power from Helion’s first plant, after the tech giant had made its third investment, worth $10 billion, into OpenAI. To the astonishment and skepticism of energy experts, Helion would commit to having its plant ready by 2028.

The year 2021 was also when Altman brought his predilection for investing to OpenAI. That May, he launched the OpenAI Startup Fund, a $100 million investment pool for supporting early stage companies with, as he described, “big ideas about how to use AI to transform the world.” Microsoft once again became an investor in the fund. To some observers, the fund’s creation was a strange decision. OpenAI was barely generating revenue and already capital intensive enough as it was; why raise yet more money for separate companies to use? Others felt it was Altman’s way of remaking YC’s powerful network effects around OpenAI. Still others viewed it simply as Altman’s force of habit. “This is Sam’s way of moving through the world,” says a person who worked with him. “Dealmaking.”

Altman liked to say that he had taken no equity in OpenAI to avoid corrupting the quest of safe AGI with his own desires for profit. He made only a yearly salary of $65,000 and accumulated his wealth through other ventures. The sentiment had a nice ring to it—and echoed his original rhetoric around why OpenAI started as a nonprofit. It was also a statement, like the nonprofit status of the organization, that by 2021 no longer reflected the full truth. Altman had a significant stake in YC, and YC, through its $10 million investment in OpenAI, could receive up to a $1 billion return. As OpenAI continued to commercialize, many YC startups and many of his other investments would also become customers or commercial partners of the company. The Wall Street Journal would subsequently calculate Altman’s net worth in June 2024 across all his holdings to be at least $2.8 billion. With the OpenAI Startup Fund, Altman added yet another complication to his altruistic narrative—one that would eventually play its own small part in his fleeting ouster.