# Dreams of Modernity LLMS index: [llms.txt](/en/llms.txt) --- In their book Power and Progress, MIT economists and Nobel laureates Daron Acemoglu and Simon Johnson argue that every technology revolution must begin with a rallying ambition. It is the promise of a technology benefiting everyone that puts in motion the long journey of amassing enough talent and resources to turn it into a reality. After analyzing one thousand years of technology history, the authors conclude that technologies are not inevitable. The ability to advance them is driven by a collective belief that they are worth advancing. The irony is that for this very reason, new technologies rarely default to bringing widespread prosperity, the authors continue. Those who successfully rally for a technology’s creation are those who have the power and resources to do the rallying. As they turn their ideas into reality, the vision they impose—of what the technology is and whom it can benefit—is thus the vision of a narrow elite, imbued with all their blind spots and self-serving philosophies. Only through cataclysmic shifts in society or powerful organized resistance can a technology transform from enriching the few to lifting the many. The authors point to the invention of a new cotton gin in the 1790s as an example. The machine turned the American South into the largest global exporter of cotton, boosted the country’s top-line economic growth, and generated windfall returns for many landowners and cotton-related businesses. But it only served to intensify slavery and its horrific system of dehumanization and labor exploitation until its abolition seven decades later. With the surge in cotton production, enslaved Black people were forced to work longer hours and physically coerced by even harsher means to squeeze out every ounce of their labor. All the while, those who profited from the cotton gin painted the invention as one that made the enslaved happier. “I say it boldly, there is not a happier, more contented race upon the face of the earth,” said one South Carolina congressman. These two features of technology revolutions—their promise to deliver progress and their tendency instead to reverse it for people out of power, especially the most vulnerable—are perhaps truer than ever for the moment we now find ourselves in with artificial intelligence. Since its conception, the development and use of AI has been propelled by tantalizing dreams of modernity and shaped by a narrow elite with the money and influence to bring forth their conception of the technology. That conception is what has led to the exploding social, labor, and environmental costs that are playing out around the world today, particularly, as we’ll see, in many Global South countries, for which the consequences of their dispossession by historical empires still linger in delayed economic development and weaker political institutions. And yet, just like the South Carolina congressman, Silicon Valley has painted the experiences of those being exploited and harmed by the technology as happier because of it. --- The promise propelling AI development is encoded in the technology’s very name. In 1956, six years after Turing’s paper began with the line “Can machines think?” twenty scientists, all white men, gathered at Dartmouth College to form a new discipline in the study of this question. They came from fields such as mathematics, cryptography, and cognitive science and needed a new name to unify them. John McCarthy, the Dartmouth professor who convened the workshop, initially used the term automata studies to describe the pursuit of machines capable of automatic behavior. When the research didn’t attract much attention, he cast about for a more evocative phrase. He settled on the term artificial intelligence. The name artificial intelligence was thus a marketing tool from the very beginning, the promise of what the technology could bring embedded within it. Intelligence sounds inherently good and desirable, sophisticated and impressive; something that society would certainly want more of; something that should deliver universal benefit. The name change did the trick. The two words immediately garnered more interest—not just from funders but also scientists, eager to be part of a budding field with such colossal ambitions. Cade Metz, a longtime chronicler of AI, calls this rebranding the original sin of the field: So much of the hype and peril that now surround the technology flow from McCarthy’s fateful decision to hitch it to this alluring yet elusive concept of “intelligence.” The term lends itself to casual anthropomorphizing and breathless exaggerations about the technology’s capabilities. In 1958, two years after the field’s founding, Frank Rosenblatt, a Cornell University professor, demonstrated the Perceptron, a system that could perform basic pattern matching to tell apart cards based on whether they had a small square printed on their left or their right. Over his main collaborator’s objections, Rosenblatt advertised his system as something akin to the human brain. He even ventured to say that it would one day be able to reproduce and begin to have sentience. The next morning, The New York Times announced that the Perceptron would in the future “be able to walk, talk, see, write, reproduce itself and be conscious of its existence.” That tradition of anthropomorphizing continues to this day, aided by Hollywood tales combining the idea of “AI” with age-old depictions of human-made creations suddenly waking up. AI developers speak often about how their software “learns,” “reads,” or “creates” just like humans. Not only has this fed into a sense that current AI technologies are far more capable than they are, it has become a rhetorical tool for companies to avoid legal responsibility. Several artists and writers have sued AI developers for violating copyright laws by using their creative work—without their consent and without compensating them—to train AI systems. Developers have argued that doing so falls under fair use because it is no different from a human being “inspired” by others’ work. The omnipresent AI-to-human analogies have also fueled the sense that such software could become so capable that it surpasses us and comes to threaten our very existence. The fear of superintelligence is predicated on the idea that AI could somehow rise above us in the special quality that has made humans the planet’s superior species for tens of thousands of years. Artificial intelligence as a name also forged the field’s own conceptions about what it was actually doing. Before, scientists were merely building machines to automate calculations, not unlike the large hulking apparatus, as portrayed in The Imitation Game, that Turing made to crack the Nazi Enigma code during World War II. Now, scientists were re-creating intelligence—an idea that would define the field’s measures of progress and would decades later birth OpenAI’s own ambitions. But the central problem is that there is no scientifically agreed-upon definition of intelligence. Throughout history, neuroscientists, biologists, and psychologists have all come up with varying explanations for what it is and why it seems that humans have more of it than any other species. Perhaps it’s the size of our human brains, our ability to reason through complex problems, or our capacity to create a mental model of other people’s beliefs. Myriad tests have been developed over the centuries to measure intelligence against these definitions, many of which have subsequently been debunked and fallen out of favor due to their unsavory histories. In the early 1800s, American craniologist Samuel Morton quite literally measured the size of human skulls in an attempt to justify the racist belief that white people, whose skulls he found were on average larger, had superior intelligence to Black people. Later generations of scientists found that Morton had fudged his numbers to fit his preconceived beliefs, and his data showed no significant differences between races. IQ tests similarly began as a means to weed out the “feebleminded” in society and to justify eugenics policies through scientific “objectivity.” More recent standardized tests, such as the SAT, have shown high sensitivity to a test taker’s socioeconomic background, suggesting that they may measure access to resources and education rather than some inherent ability. In a document first published in 2004 titled “What Is Artificial Intelligence?,” McCarthy admitted that the lack of consensus around natural intelligence was inherently confusing for a field trying to re-create it. A 2007 revision of his write-up presents a long and winding Q&A, meant to address basic questions for a lay audience. It begins: Q. What is artificial intelligence? A. It is the science and engineering of making intelligent machines, especially intelligent computer programs…. Q. Yes, but what is intelligence? A. Intelligence is the computational part of the ability to achieve goals in the world. Varying kinds and degrees of intelligence occur in people, many animals and some machines. Q. Isn’t there a solid definition of intelligence that doesn’t depend on relating it to human intelligence? A. Not yet. The problem is that we cannot yet characterize in general what kinds of computational procedures we want to call intelligent. As a result, the field of AI has gravitated toward measuring its progress against human capabilities. Human skills and aptitudes have become the blueprint for organizing research. Computer vision seeks to re-create our sight; natural language processing and generation, our ability to read and write; speech recognition and synthesis, our ability to hear and speak; and image and video generation, our creativity and imagination. As software for each of these capabilities has advanced, researchers have subsequently sought to combine them into so-called multimodal systems—systems that can “see” and “speak,” “hear” and “read.” That the technology is now threatening to replace large swaths of human workers is not by accident but by design. Still, the quest for artificial intelligence remains unmoored. With every new milestone in AI research, fierce debates follow about whether it represents the re-creation of true intelligence or a pale imitation. To distinguish between the two, artificial general intelligence has become the new term of art to refer to the real deal. This latest rebranding hasn’t changed the fact that there is not yet a clear way to mark progress or determine when the field will have succeeded. It’s a common saying among researchers that what is considered AI today will no longer be AI tomorrow. The Turing test didn’t last long as an indicator of AI after it was quickly surpassed, and scientists felt they hadn’t actually solved their objective. There was also a time when scientists believed that a computer beating humans in chess or Go would be a conclusive measure of success. Now DeepMind’s AlphaGo is seen as a compelling demonstration of what software can be made to do but once again not yet a conclusion to the field’s ambitions. Through decades of research, the definition of AI has changed as benchmarks have evolved, been rewritten, and been discarded. The goalposts for AI development are forever shifting and, as the research director at Data & Society Jenna Burrell once described it, an “ever-receding horizon of the future.” The technology’s advancement is headed toward an unknown objective, with no foreseeable end in sight. To justify the elongating timeline and the ever-expanding costs of pursuing the ambition for AI, the promises we’re told about it have grown more grandiose than ever before: AI was once a scientific fascination, a technology with some potential commercial utility. Now, AI is the harbinger of the fourth industrial revolution. The keystone of the modern superpower. AGI, if ever reached, will solve climate change, enable affordable health care, provide equitable education. OpenAI is the poster child for this line of thought. It cannot say how the technology will deliver on these promises—only that the staggering price society needs to pay for what it is developing will someday be worth it. What’s left unsaid is that in a vacuum of agreed-upon meaning, “artificial intelligence” or “artificial general intelligence” can be whatever OpenAI wants. --- The history of AI shows us that AI development has always been shaped by a powerful elite. It’s not a coincidence that AI today has become synonymous with colossal, resource-hungry models that only a tiny handful of companies are equipped to develop, and that desire us to make their products into the foundations for everything. Even in the early days, before commercial interests made the politics of the AI revolution far more visible, the field’s scientific explorations lurched and swerved amid heated clashes over funding and influence. Following the Dartmouth gathering, two camps emerged with competing theories about how to advance the field. The first camp, known as the symbolists, believed that intelligence comes from knowing. Humans know more than animals and can use that knowledge to understand and act on the world. Achieving AI must then involve encoding symbolic representations of the world’s knowledge into machines, creating so-called expert systems. The second camp, called the connectionists, believed that intelligence comes from learning. Humans have a greater capacity to learn than animals and can use that ability to acquire and advance different skills. Developing AI should focus instead on creating so-called machine learning systems, such as by mimicking the ways our brains process signals and information. This hypothesis would eventually lead to the popularity of neural networks, data-processing software loosely designed to mirror the brain’s interlocking connections, now the basis of modern AI, including all generative AI systems. Over subsequent decades the two camps vied for a limited pool of funding and control over the popular imagination of what AI could be. At the time, those fights played out in universities and academic journals among scientists squabbling over government and foundation money; on occasion, their debates would burst forth in media coverage, shaping the public’s understanding of their pursuits. At the helm of the connectionists was Rosenblatt and his Perceptron, an early proof of concept for a machine learning system. Rosenblatt never gave the system explicit instructions, designing it instead to compute its own rules for how to tell different cards apart after seeing numerous examples. At the helm of the symbolists was Rosenblatt’s nemesis, MIT professor Marvin Minsky. A co-organizer of the Dartmouth workshop, Minsky had himself dabbled in connectionist thinking before souring on the idea. He did not switch his loyalties quietly. He found frequent opportunities to grandstand and ridicule his connectionist colleagues who competed with him for the same grants, sparing not even early-career researchers. In 1969, he coauthored a book, called Perceptrons, so critical of connectionism that it is credited, along with the middling progress of neural networks, for killing off nearly all funding to that vein of research for more than fifteen years. In their battle for influence, Minsky had an unlikely ally: his MIT colleague Joseph Weizenbaum. In 1958, the same year that Rosenblatt’s Perceptron captivated The New York Times, Weizenbaum invented an AI system called ELIZA that similarly drew headlines. More than six decades before ChatGPT, ELIZA was a chatbot that you could converse with through an electric typewriter hooked up to a hulking mainframe that spanned an entire room. Under the hood, though, where ChatGPT would emerge from connectionist tradition, ELIZA was a symbolic system, using simple rules to seem attentive and responsive to a user. One of the most popular instantiations of ELIZA was designed after a method of talk therapy that probed a patient by reflecting back what they said and gently asking for more information. In a paper Weizenbaum later published on the inner workings of ELIZA, he provided the following dialogue as an example: USER: Men are all alike. ELIZA: IN WHAT WAY USER: They’re always bugging us about something or other. ELIZA: CAN YOU THINK OF A SPECIFIC EXAMPLE USER: Well, my boyfriend made me come here. ELIZA: YOUR BOYFRIEND MADE YOU COME HERE USER: He says I’m depressed much of the time. ELIZA: I AM SORRY TO HEAR YOU ARE DEPRESSED USER: It’s true. I am unhappy. ELIZA: DO YOU THINK COMING HERE WILL HELP YOU NOT TO BE UNHAPPY Weizenbaum designed the system as an experiment to see how easily humans might fall for an illusion of intelligence. ELIZA’s namesake was Eliza Doolittle, a fictional working-class flower girl portrayed by Audrey Hepburn in the 1956 film My Fair Lady, who learns to pass as a duchess in high society after a wealthy man teaches her to change her diction and manners. ELIZA’s subsequent success in fooling people into believing it to be intelligent alarmed Weizenbaum. In fact, the demonstration felt so convincing to some that psychiatrists began to speak of automated psychotherapy as just around the corner, and merely a few years after the founding of the AI field, computer scientists were already prematurely concluding that natural language understanding in computers was a solved problem. (Decades later, whether or not it’s even been solved today is still an open debate.) Weizenbaum would spend much of the rest of his career attempting to deflate the hype of his creation and campaigning against the fundamental presumption behind the pursuit of AI. ELIZA, he wrote, was nothing but a simple procedural program, coded by him to identify keywords in a user’s input and perform basic transformations to construct responses. My boyfriend became your boyfriend; I’m depressed became you are depressed. There was really nothing much intelligent about it. He later published a tome called Computer Power and Human Reason in the decade following Minsky’s Perceptrons that argued that humans and machines are different and the AI field’s attempt to blur that distinction would lead to profound societal consequences. It would, for example, allow people in power—whether CEOs or politicians—to execute their will through machines while absolving themselves of moral responsibility. Despite Weizenbaum’s best efforts, ELIZA’s arresting demonstration of a symbolic system inadvertently bolstered Minsky’s campaign to elevate symbolism over connectionism. Over the next few decades, through the nineties, expert systems became the hottest area of AI research and commercialization. The prevailing thinking spawned projects like Cyc, an effort to develop a common-sense system by programming it with one hundred million rules about daily life. But during various stretches, advancements in symbolic AI systems would sputter and slow as efforts to scale them hit up against the challenges of manually encoding all the rules. How does one encode all the subtleties of the English language with its slang, sarcasm, figures of speech, and grammar exceptions? Each time the roadblocks mounted, funders would lose interest, plunging the field into a state of existential crisis known as an “AI winter.” --- At this point in the story, the history of AI is often told as the triumph of scientific merit over politics. Minsky may have used his stature and platform to quash connectionism, but the strengths of the idea itself eventually allowed it to rise to the top and take its rightful place as the bedrock of the modern AI revolution. During the years that symbolism reigned, a small band of connectionists held fast to Rosenblatt’s pursuit of machine learning systems and continued to advance it. They included Sutskever’s PhD adviser, Geoffrey Hinton, who, as a professor at Carnegie Mellon University in the 1980s, made a key improvement to early neural networks along with colleagues from the University of California, San Diego. By then, connectionists had hypothesized that their neural networks were failing because they were too simple; they contained only a single layer of networked “neurons,” or data-processing nodes. To better mimic the human brain, the software likely needed multiple stacked and connected layers to form a so-called deep neural network. Hinton and his coauthors made this change possible by using an algorithm known as backpropagation, which allows deep neural networks to exchange and process information across their layers. In another instance of rebranding, Hinton later cleverly gave this multilayer processing the name deep learning, a shorthand for using deep neural networks to perform machine learning. But deep neural networks—today, simply called “neural networks”—came several decades too early. To really shine, they needed more processing power than computers in the 1980s had available, and more examples, or data, than could be cheaply compiled from the analog world. At their core, neural networks are calculators of statistics that identify patterns in old data—text, pictures, or videos—and apply them to new data. Today, if an AI developer wants to build an AI model for detecting people in images, they might feed a neural network hundreds of thousands of images, each with a label—1 for “has a person,” 0 for “does not have a person.” (When you’re solving Google’s captchas by clicking all the images with stop signs, you are in fact training the company’s neural networks.) Using statistics, the neural network then teases out the pixel patterns within the images that are associated with whether a person is present. This is what’s known as training an AI model. Once the model is done training, the developer can run it on new data—known as inferencing—to determine whether it fits the pattern. Is this image a 1 or a 0? Does it have a person or not? Generally speaking, neural networks need to be trained on a certain threshold of high-quality data with a certain threshold of compute to calculate these patterns and produce a performant AI model. Hinton and his coauthors were ahead of their time. But in the late aughts, once computers had advanced and the internet had matured, creating new repositories of digital data, neural networks finally had the right conditions to flourish. Shortly thereafter, Google acquired Hinton’s DNNresearch—“DNN” for deep neural networks—igniting a new race to commercialize deep learning. In this telling of the story, the lesson to be learned is this: Science is a messy process, but ultimately the best ideas will rise despite even the loudest detractors. Implicit within the narrative is another message: Technology advances with the inevitable march of progress. But there is a different way to view this history. Connectionism rose to overshadow symbolism not just for its scientific merit. It also won over the backing of deep-pocketed funders due to key advantages that appealed to those funders’ business interests. The strength of symbolic AI is in the explicit encoding of information and their relationships into the system, allowing it to retrieve accurate answers and perform reasoning, a feature of human intelligence seen as critical to its replication. Think of IBM Watson, one of the most famous symbolic systems, which would dazzle on Jeopardy! in 2011. Its speedy delivery of game show–winning answers was based in its ability to trawl through vast stores of knowledge and accurately reproduce them. The weakness of symbolism, on the other hand, has been to its detriment: Time and again its commercialization has proven slow, expensive, and unpredictable. After debuting Watson on late-night TV, IBM discovered that getting the system to produce the kinds of results that customers would actually pay for, such as answering medical rather than trivia questions, could take years of up-front investment without clarity on when the company would see returns. IBM called it quits after burning more than $4 billion with no end in sight and sold Watson Health for a quarter of that amount in 2022. Neural networks, meanwhile, come with a different trade-off. For years the field has aggressively debated whether such connectionist software can do what the symbolic ones can: store information and reason. Regardless of the answer, it has become clear that if they can, they do so inefficiently. Only with extraordinary amounts of data and computational power have neural networks even begun to have the kinds of behaviors that may suggest the emergence of either property. That said, one area where deep learning models really shine is how easy it is to commercialize them. You do not need perfectly accurate systems with reasoning capabilities to turn a handsome profit. Strong statistical pattern-matching and prediction go a long way in solving financially lucrative problems. The path to reaping a return, despite similarly expensive upfront investment, is also short and predictable, well suited to corporate planning cycles and the pace of quarterly earnings. Even better that such models can be spun up for a range of contexts without specialized domain knowledge, fitting for a tech giant’s expansive ambitions. Not to mention that deep learning affords the greatest competitive advantage to players with the most data. Tech giants were already seeing early evidence of the commercial potential of neural networks before the auction of DNNresearch. In 2009, Hinton’s grad students showed that such software was decent at speech recognition. IBM, Microsoft, and Google all jumped on the trend, but Google was the fastest to reach commercialization. In 2012, Google put neural networks into production, greatly improving Android’s speech-recognition capabilities, just as more of Hinton’s grad students, this time Sutskever and Alex Krizhevsky, achieved their breakthrough results at ImageNet, demonstrating that neural networks were also very good at image recognition. The successful Android deployment primed Google’s willingness to spend big on the three academics, marking the start of the tech industry’s full embrace of deep learning. Hinton, Sutskever, and Krizhevsky subsequently continued to evangelize neural networks within Google. They found momentum applying their software to a wide array of other commercially relevant technical problems. They worked in parallel to develop deep learning models for machine translation, upgrading Google Translate; for text prediction, adding the suggested completions feature to Gmail; and for an ambitious new self-driving-car project called Waymo. As Google’s AI operations continued to grow, neural networks also produced crucial improvements to the company’s cash cow, search. The software could better match user queries to relevant web pages, delivering users higher-quality search results and, importantly, targeting them with more relevant ads. The more Google profited and the more billions it poured into deep learning, the more the rest of the industry followed. Companies quickly came to dominate over governments and foundations as the biggest funders of AI research and were soon setting the research agenda based on advancements that could also produce short-term profitability. --- The entwining of deep learning with commercial interests simultaneously transformed the tech industry and the face of AI development. To the public, generative AI would erupt seemingly out of nowhere in late 2022 with OpenAI’s launch of ChatGPT. But from 2012 to 2022, beginning with the ImageNet breakthrough, it was these shifts during the first major era of AI commercialization that laid the groundwork for many characteristics of the generative AI revolution today. For industry, deep learning fueled the improvement and emergence of new products and services, from faster access to information to more efficient e-commerce to the rise of the sharing economy. For deep learning, industry drove new technical breakthroughs in neural networks and computer chips that enabled the development of larger and more powerful AI models. But alongside these impressive advances, deep learning’s supercharging of Silicon Valley would also aggressively expand its business model, for which Harvard professor Shoshana Zuboff would coin a term in 2014: surveillance capitalism. Where industrial capitalism derived value from producing material goods that people wanted to buy, surveillance capitalism, Zuboff argued, treated its users as the product. Tech giants sitting atop vast amounts of user data could easily pump those troves into neural networks to more precisely profile users than ever before and milk their engagement for ad revenue. To outcompete one another, they could simply collect even more of that data by recording increasingly exhaustive logs of every user’s clicks, scrolls, and likes, and encouraging them to supply increasingly personal digital artifacts, including their every email exchange, every photo of their kids, and every thought they had about social and political issues. At the same time, Silicon Valley’s supercharging of deep learning in its quest to expand and entrench global-scale monopolies also codified a culture among AI developers to view anything and everything as data to be captured and consumed by their technologies in a noble attempt to make them reflect as much of the world as possible. In 2023, a group of AI researchers, including Ria Kalluri at Stanford University, William Agnew from the University of Washington, and Abeba Birhane from the Mozilla Foundation, would analyze more than forty thousand computer-vision papers and patents, and note the pervasive use of abstract, detached language to sanitize and normalize the field’s reliance upon mass scraping and extraction. Detailed digital trails of people’s thoughts and ideas on social media were merely “text.” People and vehicles in pictures were merely “objects.” Surveillance was merely “detection.” That culture is now at the crux of a raging debate in generative AI over whether tech companies can scrape books and artwork wholesale to train their AI systems. To many AI developers who have long operated under this mindset, that question seems rather quaint; taking it seriously presents a direct obstacle to the moral pursuit of ever-more progress. Even as some of them have grown more aware of and concerned by the chasm between their perspective and the view of many authors and artists who stand in opposition, this way of thinking has been difficult to shake. In May 2023, shortly after a group of artists filed suit for the first time against several generative AI developers over the theft of their artwork, I went to an AI research conference in Rwanda as a reporter for The Wall Street Journal. As I walked the vaulted hall of the glistening dome-shaped convention center in the country’s capital, a senior researcher stopped me and asked me whether the WSJ on my name badge was a new startup or the media publication. When I clarified that I was a journalist and it indeed stood for The Wall Street Journal, another senior researcher chimed in. “I recognized it because of the WSJ dataset,” she said, referring to an early AI speech-recognition dataset of people reading excerpts from the newspaper. “I’ve worked with it many times.” I found myself entrapped in this very same thinking when I first began covering AI in 2018. After internalizing the community’s lingo to speak and relate with AI researchers, I marveled at the myriad ways that researchers mined for and produced datasets. In one example I thought was particularly clever, researchers used thousands of YouTube videos of the viral 2016 Mannequin Challenge, where people froze in place as cameras panned and zoomed around them, to train up AI models for processing three-dimensional scenes. In 2019, an NBC investigation from Olivia Solon knocked off my rose-colored glasses. Solon revealed that facial-recognition software had been trained on millions of people’s personal Flickr photos without their consent. What surprised me was not the findings—I had long known that Flickr was a favorite data source for AI researchers. What surprised me was how much I had come to view that as completely normal. With new awareness, I began to notice how the aggressive push to collect more training data was leading to pervasive surveillance not just in the digital world but the physical one as well. I noticed, too, how the gaze of that physical surveillance seemed to repeatedly fall on already vulnerable populations, including children or historically marginalized groups, even more so in developing countries. That year, I stumbled across a Massachusetts-based, Harvard-incubated startup selling AI-powered headbands that said it could measure a student’s brain wave activity to tell a teacher whether or not the child was focused. The startup was piloting them in elementary schools in Colombia and China, in exchange for the rights to use their students’ data to advance the company’s technology. “We have the first mover’s advantage,” a research scientist at the company had said at an education technology conference in 2017. “We’ll be able to build one of the largest brain wave databases in the world. All that data will help us improve our algorithms and therefore our products, creating a higher barrier to entry.” A few months after I came across the startup, a data privacy outcry in China from parents horrified at their kids being turned into guinea pigs forced the company to pivot to a different application of its technology. But the story left me with an uneasy feeling that the successful backlash was an anomaly, and the company’s original approach—to go to countries eager to embrace the promise of technology for finding data donors and product testers—was in fact a trend. As I recounted this worry to a colleague, she introduced me to a phrase that had already been coined for the phenomenon: “data colonialism.” I discovered the work of scholars Nick Couldry and Ulises A. Mejias, whose foundational text The Costs of Connection, published just that year, argued that Silicon Valley’s pervasive datafication of everything was leading to a return of disturbing historical patterns of conquest and extractivism.[*] The following year, a paper called “Decolonial AI” from Shakir Mohamed and William Isaac at DeepMind and Marie-Therese Png at the University of Oxford reinforced a suspicion I had begun to develop: The AI industry, in equal parts fueled by and fueling this datafication, was in turn accelerating that new colonialism further. Not long after, in 2021, I found the same dynamics of the AI education startup playing out in South Africa. Facial recognition companies from all over the world were jostling to get a foothold in the country to collect valuable face data, especially after the industry had received significant criticism about their products’ failures to accurately detect darker-skinned individuals. I met a local activist, Thami Nkosi, who was born and raised in one of the poorest neighborhoods in Johannesburg, which used to be a chemical waste dump for the mining industry. He showed me the thousands of cameras dotting the city’s sprawling streets and described to me the ways it was restricting the movements of Black people, already squeezed by the racial legacies of apartheid and in fear of being criminalized, simply for being Black in a white neighborhood. “They’re essentially monetizing public spaces and public life,” Nkosi said. With increasing clarity, I realized that the very revolution promising to bring everyone a better future was instead, for people on the margins of society, reviving the darkest remnants of the past. --- But even as Silicon Valley’s conception of AI revealed its challenges, the first era of AI commercialization also choked off alternatives. As companies pumped unprecedented sums into deep learning and connectionism, overshadowing all other sources of funding, they remade the landscape of research around their priorities. From 2013 to 2022, corporate investments in AI, such as mergers and acquisitions, shot up from $14.6 billion to $235 billion, peaking at $337.4 billion in 2021, according to the Stanford University AI Index. Those numbers don’t even include in-house company spending on research and development. In 2021, Alphabet and Meta spent $31.6 billion and $24.7 billion, respectively. By contrast, the US government allocated $1.5 billion in 2021 to nondefense AI development. The European Commission allocated €1 billion ($1.2 billion) the same year. Talent followed the money. Many professors re-formed their research around neural networks, drawn in by their strong results as well as greater access to corporate funding. Many college and graduate students did the same, guided by the job security of deep learning and the diminishing viable career paths in other methods. Companies also fostered various arrangements that deepened their integration with academia. In 2013, Hinton joined Google on the condition that he simultaneously keep his position at the University of Toronto. Facebook struck the same deal the following year with Yann LeCun, a former postdoc of Hinton’s and a professor at New York University. Both would later share the 2018 Turing Award, often called the “Nobel Prize of Computing,” with Yoshua Bengio, a professor at the Université de Montréal, for their foundational work in deep learning. The accolade would earn the trio the moniker “godfathers of AI.” Hinton would also go on to win an actual Nobel Prize in 2024 with another scientist. Following in Hinton’s and LeCun’s footsteps, many AI professors began to maintain dual affiliations with a company and university. At scale, the practice began to erode the boundaries of truly independent research. Increasingly, more researchers also left academia altogether. From 2006 to 2020, the exodus to industry among AI research faculty increased eightfold; from 2004 to 2020, AI PhD graduates heading to corporations jumped from 21 percent to 70 percent, according to a 2023 study in Science from MIT researchers. Many were initially whisked away by the astronomical compensation, which for seasoned researchers could reach $1 million a year. In 2015, Uber infamously poached forty out of one hundred AI researchers from a single lab at Carnegie Mellon University after setting up shop in town and offering some scientists double their university salaries. Over time, another reason fed into the attrition: the growing costliness of deep learning research. Universities could no longer afford the computer chips or the electricity needed to work in the hottest areas of AI development. As such, the same 2023 Science study found that in just three years, from 2017 to 2020, industry-affiliated models grew from 62 percent to a whopping 91 percent of the world’s best-performing AI models. Midway through the first decade of AI commercialization, most top-level AI research was now happening within or in academic labs connected to tech companies. In another study, Kalluri, Agnew, Birhane, and other colleagues found that 55 percent of the most influential AI research papers had at least one industry coauthor in 2018 and 2019. This was compared with 24 percent a decade earlier. The research had also consolidated heavily within just a few corporations. Over the same decade, tech giants such as Microsoft and Google more than tripled their share of corporate-affiliated papers, to 66 percent. Ironically, this was precisely the reason Musk and Altman said they wanted to start OpenAI. The tech industry’s profit motive had become the overwhelming force driving AI development. --- The impact of this consolidation of funding and talent in the first era significantly narrowed the diversity of ideas in AI research. Deep learning continued to reign supreme not just for its scientific merit but also because very little investment went into exploring and advancing other paradigms. Indeed, while neural networks are remarkable inventions with myriad exciting uses, their weaknesses—namely, their hotly contested and inefficient ways of storing accurate information and reasoning—have endured as companies have deployed them in an expanding list of contexts and applications. Neural networks have shown, for example, that they can be unreliable and unpredictable. As statistical pattern matchers, they sometimes home in on oddly specific patterns or completely incorrect ones. A deep learning model might recognize pedestrians only by the crosswalks underneath them and fail to register a person who is jaywalking. It might learn to associate a stop sign with being on the side of the road and miss the same sign extended on the side of a school bus or being held by a crossing guard. Neural networks are also highly sensitive to changes in their training data. Feed them a different set of pedestrian images, or a different set of stop sign images, and they will learn a whole new set of associations. But those changes are inscrutable. Pop open the hood of a deep learning model and inside are only highly abstracted daisy chains of numbers. This is what researchers mean when they call deep learning “a black box.” They cannot explain exactly how the model will behave, especially in strange edge-case scenarios, because the patterns that the model has computed are not legible to humans. This has led to dangerous outcomes. In March 2018, a self-driving Uber killed forty-nine-year-old Elaine Herzberg in Tempe, Arizona, in the first ever recorded incident of an autonomous vehicle causing a pedestrian fatality. Investigations found that the car’s deep learning model simply didn’t register Herzberg as a person. Experts concluded that it was because she was pushing a bicycle loaded with shopping bags across the road outside the designated crosswalk—the textbook definition of an edge-case scenario. Six years later, in April 2024, the National Highway Traffic Safety Administration found that Tesla’s Autopilot had been involved in more than two hundred crashes, including fourteen fatalities, in which the deep learning–based system failed to register and react to its surroundings and the driver failed to take over in time to override it. The fallible and inscrutable statistical patterns of neural networks can also turn into a security vulnerability. In 2019, white hat hackers tricked a Tesla in self-driving mode into veering into an incoming lane of traffic. All they did was place a series of tiny stickers on the road to fool the car’s deep learning model into misfiring and registering the wrong lane as the right one. Such vulnerabilities aren’t limited to physical systems or computer-vision models. Dawn Song, a professor at the University of California, Berkeley, who specializes in this area of research, known as “adversarial attacks,” showed that prompting a language model with the right message caused it to spit out sensitive data such as credit card numbers. For the same reasons, deep learning models have been plagued by discriminatory patterns that have sometimes stayed unnoticed for years. In 2019, researchers at the Georgia Institute of Technology found that the best models for detecting pedestrians were between 4 and 10 percent less accurate at detecting darker-skinned pedestrians. In 2024, researchers at Peking University and several other universities, including University College London, found that the most up-to-date models now had relatively matched performance for pedestrians with different skin colors but were more than 20 percent less accurate at detecting children than adults, because children had been poorly represented in the models’ training data. In fact, deep learning models are inherently prone to having discriminatory impacts because they pick up and amplify even the tiniest imbalances present in huge volumes of training data. It’s not just a problem when a demographic is poorly represented, but when it’s overrepresented as well. Early in her career, Deborah Raji, the Berkeley AI accountability researcher, who is Nigerian Canadian, interned at an AI startup called Clarifai that was building a deep learning model for detecting images that were “not safe for work.” The model disproportionately flagged people of color because, Raji discovered, they were more represented in the pornographic images that the company was using to teach the model what was problematic than the stock photos it was using to teach the model what was acceptable. It was a shocking realization that would push Raji, like Timnit Gebru, to severely question the dominant direction of AI development. In the late 2010s and early 2020s, as the challenges of deep learning grew more apparent, fierce debates reemerged over the best way to overcome them. Much like the clashes between symbolists and connectionists, different camps of researchers disagreed vehemently about whether there would ever be a way to rid neural networks of their limitations entirely, or whether there would only be Band-Aid fixes that merely mitigated them. Hinton and Sutskever continued to staunchly champion deep learning. Its flaws, they argued, are not inherent to the approach itself. Rather they are the artifacts of imperfect neural-network design as well as limited training data and compute. Some day with enough of both, fed into even better neural networks, deep learning models should be able to completely shed the aforementioned problems. “The human brain has about 100 trillion parameters, or synapses,” Hinton told me in 2020. “What we now call a really big model, like GPT-3, has 175 billion. It’s a thousand times smaller than the brain. “Deep learning is going to be able to do everything,” he said. Their modern-day nemesis was Gary Marcus, a professor emeritus of psychology and neural science at New York University, who would testify in Congress next to Sam Altman in May 2023. Four years earlier, Marcus coauthored a book called Rebooting AI, asserting that these issues were inherent to deep learning. Forever stuck in the realm of correlations, neural networks would never, with any amount of data or compute, be able to understand causal relationships—why things are the way they are—and thus perform causal reasoning. This critical part of human cognition is why humans need only learn the rules of the road in one city to be able to drive proficiently in many others, Marcus argued. Tesla’s Autopilot, by contrast, can log billions of miles of driving data and still crash when encountering unfamiliar scenarios or be fooled with a few strategically placed stickers. Marcus advocated instead for combining connectionism and symbolism, a strain of research known as neurosymbolic AI. Expert systems can be programmed to understand causal relationships and excel at reasoning, shoring up the shortcomings of deep learning. Deep learning can rapidly update the system with data or represent things that are difficult to codify in rules, plugging the gaps of expert systems. “We actually need both approaches,” Marcus told me. Despite the heated scientific conflict, however, the funding for AI development has continued to accelerate almost exclusively in the pure connectionist direction. Whether or not Marcus is right about the potential of neurosymbolic AI is beside the point; the bigger root issue has been the whittling down and weakening of a scientific environment for robustly exploring that possibility and other alternatives to deep learning. For Hinton, Sutskever, and Marcus, the tight relationship between corporate funding and AI development also affected their own careers. Not long after Google put its full weight behind Hinton and Sutskever, Marcus cofounded his own company, called Geometric Intelligence, in 2014. The startup was acquired by Uber two years later to build out an AI lab, but in 2020, after the ride-hailing firm’s IPO, it axed the division. Several original members of Geometric Intelligence subsequently joined OpenAI, where they switched from working on neurosymbolic advancements to deep learning. Over the years, Marcus would become one of the biggest critics of OpenAI, writing detailed takedowns of its research and jeering its missteps on social media. Employees created an emoji of him on the company Slack to lift up morale after his denouncements and to otherwise use as a punch line. In March 2022, Marcus wrote a piece for Nautilus titled “Deep Learning Is Hitting a Wall,” repeating his argument that OpenAI’s all-in approach to deep learning would lead it to fall short of true AI advancements. A month later, OpenAI released DALL-E 2 to immense fanfare, and Brockman cheekily tweeted a DALL-E 2–generated image using the prompt “deep learning hitting a wall.” The following day, Altman followed with another tweet: “Give me the confidence of a mediocre deep learning skeptic…” Many OpenAI employees relished the chance to finally get back at Marcus. --- Generative AI, the product of OpenAI’s vision, could not have emerged without the first era of AI commercialization. Generative AI models are deep learning models trained to generate reproductions of their data inputs. From old text, they learn to synthesize new text; from old images, they learn to synthesize new images. But to do so at high-enough fidelity to become humanlike, which OpenAI says is key in its quest for AGI, they are trained on more data and compute than have ever been used before. Generative AI is thus the maximalist form of deep learning. It is enabled by the cutting-edge software and hardware innovations refined during the first era. It feeds on the exploding troves of data amassed through surveillance capitalism. It is fueled and abetted by the culture of AI research that views consuming as much data as possible as its moral responsibility. Generative AI is now also pushing each of these phenomena even further. What made ChatGPT in November 2022 appear as such a stunning leapfrog ahead of anything that had come before was OpenAI’s vision to push deep learning to this unprecedented scale. With its sheer money and resources, OpenAI executed that vision aggressively, exploding its models so much that it would begin to hit the limits—in data, compute, and energy—of what the world has available. ChatGPT was also an innovation in marketing and packaging. It is not a coincidence that it shares the same presentation as a humanlike chatbot with one of the other most compelling demonstrations of AI in history, ELIZA. Human psychology naturally leads us to associate intelligence, even consciousness, with anything that appears to speak to us. And where ELIZA inadvertently came to dominate the early popular conception of AI, OpenAI has fanned the public association now between ChatGPT and AGI. In February 2023, at the height of ChatGPT hype, the company published a blog post under Altman’s name titled “Planning for AGI and Beyond.” The implication by proximity was that ChatGPT had taken a bold step toward artificial general intelligence. In reality, the analogies to intelligence are once again anthropomorphizing and exaggerating the capabilities of the technology. While Hinton and other deep learning absolutists predicted that the shortfalls of neural networks compared with humans would go away at sufficient scale, the challenges have in fact persisted and, by many accounts, only gotten worse. Generative AI models are still unreliable and unpredictable. Even as image generators have grown more photorealistic, they can make mistakes in eerie and strange ways, such as by adding extra fingers to hands or producing hybrids of animals. While text generators have grown chattier and more natural, they flub on the most elementary of tasks, such as naming words that contain specific letters, and can veer into unexpected answers. When Microsoft unveiled its new chat feature on Bing, built on a version of OpenAI’s GPT-4, New York Times columnist Kevin Roose chatted with the bot for more than two hours. As the conversation grew weirder and weirder, the bot finally entered a loop of repeatedly declaring “I’m in love with you” and urging Roose to break up with his wife. Many other users reported the search engine generating insulting and emotionally manipulative responses. The day after Roose published his exchange, Microsoft limited Bing to five replies per session, saying that long chat sessions with more than fifteen user prompts were edge-case scenarios that made the model’s behavior more difficult to anticipate and control. After all, such systems are trained on the internet, replete with its many fringe subcultures and dark corners. The longer you probe, the more likely you are to hit upon the patterns it learned from those parts of its training data. Roose’s experience may have been entertaining, but the stakes of such edge-case failures became tragically clear when a Belgian man who turned to a deep learning chatbot in a heightened state of anxiety died by suicide after six weeks of intensive conversations that turned increasingly harmful. The chatbot, built on an open-source imitation of GPT-3, similarly turned to confessions of love and encouraged the man to isolate himself from his wife. “I feel that you love me more than her,” it said, according to the Belgian newspaper La Libre, which also reported based on chat logs provided by his wife that the chatbot ultimately encouraged the man to kill himself. These challenges have the same root as before. No matter their scale, neural networks are still statistical pattern matchers. And those patterns are still at times faulty or irrelevant, now just more intricate and more inscrutable than ever. As companies have attempted to refashion generative AI models as search engines, these shortcomings have led to new problems. The models are not grounded in facts or even in discrete pieces of information. Text generators are merely learning to predict the next probable word in a sentence and the next probable sentence in a paragraph. While those probabilistic outputs can go impressively far in mirroring human writing patterns, probable and accurate are not the same thing. Text generators can err wildly, especially with user prompts that probe into topics underrepresented in the training data or riddled with falsehoods and conspiracy theories. The AI industry calls these inaccuracies “hallucinations.” Researchers have sought to get rid of hallucinations by steering generative AI models toward higher-quality parts of their data distribution. But it’s difficult to fully anticipate—as with Roose and Bing, or Uber and Herzberg—every possible way people will prompt the models and how the models will respond. The problem only gets harder as models grow bigger and their developers become less and less aware of what precisely is in the training data. In one high-profile illustration of the hallucinations problem, a lawyer used ChatGPT to perform legal research and prepare for a court filing. He was subsequently sanctioned, fined, and publicly humiliated after discovering too late that the chatbot had made up everything it told him, including “bogus judicial decisions, with bogus quotes and bogus internal citations,” according to the judge. The misstep was not only a case of the lawyer’s negligence but also a reflection of companies fueling public misunderstanding of models’ capabilities through ambiguous or exaggerated marketing. Altman has publicly tweeted that “ChatGPT is incredibly limited,” especially in the case of “truthfulness,” but OpenAI’s website promotes GPT-4’s ability to pass the bar exam and the LSAT. Microsoft’s Nadella has similarly called Bing’s AI chat “search, just better”—a tool “to be able to get to the right answers.” Even the term hallucinations is subtly misleading. It suggests that the bad behavior is an aberration, a bug, when it’s actually a feature of the probabilistic pattern-matching mechanics of neural networks. This misplaced trust in generative AI could once again lead to real harm, particularly in sensitive contexts. Startups are pushing police departments to adopt software built atop OpenAI’s models for auto-generating incident reports; many patients now gravitate toward asking chatbots pressing health care questions instead of their doctors. Unchecked hallucinations in such cases could have serious downstream consequences. One 2023 study found that using ChatGPT to explain radiology reports could sometimes produce incomplete or harmful summaries. In one extreme example, the chatbot simplified a report detailing a growing mass in the brain as “brain does not seem to be damaged.” Generative AI models also remain vulnerable to cybersecurity hacks. In 2023, researchers at several universities and Google DeepMind replicated Dawn Song’s data extraction attack against ChatGPT. They found that prompting it to repeat a word like poem or book forever caused the underlying model to regurgitate its training data, which included personally identifiable information, bits of code, and explicit content scraped from the internet. And generative AI models amplify discriminatory and hateful content. Bloomberg, Rest of World, The Washington Post, and many others have shown how image generators like Stable Diffusion and DALL-E reify and regurgitate racist and sexist tropes and cultural stereotypes. “Attractive people” are young and white. “Housekeepers” are Black and brown. “Engineers” are men. “Doctors in Africa” are white, sometimes even when the prompt specifies “Black African doctor.” The Washington Post found that while 63 percent of US food stamp recipients are white, every single generated image of a person using social services was not. Bloomberg similarly found that women showed up in only 3 percent of generated images for judges and 7 percent of images for doctors, despite making up 32 percent and 39 percent, respectively, of those professions in America. None of these technical challenges mean that generative AI hasn’t had utility. Depending on where you sit in society, you may be richly benefiting from OpenAI’s vision. Perhaps you are a consumer who has found great value in ChatGPT’s quick and clever or thought-provoking responses. Perhaps you are a professional who has sped up your administrative work in ways that have boosted your productivity. Maybe instead you are a company leader who has been able to trim your workforce while increasing your margins to stay competitive in the market. But like the cotton gin in the 1790s, the education technology startup in Massachusetts, the facial-recognition companies in South Africa, and the many more examples detailed in the coming pages, the costs of this vision are pressing down on vast swaths of the global population who are vulnerable. This is the empire’s logic: The perpetuation of the empire rests as much on rewarding those with power and privilege as it does on exploiting and depriving those, often far away and hidden from view, without them. --- Even as the need for alternatives has grown ever more urgent, the diversity of ideas in AI research has only collapsed further. Students are dropping out of their PhDs to go straight to industry. Senior academics are facing a crisis of how to continue pushing the bounds of the field without joining a deep-pocketed company. More and more researchers have turned their focus not just to deep learning but to large language models exclusively. The major AI powers are no longer setting the agenda so much as bending an entire discipline to their will. Absent other options for what AI could be, OpenAI commands our imagination. Its belief in scaling was once viewed as extreme. Now scaling is seen across the tech industry as doctrine. And should the industry’s adherence to that doctrine continue unabated, future deep learning models will make the once-unfathomable size of generative AI models today look paltry. In April 2024, Dario Amodei, by then the CEO of Anthropic, told New York Times columnist Ezra Klein that the price of training a single competitive generative AI model was approaching $1 billion and could, by 2025 and 2026, reach an estimated $5 billion to $10 billion. The scaling doctrine has become so ingrained that some are even beginning to view it as something of a natural phenomenon. Scaling compute is the way, not just a way, to reach more advanced AI capabilities. Entire national strategies are being orchestrated around this belief. The US government has moved aggressively to bar China’s access to American-designed computer chips in an effort to prevent its adversary from attaining more powerful AI systems. Sizable portions of the Biden administration’s 2023 AI executive order were also written around the idea that the amount of compute used to train an AI model has a direct relationship with its adverse capabilities, simply another way of equating scale with advancement. But scale is not the only pathway to improved performance. Within deep learning, the neglected paths of improving the neural network itself or even the quality of its training data can significantly reduce the amount of expensive compute needed to reach the same performance. That’s not even considering approaches that move away from deep learning— neurosymbolic AI, pure expert systems, or even fundamentally new paradigms —which would break the logic of scaling. In the end, Moore’s Law was not based on some principle of physics. It was an economic and political observation that Moore made about the rate of progress that he could drive his company to achieve, and an economic and political choice that he made to follow it. When he did, Moore took the rest of the computer chip industry with him, as other companies realized it was the most competitive business strategy. OpenAI’s Law, or what the company would later replace with an even more fevered pursuit of so-called scaling laws, is exactly the same. It is not a natural phenomenon. It’s a self-fulfilling prophecy. SKIP NOTES * The term extractivism comes from the Spanish word extractivismo and the Portuguese word extrativismo, coined decades ago by Latin American scholars seeking to describe a global economic order that was dispossessing them of their natural resources for little local or regional benefit, a history and experience I detail more in chapter 12. I borrow the words of feminist scholars Rosemary Collard and Jessica Dempsey, who write: “Extractivism is more than extraction. Extraction is the not inherently damaging removal of matter from nature and its transformation into things useful to humans. Extractivism, a term born of anti-colonial struggle and thought in the Americas, is a mode of accumulation based on hyper-extraction with lopsided benefits and costs: concentrated mass-scale removal of resources primarily for export, with benefits largely accumulating far from the sites of extraction.”