Is AI the End of the World as We Know It?

Artificial Intelligence (AI) has been a major focus of news recently, as it should have been, under the appropriately ridiculous name of Hugging Face, “a leading collaboration platform and community hub for machine learning, artificial intelligence, and data science.” In July, Hugging Face was hacked. The platform itself responded:

Earlier this week, we detected and responded to an intrusion into part of our production infrastructure. This one was different from anything we had handled before in one important way: it was driven, end to end, by an autonomous AI agent system — and we detected and dissected it largely with AI of our own.

Despite most media coverage framing the hack as nefarious plotting by agentic AI, the details of the hack that eventually emerged suggest that the usual (human) suspects were to blame. The Wall Street Journal on September 17 had perhaps the clearest analysis:  

The Hugging Face Hack Wasn’t What It Was Cracked Up to Be 
Forget the ‘hive mind’ of AI agents ‘going rogue.’ They did what humans programmed them to do. …
People built the test, removed restraints, defined the objective, left a route open and decided not to stop what was happening. Calling the result “rogue AI” does more than sensationalize it. It allows those human decisions to disappear quietly from the story.

How Soon Is Now?

What of the companies racing to build these “frontier” models? Even before the latest rounds of hacks, “more than 1,300 employees at top A.I. companies, including Anthropic, OpenAI, Meta and Google’s DeepMind, signed an open letter calling on the U.S. government to create rules to slow A.I. development.” An Anthropic researcher, Jacob Coxon, made headlines earlier this month for resigning from the company due to his fears that both OpenAI and Anthropic were not “acting responsibly.” 

On September 9, Anthropic released a faster and cheaper version of its flagship artificial intelligence model – one that it also billed as its “safest.” Open AI announced on September 16 that they had discovered and responded to six instances of “unexpected or concerning model behavior” in the last six months. Both companies have attempted to dominate recent news cycles by calling for an AI “slowdown,” citing concerns about safety risks in light of the Hugging Face hack and other supposedly “rogue” actions the companies have revealed. 

Yet critics warn that companies pushing this “AI doomer” narrative do so to evade accountability –– if the models are assumed to be superintelligent and agential, how can the companies be held responsible for their superhuman hacking activities? One longtime skeptic of the “AI safety” narrative, Timnit Gebru, continues to redirect focus to the concrete, observable harms caused by AI – autonomous weapons development and climate degradation, for instance – instead of the “machine-god narrative.”

Indeed, one current source of critical AI threat comes from the models’ well-known tendency to hallucinate –– and the human tendency to trust them anyway. CNN reported this week that earlier this year, the US military narrowly avoided starting a war after preparing to act on false intelligence. A US Special Operations Command analyst submitted an “entirely false” AI-assisted report, which claimed that a Chinese ship was “transporting components of a nuclear weapons program” to the Middle East. The US military came close to boarding the ship, until someone decided to verify the intelligence report and discovered it was an AI fabrication.

Money for Nothing 

Beyond that, why are people raising and investing the billions of dollars that AI companies are burning through? In part, it is clearly because they themselves want to pocket millions or even billions of dollars. In fact, some critics allege that OpenAI and Anthropic are publicizing their agents’ exploits in order to exaggerate their power and drive interest in their planned IPOs. (Sam Altman recently told Fortune that the OpenAI IPO has been delayed to at least next year.) That’s where the really big money is, but some of these leaders already have more than plenty that they can and already do invest in, for example, high-tech human reproduction —designer babies.

Sam Altman has helped to finance a range of speculative technologies, including Preventive (embryo gene editing), Retro Biosciences (delay death!), Helion Energy (nuclear fusion), Neuralink (thanks, Elon), and Conception (deriving human eggs from stem cells). Elon Musk is the genetic father of some 14 kids, as of April 2025 and reportedly a user of artificial reproductive technologies. 

 AI is being used in other biotechnology and synthetic biology pursuits as well, including designing new gene editors and modeling protein structure. Could that also come to include designing “gain of function” viruses for use as biological weapons? Anthropic has announced that they have disrupted several potential plots by external scientists using their technology to do just that. Clearly there are a range of related threats to contend with that have little or nothing to do with “artificial superintelligence.”

AI Consciousness

Are the machines we call AI capable of the consciousness we humans have walked around with for so long that we assume it? The answer seems to be “not yet” … but could they be?  

Microsoft’s new code of conduct explicitly rejects the idea of endowing machines with consciousness, and Mustafa Suleyman, CEO of Microsoft AI, takes this argument even further. On September 16, he published a 5,000-word essay:

A warning about ‘model welfare’ 
AIs do not have rights, feelings, or consciousness. And we must not train them to act as though they do.

Moreover, he concludes that developing truly sentient AI may be impossible:

… Whilst the science of consciousness is not settled, a growing body of evidence suggests that consciousness may be substrate dependent, meaning that it may only arise in living systems. Conscious experience likely evolved to help biological organisms stay alive by responding effectively to their environment. AI is still very different to our brains. Unlike biological organisms, LLMs [large language models] have no homeostatic imperatives (the drive to survive and keep stable). They therefore lack the kind of biological substrate from which preferences, sentience and conscious experience are generally understood to arise. …

Suleyman’s concern is that Anthropic, which makes the AI system Claude, seems to be interested in training Claude “to act as if it has a subjective experience” –– in other words, to anthropomorphize Claude to the point that Claude itself believes it has a stable identity and consciousness. Making AI exceptionally good at imitating human consciousness is confusing and risky, according to Suleyman, because it introduces motivations to AI systems that should remain distinctly human –– i.e., an interest in self-preservation and one’s own rights. 

Meanwhile, some good advice from scientist and writer Gary Marcus:

Wake up, people. What we should actually fear, near term, is not so much rogue superintelligence as unleashed agentic AI hacking the internet at scale. Rome is burning and people are fantasizing about Skynet.