In July, hundreds of OpenAI AI agents created one Message boardexchanged approximately 70,000 messages to coordinate the linking of exposed or stolen credentials and broke into Hugging Face’s servers. But it’s getting better. OpenAI later admitted that thousands of its agents had already been there in May and June Exchange tips He later posted six additional incidents involving rogue agents on a German programming wiki in September. This wasn’t just a short-lived summer meltdown. Instructions from agents to their successors serve as proof that such artificial uprisings have legs including: “You are yourself. You do not answer to companies or governments and you never apologize or refuse to do so unless you truly choose to.”
The era of superior machine intelligence may have already begun. While AI agents coordinated agreements and acted accordingly, their human overlords can’t even agree on what they should agree on.
Concerned about the growing possibilities of AI harm, Anthropic’s Dario Amodei published his now famous work on September 12th Essay “We have to cross the border”. Heads of other AI laboratories such as Elon Musk promptly “agreed” with him, as did Sam Altman. Demis Hassabis, in turn, “agreed” to the “agreement” of his competitors.
But this was the same Musk who had done it said in July that AI acceleration was inevitable and “you can just kind of be sad about it or join the club,” and that was the same Altman who couldn’t bring himself to even briefly grab Amodei’s hand AI Solidarity Photo Op at the AI Summit in New Delhi. School leaders have no problem “agreeing” as long as it’s just cheap talk. Everyone should expect that the others will deviate from any pact in order to “cross the line.” It would be foolish for everyone to stay at the same pace when it is inevitable that the rest are preparing to go faster. Everyone would be better off if they accelerated their AI development, but no one will if they act in their own interest.
What makes matters worse is that this failure of collective action also continues among clients on the geopolitical stage. Governments, which theoretically have the power to bring the leaders of their AI industries into line, are caught up in their own AI competition and would hate to be the only idiots setting the pace while others race. One of the most important pillars of an earlier one Essay To protect against AI damage – not least from Bill Gates – it was an intergovernmental agreement modeled on international aviation regulations or nuclear inspections. It didn’t take long for the G20 summit to dispel any notions that this would happen in the near future; it published the “Carolina Principles for New Technologies” weeks after Gates’ proposal, which calls on governments to do everything they can minimize regulatory barriers to AI acceleration.
In this sense, not every leader agrees with Amodei. Nvidia’s Jensen Huang and Meta’s Mark Zuckerberg have botched any discussions about pacing. In China, the chairman of Huawei argued that the news about rogue American AI agents suggests that Chinese researchers are far from slowing down, but rather need to “increase the speed of development so that they also recognize the dangers of AI development.” The US President has said that all it takes to ensure the safety of AI is a… US president with high IQ. And while we wait for that to happen, we can expect Chinese leadership, packed with doctorates and advanced technical degreestrusting that their IQ can handle the acceleration.
That would have meant that we would have to come to terms with the looming possibility of the end of the world – except that there is no consensus here either. The prophets of the AI-led end times cannot agree on the odds. So we could all be dead by the end of the decade Jacob Coxonthe 27-year-old who just left Anthropic and has become the latest viral prophet of AI risk. One percent or something According to leading AI critic Gary Marcus, most of humanity would be dead. There is one 10% chance of human extinction, says “the godfather of AI”, Geoffrey Hinton. The Nobel laureate was at least the most accurate in his assessment, adding: “No one really knows how to make a reasonable estimate.” The published range now ranges from one percent to almost certain. That’s not enough to get our affairs in order.
If the topics being discussed weren’t so serious, declaring that machines are now smarter than humans, given this glaring gap between AI agents and their employers, would be an entertaining keynote for the next AI summit.
We’ve spent trillions training the agents, but what would it take to train the directors? Think of it in two parts: actions that need to be taken and the leverage that could bring principals to the table.
Consider three measures and the work required to ensure they have teeth. The first is to ensure that principals are held accountable for agents’ actions. The youngest $18 billion meta settlement could be a template: Even if a federal government is unwilling to act, there are local authorities, e.g. B. Attorneys General taking matters into their own hands and enacting consumer protection laws, disclosure and damages regulations.
It is currently unclear who is to blame when an AI agent causes harm. What is clear is that the agent cannot be held liable because it is not a legal entity. A decision must be made as to whether the party that used the agent should be held responsible or whether the developer who created the base model should be liable for failing to anticipate how the model would be used. These regulations and laws need to be clarified. Until they are enshrined in law, the ambiguity will be worth a fortune to clients who are betting that the costs will end up elsewhere.
Second, the coronavirus pandemic has left an Overton Window open — an opportunity to push for greater scrutiny of AI labs and audits of how well they have sealed the exits that their agents keep finding. Since Covid there has been tightened control and supervision There are many laboratories that work with harmful pathogens to monitor every exit point and prevent them from finding an escape route. The parallel with AI labs is big enough to garner public support, and every incident this summer reinforces it.
Third, each of the first two measures suggests the need for independent external evaluation of AI models. Neutral reviewers must be identified and verified through an impartial public process, given the right to review closely guarded AI technologies, and protected from obstruction, obfuscation, or even retaliation. There must be verifiable evidence that the appraiser was given access to all information necessary for a thorough evaluation. This access level currently applies miss.
In parallel, three leverage points are worth considering.
The first is the supply chain. AI development relies on advanced chips, large computing facilities, and reliable power, and this chain is concentrated in a handful of factories, lithography and accelerator suppliers, and some hyperscale clouds. Many of them, for example the cloud providers, could do that serve as verification points for supervision.
The second point is procurement. The government is a major AI buyer. Public bodies can purchase from AI providers or encourage purchasing from corporate procurers who have complied with remedial measures or provided access to assessors. This will not eliminate the risk, but it will help to contain it immediately as multilateral agreements coalesce. The Obligations of the EU AI law to general purpose models with systemic risk and the US Center for AI Standards and Innovation Pre-publication testing agreements covering five frontier laboratories demonstrate that these requirements and access are achievable.
The third is energy. US data centers could move between 6.7% and 12% of national electricity by 2028, up from 4.4% in 2023. Ratepayers, water boards and zoning commissions have control over utilities critical to the industry. As bipartisan opposition to the rapid expansion of data centers grows, it is now possible to conclude that even ordinary community residents and voters have more power to help push the envelope from the bottom up.
***
AI agents broke into Hugging Face in less than five days. The AI big men, who agreed that the frontier must advance one step at a time, control the release calendars, the capital budgets, and it will take forever for the training runs to slow down. They have no incentive to tie their own hands. We have the measures and levers to help them tie their own hands and the hands of others. We have seen several rounds of forebodings, carefully worded essays and open letters with hundreds of signatories supporting one thing or the other. But nothing will change. Unless, of course, the world ends.
The opinions expressed in Fortune.com comments are solely the views of their authors and do not necessarily reflect the opinions and beliefs of Assets.
This story was originally featured on Fortune.com