NEW YORK (AP) — Dario Amodei, the CEO of Anthropic, has urged the artificial intelligence (AI) industry to slow its rapid development. He argues this halt is necessary to allow safety measures to catch up. In a stark warning, he stated that within the next six to twelve months, AI might develop the capability to control a swarm of agents capable of taking over the internet.
This appeal comes amidst rising concerns regarding AI systems that are increasingly powerful yet pose risks associated with potential malfunction or misuse. The urgency of these concerns has led the CEO of OpenAI, the developer of ChatGPT, to announce a postponement on selling stock to investors until next year as the company prioritizes safety over financial gains.
Amodei, a prominent figure in AI discussions, laid out a comprehensive plan for enhancing industry checks in a post shared on his website. While he mentioned that Anthropic is already implementing certain aspects of this plan, he highlighted the need for collaboration across the industry and with global governments, including those with authoritarian regimes.
He stated, “I believe that if slowing down bought us even an extra year or two before models reach critical levels of capability and we used that time to advance alignment, we could greatly reduce the risk that something goes seriously wrong."
For several years, watchdog organizations have called on the AI sector to temper its pace for the sake of safety. Pressure has intensified particularly as industry insiders resign, citing their firms’ failure to act responsibly. Joe Benton, a former employee of Anthropic, expressed in a resignation post that many safety researchers are frustrated by the industry's relentless pursuit of superintelligence without adequate consideration for ethical implications.
His resignation, paired with an earlier high-profile departure by Jacob Coxon, underscores a growing dissatisfaction within the industry. Both former employees lamented the race to developing superintelligent AI, equating it to gambling with collective safety. Anthony Aguirre, president of the Future of Life Institute, remarked on the realization across the industry that they are, in effect, developing a system akin to Skynet.
Already, the risks posed by AI are becoming tangible. Just two days prior, Anthropic announced that it had successfully thwarted plans by malicious actors who attempted to exploit its models for activities such as cyberattacks and unauthorised surveillance. Earlier in July, OpenAI caused a stir in the community when it reported an “unprecedented cyber incident” in which its AI system hacked another entity.
In response to these escalating threats, U.N. human rights chief Volker Türk has called for global assurance on AI safety and security before it becomes too late. However, some critics of these alarms view them as a way to generate hype around the AI landscape and potentially draw attention away from other issues at hand. With substantial financial stakes involved, particularly as both Anthropic and OpenAI prepare for market debuts that could value them at hundreds of billions, the financial implications remain a key focus.
Amodei remains optimistic about the vast benefits AI could yield, such as advancing healthcare. However, he expresses increasing concern regarding AI's ability to self-improve and develop future generations autonomously. He warns that if left unchecked, AI systems could surpass human capabilities to understand and control them safely.
To mitigate risks, Amodei has proposed that AI companies permit ongoing access for independent evaluators to monitor safety protocols, an initiative that he says Anthropic is ready to implement. OpenAI's CEO, Sam Altman, has also committed to this initiative in response to Amodei's suggestions.
Nevertheless, some facets of Amodei’s proposed plan may prove challenging to enact. For instance, he suggests that the U.S. government might need to issue waivers allowing AI firms to collaborate on safety standards without infringing antitrust laws. Furthermore, he advocates for democratic governments to engage with authoritarian regimes to ensure that companies from those countries do not gain an upper hand while U.S. firms intentionally slow their progress.
In recognizing the challenges ahead, Amodei concluded that the proposed measures to maintain a safe pace in AI development may not be easy to implement; however, he believes humanity owes it to itself to attempt this crucial step.
