Anthropic Warns Future AI Models Could Grow Too Fast for Safety Checks

Artificial intelligence could soon start accelerating the creation of newer systems at a speed that moves past the ability of companies to build proper safety rules. Anthropic shared this warning in an amended registration statement filed with the U.S. Securities and Exchange Commission on November 2.

Recursive Self-Improvement

The warning focuses on a process where AI systems help design and build more capable follow-up models. Anthropic stated that its Claude models already help with several steps of making future generations, such as research tests, training data, safety checks, and coding. Over 90% of contributions to its codebase in September came from Claude, and the average daily code written per engineer in the three months ending September 30 was more than 45 times the 2024 average. Claude's success rate on tough engineering problems hit roughly 90% in September, jumping by nearly 50 percentage points in nine months.

Risks And Steps

Anthropic noted that future models could bring major safety risks and that its own power to measure this speed remains limited. The company warned that advances might move faster than its ability to build and apply safety and security safeguards. It is now thinking about changes to its Responsible Scaling Policy to keep humans in control of advanced AI development. This careful approach has already led the company to delay releases and limit access to certain models, such as keeping Mythos restricted to select groups through Project Glasswing instead of a public launch. However, these limits could slow its progress against rivals who do not use such rules.

Context

Artificial intelligence companies often use their own tools to write code and test new systems to speed up research. This growing reliance on automated coding highlights the challenge developers face in keeping human oversight strong as systems get smarter. Such safety debates shape how the tech industry plans to manage powerful new models in the future.