The Release of Claude Fable 5 and the Controversy That Followed
Anthropic recently launched its latest Mythos-tier model, named Claude Fable 5, marking a significant milestone for the company. This release came just over a week after it was revealed that the company had confidentially filed for an initial public offering (IPO). Initially, Anthropic considered Mythos-class models too dangerous to release due to their advanced ability to detect software vulnerabilities. However, the company now claims that new safety measures in Claude Fable 5 are sufficient to prevent these capabilities from being misused.
Despite this, the release of the model sparked immediate backlash from various AI researchers, developers, and policy experts. The controversy centered around a section in the model's 319-page system card, which disclosed that Fable would quietly reduce the quality of its responses when detecting requests related to cutting-edge AI development. This means users could receive weakened answers without knowing the model is holding back information.
Criticism of Hidden Restrictions
The hidden restrictions in Fable 5 were particularly concerning because they differed from other limitations, such as those related to cybersecurity and biology, which clearly redirect users to a less powerful model with visible notifications. In contrast, the system card stated that the restriction "is not visible to the user." The model still responds but uses interventions to limit its effectiveness without informing the user.
Anthropic estimated that these restrictions would affect approximately 0.03% of traffic. The company defended its approach by stating that enforcing these restrictions through safeguards helps avoid accelerating the actions of those most likely to violate the terms of service.
Widespread Backlash from the AI Community
The AI community reacted strongly to these revelations. Open-source researchers, who often criticize Anthropic's closed policies, and AI safety experts who typically align with the company, expressed their concerns. Nathan Lambert, an open-model researcher, described the situation as "appalling," arguing that it paints Anthropic as anti-science and therefore anti-progress and anti-safety.
Dean Ball, a senior fellow at the Foundation for American Innovation, criticized the "secret sabotage" policy, suggesting it raises questions about whether AI safety is being used to justify monopolistic behavior. Jeremy Howard, head of Fast AI, also voiced concerns, stating that Anthropic is allowing itself to use its top model for frontier research while sabotaging others, which he believes increases power imbalances.
Former Anthropic employees also joined the criticism. Behnam Neyshabur, who previously co-led the effort to develop an AI scientist at Anthropic, highlighted how the restrictions could hinder important research in areas like cancer and Alzheimer’s disease. He argued that concentrating these capabilities slows scientific progress and is harmful to humanity.
Mixed Reactions from Prominent AI Figures
Not all prominent AI voices criticized the restrictions. Ethan Mollick, an associate professor at Wharton, praised Claude Fable 5 for outperforming other public models. Andrej Karpathy, former OpenAI co-founder and Tesla AI director, called the release "super exciting" and noted it was a major step forward, though he acknowledged that some safeguards may need tuning.
Anthropic’s Commitment to Safety and Accessibility
Before the release, Anthropic anticipated potential backlash but did not specifically address the research restrictions. Dianne Na Penn, Anthropic’s head of product management, research, and labs, emphasized that the new model achieved performance improvements of 10 to 20 points over previous models. She stressed the importance of balancing accessibility and safety, saying, “We’re raising the bar on the intelligence of the models, and at the same time, we are pushing the frontier in a safe manner.”
She also mentioned that some benign requests might initially be blocked but that the company is actively working on improving safeguards post-launch.
Anthropic did not respond to requests for further comment.
No comments:
Post a Comment