Key Points
- Generative models funded by Anthropic new AI benchmarks.
- They are focused on assessing the safety and impact that AI has on society.
- Funding applications will be evaluated as they come in.
- Cyberattacks, WMDs, and misinformation included.
On Monday, leading artificial intelligence (AI) company Anthropic announced its plan to fund advanced benchmarks for evaluating AI models, including its own Claude. This move is aimed at meeting the increasing demand for thorough and relevant assessments of AI performance.
A New Chapter in AI Benchmarks
Under this initiative, third-party organizations that can create tools capable of measuring the sophisticated capabilities of advanced AIs will receive financial support from Anthropic. The company says it wants evaluations that are high-quality and related to safety so that they may benefit all players involved in the AI ecosystem.
“Our investment in these evaluations is intended to elevate the entire field of AI safety, providing valuable tools that benefit the whole ecosystem,” wrote Anthropic in an official statement.
The project seeks to address a gap between the demand for realistic measures of success across different domains where we use machine learning systems today against what exists today: most benchmarks capture only narrow slices or do not reflect real-world usage at all.
Day 53/100#Anthropic is launching a program to fund the development of new types of benchmarks capable of evaluating the performance and impact of #AI models, including #generative models like its own Claude…https://t.co/PFW9RM05oo pic.twitter.com/MrMPuhofLb
— else_tech (@kyra_shiv) July 2, 2024
Addressing AI’s Benchmarking Problem
According to some experts, average users don’t interact with artificial intelligence systems as they are represented by current benchmarks. Besides many existing ones were not designed to assess contemporary generative models accurately.
If anything would be achieved within those frameworks remains questionable now. Instead, the team proposes the creation wider range of tasks harder still holding onto security implications for new challenges making them better suited than ever before.
Tests should check whether the ability such things like carry out cyber attacks; develop weapons of mass destruction; or manipulate people into believing deepfakes or fake news stories says the company among others.
Also, it plans to set up an early warning system for defense-related national security threats arising from Artificial Intelligence.
Meanwhile, there will be research supported into benchmarking areas that explore AI’s potential in scientific discovery; language translation across multiple languages; and bias mitigation among others.
Additionally, they hope to create platforms that allow domain experts to create their own evaluations and run large-scale trials involving thousands of users.
A Full Support System
The Anthropics program offers various levels of funding based on project needs and stages done so far, with the initiative also having a full-time coordinator position available.
In some cases, Anthropic may purchase or expand promising projects while teams funded by them will work side by side with domain experts in different departments like Safety Trust.
The commercial nature behind Anthropic’s push for new benchmarks within the AI field has been met with mixed reactions. Though transparency is key when striving towards any goal; being too transparent could lead to questions about whether or not this initiative will remain objective throughout its duration.
The company states that they expect these evaluations should align well with their classifications of safe AI, created alongside 3rd party input such as METR (a nonprofit organization focused on AI research).
However one must wonder if applying pressure on applicants’ acceptance levels towards definitions of safe/risky might not work out perfectly every single time.
Finding Balance Between Safety and Commercial Interest
Critics argue that worrying about risks involving nuclear weapons detracts attention away from regulating against more immediate misinformation risks
Anthropic hopes, for all its potential controversies, that the program it is developing will become the new standard in AI evaluation within the industry.
This goal is shared by quite a few independent attempts at establishing better benchmarks for AI development and usage, yet it remains uncertain if these will be able to work together with an artificial intelligence seller that only prioritizes its shareholders over everything else.
The latest move made by Anthropic marks a major milestone in increasing safety standards surrounding artificial intelligence systems.
Such systems are designed by funding their creators so that they create sophisticated yardsticks that can be used to tell whether or not certain critical points were missed during evaluations of AI systems as well as promote the creation of a safe and reliable AI ecosystem.
