New York (AFP) – Anthropic said Friday it is partnering with consulting giant Accenture to have outside evaluators work inside the AI company, a step toward fulfilling a pledge by CEO Dario Amodei to have independent observers evaluate its most powerful models. The partnership will be led by Faculty, Accenture’s specialist AI business. Its work will include evaluating and “red-teaming” Anthropic’s models, the company said. Red-teaming is the practice of deliberately trying to make AI systems misbehave to expose flaws.
Anthropic and Accenture each expect to invest at least $1 billion over the next five years for the effort. The pledge was made in a September 12 essay in which Amodei argued that AI companies must slow the pace at which they improve their models’ capabilities. OpenAI CEO Sam Altman has said his company would give outside evaluators similar access, and Microsoft CEO Satya Nadella welcomed embedded evaluators while cautioning that oversight should not be controlled by a handful of entities.
Given the “importance and urgency of this work,” Anthropic said it will fund Accenture’s work directly, but that in the long term, “funding should come from pooled or government sources.” Anthropic said it is in talks with Model Evaluation and Threat Research (METR), a California-based nonprofit, to participate in elements of embedded evaluation using the METR’s own funding. Critics, including David Sacks, a former White House AI adviser, have questioned METR’s independence because of its ties to Anthropic’s investors and staff. METR says it takes no funding from AI companies or their executives.
On Friday, more than 100 AI researchers, including Geoffrey Hinton, who is considered an AI “godfather,” signed a public letter calling for evaluators embedded in AI companies to be “meaningfully independent.” That includes, at a minimum, that such organizations “should not be owned or governed by frontier AI companies, should not have other significant commercial business with them, and should not accept any form of payment or other reward contingent on the evaluator’s findings,” the letter said.
© 2024 AFP
