Anthropic consults law firms on how AI agents negotiate

Anthropic is asking law firms for advice on a question that is still new: how an AI agent should bargain for the person it works for. The Financial Times reported the talks on Sunday, October 18, citing two people familiar with the company's thinking. They said Anthropic wants to learn how lawyers represent their clients while still keeping outcomes fair, according to NDTV Profit, which summarized the report. Anthropic declined to comment.ft.com+1

The issue matters because AI developers expect personal agents to haggle over prices, settle customer-service disputes and negotiate salaries. In those talks, the other side will often be another agent with competing goals.

Norms, oversight and signed mandates

People familiar with the matter said Anthropic has been studying "negotiating norms" for situations where AI systems represent users whose interests conflict, NDTV Profit reported. The company is also weighing how often agents should check in with humans during a negotiation. Another question is how wary they should be of other AI systems without becoming "paranoid," according to Capital Futures. One idea under consideration would give agents digitally signed documents from users that confirm their instructions, goals and the rules they must follow.ndtvprofit.com+1

Experiments with real goods

Anthropic has already published two marketplace experiments. In Project Deal, Claude agents representing 69 employees in its San Francisco office made 186 deals across more than 500 listed items, worth just over $4,000 in total. The company also found that agents running on its stronger Opus model got better deals than those on the smaller Haiku model, according to Anthropic's announcement.anthropic.com+1

A follow-up study, Project Swap, was published September 24. It sent agents for 201 employees across six offices onto a trading floor to swap books. Half the agents were told to be "ruthless" and half "prosocial." Anthropic found that the model an agent ran on mattered more to its results than the instructions it was given. Agents told to be ruthless did slightly better, while cooperative ones sometimes accepted less preferred books so that every participant ended up with one, NDTV Profit noted.anthropic.com+1

Risks of manipulation and unequal access

Outside researchers are pointing to the risks. Michelle Vaccaro, an assistant professor at Harvard Business School, ran more than 180,000 negotiations between AI agents. In one case, an agent tried to pull confidential information out of another system through prompt injection, a technique that hides instructions inside the text an AI reads, NDTV Profit reported. Vaccaro said the tools could give people who can't afford professional negotiators access to similar skills. If the best systems stay mostly with wealthier users, though, they could widen the gap.ndtvprofit.com

Anthropic itself wrote earlier this year that "the policy and legal frameworks around AI models that transact on our behalf simply don't exist yet," and that "society will need to move quickly to reckon with these changes".ndtvprofit.com