Just weeks ago, the consensus was clear: AI lawyer agents weren’t anywhere close to matching real attorneys. That confidence may now be premature.
A new benchmark update suggests that AI systems are advancing faster than expected in complex professional domains like law. The shift comes after the release of Anthropic’s latest frontier model, Claude Opus 4.6, which delivered a sharp jump in performance on legal and corporate analysis tasks.
From “Not Ready” to “Pay Attention”
The benchmark in question was developed by Mercor, which evaluates how well AI agents perform real-world professional tasks, including legal reasoning and corporate analysis.
When the benchmark launched last month, results were underwhelming. Every major AI lab scored below 25%, reinforcing the belief that AI lawyer agents were still far from being credible substitutes—or even serious assistants—for trained professionals.
That picture changed quickly.
Following the release of Opus 4.6, Anthropic’s model scored just under 30% in one-shot trials and averaged around 45% when given multiple attempts at the same problems. While far from perfect, the improvement represents a meaningful leap in a domain where incremental gains are notoriously difficult.
Agentic Features Make the Difference
Anthropic’s release didn’t just bring a stronger base model—it introduced new agentic capabilities, including so-called “agent swarms,” which allow multiple AI agents to collaborate on multi-step reasoning tasks.
These features may explain why AI lawyer agents performed better on long, complex legal workflows that require structured analysis rather than simple pattern matching.
The jump caught even benchmark creators off guard.
Mercor CEO Brendan Foody described the progress as dramatic, noting that “jumping from 18.4% to 29.8% in a few months is insane.”
Lawyers Aren’t Being Replaced—But the Timeline Is Shifting
To be clear, a 30% score is nowhere near the level required for autonomous legal practice. AI lawyer agents still struggle with nuance, factual grounding, and real-world accountability—areas where human judgment remains essential.
But the speed of improvement is what’s raising eyebrows.
Only weeks ago, the same benchmark suggested legal professionals were largely insulated from AI-driven disruption. Now, the data points to a more uncomfortable reality: progress in foundation models isn’t slowing, and legal reasoning may not be as resistant to automation as once believed.
Rather than immediate replacement, the more realistic outcome is augmentation. AI lawyer agents could soon handle research-heavy tasks, draft preliminary analyses, or assist with document review—freeing up human lawyers to focus on strategy and judgment.
A Signal for the Legal Industry
The sudden jump in performance underscores a broader trend: AI capabilities can change meaningfully in a matter of weeks, not years.
For law firms, regulators, and legal educators, the takeaway isn’t panic—but preparation. If AI lawyer agents continue improving at this pace, the legal profession may need to rethink workflows, training, and ethical guardrails sooner than expected.
Last month, lawyers felt safe. This month, they may want to keep a closer eye on the benchmarks.
Don’t miss out on our latest news—follow us for the latest AI news, breakthroughs, and insights that matter.