As AI models continue to evolve at lightning speed, leading labs like OpenAI, Google, and Meta increasingly rely on crowdsourced platforms like Chatbot Arena to gauge their models’ AI benchmarks performance. But some experts are raising red flags, warning that this popular approach may be flawed — both academically and ethically.
The idea behind platforms like Chatbot Arena is simple: users anonymously test two competing AI models and vote for the response they prefer. These votes help form leaderboards, which labs sometimes point to as proof of a model’s superiority. However, critics argue the method lacks scientific rigor.
“To be valid, a benchmark needs to measure something specific,” said Emily Bender, linguistics professor at the University of Washington and co-author of The AI Con. “Chatbot Arena hasn’t shown that voting for one output over another actually correlates with preferences, however they may be defined.”
Beyond academic concerns, some believe AI labs are using these benchmarks more for marketing than for meaningful evaluation. Asmelash Teka Hadgu, co-founder of AI firm Lesan and a fellow at the Distributed AI Research Institute (DAIR), argues that benchmarks are being “co-opted” by companies to back exaggerated performance claims.
Hadgu cited the recent controversy surrounding Meta’s Llama 4 Maverick model, which reportedly was fine-tuned to perform well in Chatbot Arena, but never publicly released. Instead, a weaker version was introduced — leaving some to question the integrity of the benchmarking process.
The Need for More Reliable and Fair Evaluation
Hadgu and Kristine Gloria, former lead of the Aspen Institute’s Emergent and Intelligent Technologies Initiative, also raised concerns about the treatment of volunteer evaluators. Both suggested that AI labs should fairly compensate those who test and rank AI models, a practice rarely seen in current benchmarking systems.
“Benchmarks should never be the only metric for evaluation,” Gloria said. “With the industry moving so quickly, static datasets and volunteer testing can rapidly become outdated or misleading.”
Others in the AI space agree that crowdsourced platforms alone aren’t enough. Matt Frederikson, CEO of Gray Swan AI — a firm specializing in red teaming AI models — emphasized the importance of layering multiple evaluation strategies.
“Public benchmarks are useful, but they’re no substitute for internal evaluations, algorithmic red teams, and domain-specific testing,” Frederikson said. “Results need to be transparent and open to challenge.”
Even some of the people behind these platforms acknowledge the limitations. Wei-Lin Chiang, an AI doctoral student at UC Berkeley and co-founder of LMArena, which runs Chatbot Arena, said the platform isn’t meant to be a definitive test, but rather a reflection of community preferences.
“We welcome the use of other tests,” Chiang said. “Our goal is to create a trustworthy, open space that measures our community’s preferences about different AI models.”
In light of incidents like the Maverick model discrepancy, LMArena has updated its policies to strengthen the integrity of its rankings.
“As long as the leaderboard faithfully reflects the community’s voice, we welcome it being shared,” Chiang said, emphasizing that the platform’s mission is to promote open, transparent engagement rather than replace more formal evaluation methods.
Get the Latest AI News on AI Content Minds Blog