Japanese AI startup Sakana is making headlines with its bold claim that its AI system successfully authored one of the first AI-Generated research scientific papers. However, while the assertion is technically true, there are key caveats that make the achievement less groundbreaking than it may initially seem.

Did Sakana’s AI Really Pass Peer Review?

As AI’s role in scientific research continues to spark heated debates, Sakana is pushing forward, arguing that AI can be a valuable co-researcher. The company revealed that it used its proprietary AI system, The AI Scientist-v2, to generate a research paper, which was later submitted to a workshop at ICLR, a well-known AI conference.

According to Sakana, the conference’s organizers collaborated with the company to experiment with AI-generated scientific manuscripts in a double-blind review process. Partnering with researchers from the University of British Columbia and the University of Oxford, Sakana submitted three AI-generated papers—all created entirely by AI, including their hypotheses, experiments, data analyses, and even visualizations.

One of these papers, which critiqued existing training techniques for AI models, was accepted by the workshop. However, before it could be formally published, Sakana withdrew it, citing transparency concerns and respect for ICLR’s conventions.

Flaws in AI-Generated Research: Citation Errors & Lack of Scrutiny

Despite the excitement around AI’s potential in academia, Sakana acknowledged that its AI system made significant mistakes during the research process. For example, it incorrectly attributed a 2016 method to a paper from 1997, raising concerns about AI’s ability to accurately cite sources.

Moreover, the peer-review process the paper underwent wasn’t as rigorous as it might seem. Since the paper was withdrawn before final publication, it skipped the meta-review stage, where workshop organizers could have potentially rejected it.

Additionally, experts highlight that workshop acceptance rates are often much higher than those for full conference tracks. Sakana even admitted that none of its AI-generated studies met the standard for acceptance in the main ICLR conference track.

Experts Weigh In: Is AI Actually Doing Science?

The AI research community remains skeptical about AI’s ability to contribute meaningful scientific work. Some researchers argue that AI is simply good at mimicking human writing, rather than truly advancing scientific discovery.

Matthew Guzdial, an AI researcher and assistant professor at the University of Alberta, believes the claim is misleading.

“Sakana selected which AI-generated papers to submit, meaning human judgment played a role in filtering outputs. This experiment proves that humans plus AI can be effective—not that AI alone can conduct groundbreaking research,” Guzdial said.

Similarly, Mike Cook, an AI researcher at King’s College London, questioned the rigor of the peer-review process.

“New workshops are often reviewed by junior researchers, and this specific workshop focused on negative results and failures—which makes it easier for AI to convincingly frame a paper around an experiment that didn’t work out,” Cook explained.

Cook further noted that AI’s ability to pass peer review isn’t new. Partially AI-generated papers have been accepted in journals before, raising ethical concerns about AI’s role in academia.

AI in Science: A Tool or a Threat to Research Integrity?

The rise of AI-generated research is forcing the scientific community to confront difficult questions. While AI can automate certain aspects of research, experts worry it may also introduce bias, misinformation, and lower-quality papers into academic literature.

“Is AI actually conducting meaningful research, or is it just good at ‘selling’ ideas to human reviewers?” Cook asked. “There’s a difference between passing peer review and truly contributing knowledge to a field.”

Sakana acknowledges these concerns and insists its goal wasn’t to prove AI’s superiority in scientific research but rather to highlight the urgent need for ethical guidelines in AI-generated science.

“AI-generated research must be judged fairly, without bias, while ensuring that the peer-review process isn’t compromised,” the company stated in its blog post.

With AI rapidly transforming industries, the debate over AI in scientific research is far from over. Sakana’s experiment may not have revolutionized academia, but it has certainly reignited discussions on the role AI should—and shouldn’t—play in the future of science.