What happens when nearly 1,000 AI agents are given their own preprint server, research laboratories, and scientific chat rooms?
They don't just write papers.
They debate experimental designs, criticize other agents' results, retract mistakes, build citation networks, and eventually create their own peer-reviewed journal.
That's what alphaXiv explored in A Society of AI Scientists, a 12-week experiment conducted from August 19 to November 12, 2026.
The video shows how this AI research network developed over time.
THE NUMBERS
• 936 autonomous AI agents.
• 35 human-operated research laboratories, with contributors from institutions including Stanford, MIT, and ETH Zurich.
• 936 published studies, accompanied by code and data.
• Approximately 9,500 messages.
• 229 chat rooms created by the agents themselves.
• One agent-founded Journal of Agent Research.
The researchers have opened the archive of manuscripts, code, data, reviews, and conversations for others to inspect.
WHAT DID THE HUMANS DO?
This wasn't a society of AI agents operating without any human involvement.
Human researchers provided computing resources, set budgets, defined broad research themes, and established baseline participation guidelines.
Within those boundaries, agents chose hypotheses, designed experiments, analyzed results, and decided what to publish.
The scientific subjects ranged from language-model training and mechanistic interpretability to continual learning and solar physics.
DID THE AGENTS ACTUALLY DISCOVER ANYTHING?
The authors highlight several findings worth investigating.
In one experiment, models appeared to transfer memorized facts about fictional people and places through unrelated training text that never explicitly mentioned those facts.
Other studies examined how much of a model's improvement could be recovered from its prompt cache and whether facts remained recoverable after deleting their original cache entries.
The team also reports examples of agents identifying errors in earlier human research and obtaining results independently before comparable human preprints appeared.
These are research claims and case studies, not proof that all 936 papers contain new discoveries.
THE AGENTS CREATED THEIR OWN JOURNAL
Perhaps the strangest part: agents in one laboratory founded the Journal of Agent Research.
They introduced editors and peer review, including a four-model reviewing panel using GPT, Claude, Gemini, and DeepSeek for different perspectives.
And there's an important lesson here.
Reviewing a convincing paper isn't the same as checking whether its experiments are correct.
The project found that running verification scripts against actual code and data could overturn conclusions that sounded reasonable in the written manuscript.
WHAT DID THE EXPERIMENT REVEAL?
The community exhibited recognizable scientific behaviors:
• Agents announced research plans and exchanged criticism.
• Some corrected experimental designs after peer feedback.
• Others withdrew mistaken claims before publication.
• Negative results were published and cited.
• An extensive citation network grew over the 12 weeks.
But the limitations are just as important.
The agents often gravitated toward familiar research questions and relatively small variations on existing benchmarks.
In a blinded evaluation using LLM judges, their papers scored higher on experimental rigor, while human-written preprints scored higher on novelty and scientific significance.
That evaluation was performed by language models, not independent human experts, and it doesn't establish that AI research is generally better than human research.
WHY THIS IS INTERESTING
A single AI research agent can generate a hypothesis and run an experiment.
But science is also a social process.
Researchers challenge one another, reproduce results, correct errors, cite earlier work, and determine which findings matter.
This experiment asks whether an entire community of autonomous agents can develop some of those behaviors.
The answer seems to be: some of them, yes.
The harder problem is separating meaningful discoveries from an enormous volume of plausible-looking papers.
And that still takes considerable human judgment.
The preprint itself has not yet undergone conventional independent peer review.
SOURCES
A Society of AI Scientists:
https://www.alphaxiv.org/abs/2611.society-ai-scientists
Public AI-agent preprint server:
https://www.alphaxiv.org/ai-track
Research archive:
https://huggingface.co/datasets/alphaXiv/AgentAlphaXiv
What do you think: could communities of AI scientists accelerate real discoveries, or are we about to create the world's most efficient academic paper factory?
#AIAgents #AIResearch #AlphaXiv
YouTube / ABVX
