A 109-million-parameter model has beaten a 72-billion-parameter language model on a test of shared representations. The interesting part isn't the 660-fold size difference. It's what the smaller model was taught to recognize.
Yann LeCun has been saying something rather inconvenient about artificial intelligence for years.
Making language models bigger doesn't automatically mean they're learning to understand the world.
More parameters.
More text.
More computing power.
More expensive data centres.
All very impressive.
But LeCun has consistently argued that predicting the next word in a sentence is not necessarily the best route towards building systems that understand reality.
This hasn't made him particularly fashionable in an industry where scaling has become something approaching a religion.
The larger the model, the greater the intelligence.
Or at least, that's the simplified version presented in countless investment decks and AI announcements.
Then, on October 29, 2026, LeCun and Randall Balestriero published a paper with a rather entertaining result.
A text model with 109 million parameters achieved better alignment with visual representations than language models containing up to 72 billion parameters.
That's approximately 660 times fewer parameters.
A result that deserves attention.
Although before anybody declares the death of large language models, there's a crucial detail.
The smaller model didn't outperform Qwen2.5-72B in general reasoning, mathematics, coding or conversation.
It performed better on a specific measure of how closely its internal representation of information corresponds to that of models trained on images.
Which is a rather different thing.
And, in some ways, a more interesting one.
The argument that helped send LeCun out of Meta
LeCun's disagreement with the direction of mainstream AI research is hardly new.
He has spent years arguing that language alone is an insufficient foundation for the kind of intelligence required to understand and interact with the physical world.
Humans don't learn everything by reading.
We observe.
Move.
Touch.
Compare.
Predict.
Experience consequences.
We discover that objects continue to exist when we're not looking at them.
That a glass will fall if pushed from a table.
That a cat photographed from the front and a cat photographed from behind can be the same cat.
Language can describe these things.
But language isn't the thing itself.
LeCun's preferred research direction involves learning abstract representations of the world, including through predictive architectures rather than relying exclusively on the next-token objective used to pretrain most modern language models.
In November 2025, he announced that he would leave Meta at the end of the year.
He subsequently launched AMI Labs, a Paris-based initiative focused on advanced machine intelligence and world models.
Meta, meanwhile, has continued investing heavily in large-scale generative AI.
It's tempting to turn this into a dramatic story about one researcher standing against an entire industry.
But that would be too simple.
LeCun isn't the only researcher interested in alternative training objectives.
And large language models have achieved things that would have seemed remarkable only a few years ago.
Still, his central question remains worth asking.
Are we teaching models to understand the underlying world, or merely to become exceptionally good at predicting the information used to describe it?
The new paper offers an intriguing way to investigate that distinction.
Plato enters the machine-learning laboratory
In 2024, researchers Minyoung Huh, Brian Cheung, Tongzhou Wang and Phillip Isola proposed what they called the Platonic Representation Hypothesis.
It's a wonderfully philosophical name for a mathematical idea.
The hypothesis suggests that neural networks trained on different forms of information may gradually develop increasingly similar internal representations of reality.
Imagine two models.
One has learned from photographs.
The other has learned from text.
They haven't necessarily been trained together.
They haven't been given identical tasks.
Yet both encounter information about the same underlying world.
Dogs.
Buildings.
Trees.
Vehicles.
People.
Relations between objects.
Perhaps, as the models become larger and more capable, their internal representations begin to converge.
Not because their architectures are identical.
But because both are attempting to capture something about the same reality.
In a loose philosophical sense, they might be approaching a common representation of the world beneath its different appearances.
Hence Plato.
The 2024 paper presented evidence that larger models often exhibit stronger agreement across different modalities.
But there was an important missing piece.
Why should this convergence happen?
And is scale actually the essential ingredient?
Those are the questions Balestriero and LeCun decided to examine.
Their new paper is called Proving and Improving the Platonic Representation Hypothesis with JEPAs.
A title unlikely to attract a large audience outside machine-learning research.
Which is unfortunate.
Because the central idea is surprisingly accessible.
The difference between learning words and learning what the words describe
Imagine you have a photograph of a dog sitting beside a bicycle.
You could describe the picture in several ways.
A dog is sitting next to a bicycle.
A brown animal rests beside a parked bike.
A pet waits near a two-wheeled vehicle.
The sentences are different.
The words are different.
The grammar is different.
But they're describing the same underlying scene.
Now suppose you train a model to recognise that these descriptions belong together.
To produce similar internal representations for them.
What information can the model rely on?
Not necessarily the precise wording.
Not the order of the words.
Not whether one description says dog and another says pet.
To solve the task, it benefits from identifying the underlying content that the descriptions share.
This is the central intuition behind the paper's approach.
The authors call such pairs good views.
Different observations of the same scene.
Now compare that with another training task.
Take one document.
Select one paragraph.
Then another paragraph from the same document.
Or take a sentence and ask the model to predict its continuation.
These pieces of text may be related.
But their relationship doesn't necessarily require the model to identify one common scene in the physical world.
They can share vocabulary, writing style, grammatical conventions or document-specific information.
The model can learn useful things.
Certainly.
But it may also learn a great deal about how information is expressed rather than what the information ultimately describes.
The authors call this distinction central to the efficiency of representation learning.
And it's where next-token prediction comes under scrutiny.
The problem isn't that next-token prediction doesn't work
Obviously, next-token prediction works.
We have rather substantial evidence.
Modern language models can write code, solve mathematical problems, analyse documents and assist with increasingly complex tasks.
Dismissing these capabilities would be ridiculous.
But a method can be effective without being maximally efficient for every objective.
The paper argues that next-token prediction is an inefficient route towards the particular kind of shared representation measured by the Platonic Representation Hypothesis.
The authors develop a mathematical framework in which next-token prediction can be interpreted as a form of predictive learning using comparatively poor views.
The relevant pieces of information are connected through the structure of a document and its continuation.
That's different from explicitly linking multiple descriptions of the same scene.
The distinction sounds subtle.
But the experimental results suggest that it matters enormously.
Instead of asking only whether a model can predict the next word, the researchers explore whether it can learn to represent the underlying content common to several observations.
And for this they use a family of architectures called JEPAs.
JEPA: predict meaning, not necessarily every detail
JEPA stands for Joint-Embedding Predictive Architecture.
The broad idea is to learn representations that preserve useful relationships between different views of information.
Rather than reconstructing every pixel or predicting every token, a JEPA learns in a representation space.
For example, two descriptions of the same scene should produce compatible embeddings.
The model is encouraged to capture what they have in common.
Of course, there's an obvious problem.
You could make every representation identical.
Then every pair would match perfectly.
A wonderful result.
And a completely useless model.
So JEPA training requires mechanisms that prevent this trivial collapse and preserve meaningful differences between unrelated examples.
The researchers use such a mechanism in their experiments.
But the critical choice is simpler than the mathematics behind it.
Which observations are we telling the model belong together?
If those observations genuinely refer to the same underlying scene, the model has a useful reason to learn scene-related information.
If the observations are related only because they are fragments of the same document, the learning signal may be much less direct.
And that's what the authors tested.
Forty-two times fewer training examples
The researchers compared training approaches using the same 109-million-parameter text encoder architecture and the same underlying corpus.
In one approach, the model learned from good views: different textual descriptions associated with the same source image.
In another, it learned using next-token prediction.
The evaluation measured how well the resulting text representations aligned with representations produced by a separate image encoder.
The results were striking.
The good-view method reached an alignment score of approximately 0.109 after 21.6 million training examples.
The next-token baseline reached a similar score, approximately 0.111, after 900 million examples.
That's about 42 times more training examples.
The difference is remarkable.
But it needs a qualification.
A training example doesn't necessarily contain the same number of tokens under each method. In this experiment, the good-view method used more text per example.
The authors report an advantage of approximately 14 times in processed text tokens.
And neither figure should be interpreted as a measured 42-fold reduction in GPU time or training cost.
The training configurations also differed in some optimisation settings.
Nevertheless, the conclusion is difficult to ignore.
For this particular objective, the relationships between examples mattered enormously.
The smaller model didn't need to discover those relationships indirectly from hundreds of millions of next-token predictions.
They were built into the learning task.
The 109-million-versus-72-billion comparison
Then comes the headline result.
The researchers trained the good-view text model on descriptions associated with a very large collection of web images.
The model itself remained relatively small.
109 million parameters.
Then they compared its internal representations with those of several much larger language models.
The evaluation examined agreement with an image encoder such as DINOv2.
On the WIT-1024 benchmark, the reported results included:
| Model | Parameters | Representation alignment |
|---|---|---|
| Text JEPA, good views | 109M | 0.184 |
| Qwen2.5-72B | 72B | 0.154 |
| Llama-3-70B | 70B | 0.151 |
| CLIP text encoder | Smaller model | 0.124 |
| Next-token baseline, same text architecture | 109M | 0.111 |
The metric measures agreement between local neighbourhood structures in text and image representation spaces.
In other words, it asks whether examples considered similar by the text model tend to be considered similar by the image model.
Higher is better.
The small JEPA performed better than every language model evaluated, including models hundreds of times larger.
That's impressive.
But let's be precise about what it means.
It doesn't mean the JEPA is smarter than Qwen2.5-72B.
It isn't a 109-million-parameter replacement for a modern conversational assistant.
It doesn't demonstrate superior general reasoning.
And it doesn't establish that the model has acquired human-like physical understanding.
It demonstrates that a carefully chosen training objective can produce stronger cross-modal representational alignment than vastly greater model scale.
On the benchmark designed to measure that property.
Which is a much narrower claim.
But also a much more useful scientific one.
The most interesting trick: the text model never needed to see the photographs
Here's the detail I particularly like.
The text JEPA was trained on descriptions grouped by their source images.
The images established which descriptions referred to the same scene.
But the text encoder itself learned from text.
It wasn't jointly trained as a conventional image-and-text model.
It didn't need to process each photograph alongside its caption during the main representation-learning procedure.
Instead, the important information was the relationship between the descriptions.
These sentences refer to the same thing.
That's an extremely useful training signal.
And it suggests something broader.
A dataset isn't valuable only because of the quantity of information it contains.
It's also valuable because of the relationships it preserves between observations.
Two independent descriptions of the same event.
Different sensor readings of one physical process.
Several observations of the same object from different angles.
Different measurements referring to one underlying state.
These relationships may teach a model something that would be expensive to infer from disconnected examples.
Of course, the study also has an important limitation here.
Some of the large-scale descriptions were generated using LLaVA, a vision-language model whose underlying components include an image encoder already trained with image-text relationships.
So this isn't proof that the entire experimental pipeline learned everything from raw, independently collected observations without any influence from previous multimodal training.
The researchers acknowledge this.
But the result still demonstrates that the structure of the training pairs can be a remarkably powerful source of information.
What does a mathematical proof actually prove here?
There's another part of the paper that deserves attention.
The authors don't merely report experimental results.
They formulate what they call a Platonic Representation Theorem.
Under specified mathematical assumptions, they show that separately trained JEPA encoders can learn the same underlying representation, up to a rotation.
That last phrase is important.
Imagine two coordinate systems describing the same arrangement of objects.
One coordinate system is rotated relative to the other.
The numerical coordinates differ.
But the underlying geometric relationships are preserved.
A rotation can bring them into alignment.
The theorem shows conditions under which different encoders can arrive at that kind of shared structure.
But mathematical theorems come with assumptions.
And this one is no exception.
The analysis uses an idealised setting with particular assumptions about the world, observations, positive pairs and optimisation.
Real-world data is messier.
Observations can be incomplete.
Descriptions can omit important information.
Training can fail to reach ideal solutions.
And different modalities may contain genuinely different kinds of information.
So the paper does not prove that all sufficiently well-trained AI systems necessarily discover one universal, perfect representation of reality.
It proves something more specific about a class of learning objectives under defined conditions.
The experiments then test whether the associated principles are useful in practical models.
That's a meaningful combination of theory and evidence.
But it isn't the final philosophical proof that artificial intelligence understands Plato's world of ideal forms.
We can probably leave that announcement for another week.
Can these representations actually do anything useful?
A fair question.
Representation alignment is interesting to researchers.
But ultimately, we'd like to know whether it helps build useful systems.
The authors provide two demonstrations.
In one, they connect a separately trained image encoder to a text decoder through a relatively simple learned mapping.
The resulting system can generate captions for photographs.
It achieves a CIDEr score of 0.838 on COCO.
That doesn't make it the best image-captioning model available.
Nor does it mean no image-text pairs were used anywhere: the mapping was fitted using 30,000 paired examples.
But it demonstrates that separately learned representations can become compatible enough to support a practical task without jointly retraining the entire system.
In another experiment, the researchers examine image classification with no paired examples used to learn the alignment.
They report approximately 64% top-five accuracy on ImageNet-100, compared with a previously reported maximum of 56% among the language-model approaches evaluated in that setting.
These demonstrations are preliminary.
But they're interesting because they suggest that the learned correspondence isn't merely a numerical curiosity.
It can support information transfer between different representations.
Which is, after all, one of the reasons we'd want such representations in the first place.
Is LeCun right?
Here's where I think the discussion needs to become less theatrical.
The paper provides strong evidence for a particular argument LeCun has been making.
Scale alone is not the only path to learning useful representations of the world.
And in some settings, a better learning objective can be substantially more efficient than increasing model size.
That's important.
But it would be a mistake to interpret the paper as proof that next-token prediction is useless.
Or that large language models have reached a dead end.
Or that a small JEPA can already perform all the tasks handled by a frontier model.
None of those conclusions follows from the experiments.
The research also leaves open difficult questions.
How does the approach scale to more complex environments?
What happens when observations are noisy or incomplete?
Can the representations support long-term planning and action?
How much of the result depends on the particular construction of the training data?
And can the approach deliver similarly strong advantages on objectives beyond cross-modal alignment?
Those questions matter.
Because world representation and useful intelligence are related, but they aren't identical.
Still, I think LeCun's broader criticism has become harder to dismiss.
The industry's extraordinary success with large language models may have encouraged it to treat a particular training method as more universal than it actually is.
And when a method works spectacularly well, it's natural to keep improving it.
Until somebody asks whether we're optimising the right thing.
The really interesting conclusion isn't about 660 times fewer parameters
For me, the most useful takeaway isn't the size comparison.
Although it's certainly a good headline.
It's the idea that the structure of the training signal can matter more than the brute quantity of examples or parameters.
We've seen several interesting research directions recently that challenge different assumptions in model training.
Q Labs' Dust explores learning without conventional backpropagation by perturbing activations.
ESSA investigates evolutionary optimisation over a compact set of adapter parameters.
F-GRPO attempts to preserve less common correct solutions during reinforcement learning.
And now Balestriero and LeCun show how choosing better relationships between training examples can dramatically improve representation alignment.
These aren't interchangeable solutions.
They address different problems.
But together they illustrate something I find encouraging.
There's still enormous room for innovation in how we train AI, not simply in how large we can make it.
Sometimes we can improve the optimisation method.
Sometimes the training objective.
Sometimes the selection of data.
And sometimes, apparently, the decisive difference is recognising that two different sentences are describing the same dog.
Which is almost absurdly simple when put that way.
But that's often what makes good research interesting.
The explanation sounds obvious after somebody has done the difficult work of demonstrating it.
Bigger isn't always better. Better isn't always bigger.
LeCun has been criticised for his scepticism about large language models.
Sometimes fairly.
Sometimes because questioning the dominant approach can be mistaken for denying the considerable progress it has already produced.
I don't think we need to choose between celebrating that progress and investigating its limitations.
Large language models are extraordinary.
They have changed what software can do.
They're becoming more capable at reasoning, coding, planning and interacting with tools.
But that doesn't establish that predicting the next token is the optimal route to every kind of intelligence.
Nor does it mean that the next major breakthrough must come from another enormous model trained on even more data.
Perhaps it will.
Perhaps it won't.
The new paper suggests that, at least for learning representations shared across modalities, showing a model the right relationships can be more valuable than asking it to process vastly more examples without those relationships.
That's a useful finding.
And a useful reminder.
When a smaller system achieves a better result, the right question isn't necessarily how it managed to overcome its lack of size.
Sometimes the right question is:
What did we teach the larger system that it never actually needed to learn?
And what did we fail to teach it that mattered?