Two papers from T-Tech explore very different ways to improve language-model fine-tuning. One replaces the backward pass with evolution. The other stops reinforcement learning from forgetting less common correct answers.
The conventional solution to making AI models better is wonderfully straightforward.
Buy more GPUs.
Buy more GPUs to train a larger model.
Buy more GPUs to fine-tune it.
Then buy some more GPUs because the first lot are busy explaining to one another what the gradients are.
Naturally, I'm simplifying.
But the expense of training modern language models isn't only about performing mathematical operations.
It's also about memory.
Communication.
Synchronisation.
And making dozens or hundreds of expensive processors cooperate without spending half their time waiting for one another.
Which raises an interesting question.
What if we improved the way models learn instead of merely increasing the amount of hardware available to them?
Researchers from T-Tech, the technology division associated with T-Bank, have presented two papers at EMNLP 2026 that approach this question from quite different directions.
The first, ESSA, explores fine-tuning without gradient-based optimisation.
The second, F-GRPO, tries to solve a subtler problem: conventional reinforcement learning can make a model more likely to produce a familiar correct answer while making it less likely to discover a rarer, equally correct one.
One changes how the model is updated.
The other changes what the update rewards.
Both are interesting.
And neither requires us to pretend that backpropagation is suddenly obsolete.
First, let's get rid of the backward pass
Most modern language-model training relies on gradients.
The model produces an output.
We measure how far it is from the desired result, or calculate a reward.
Then we determine how to adjust the parameters to improve its future behaviour.
Backpropagation is the machinery that makes those calculations practical.
It's extraordinarily effective.
But it's also expensive.
Particularly when you're training across a large cluster of GPUs.
You need to store intermediate information for the backward pass, manage optimizer states and coordinate parameter updates across devices.
The more complicated the training setup becomes, the more coordination matters.
The ESSA researchers asked whether some of this could be avoided.
Their answer was to borrow an idea from evolutionary optimisation.
Instead of calculating how to improve the model, generate different candidate modifications and test which ones perform better.
It sounds almost suspiciously simple.
And in principle, it is.
Imagine having a model that can solve mathematical problems.
You create several slightly different versions.
Give them problems.
Measure their results.
Use the better-performing candidates to guide the next round of variations.
Repeat.
Eventually, you hope to discover parameter configurations that perform better than the starting point.
You haven't calculated a gradient through the model.
You've conducted a search.
Of course, there's an enormous problem with this idea.
A modern language model can contain billions of parameters.
Searching for improvements by randomly changing billions of numbers would be a spectacularly expensive way to accomplish very little.
So ESSA does something clever.
It makes the search space dramatically smaller.
Seven billion parameters. But we're only touching a tiny fraction.
The method is called Evolutionary Strategies for Scalable Alignment, or ESSA.
It builds on LoRA, Low-Rank Adaptation.
LoRA is already a useful technique for adapting large neural networks without modifying every parameter in the underlying model.
Instead, you introduce relatively small trainable matrices that represent adjustments to selected parts of the network.
This dramatically reduces the number of parameters that need updating.
But ESSA goes further.
It takes those adapter matrices and decomposes them using singular value decomposition, or SVD.
Without turning this into a linear algebra lecture, the useful idea is that a matrix can be represented through directions and their associated strengths.
ESSA freezes the directions and searches over the strengths.
The singular values.
So rather than exploring millions of ordinary model parameters, the evolutionary algorithm operates on a compact collection of values that influence the LoRA updates.
For some configurations, that search space contains only a few thousand scalars.
Compared with the billions of parameters in the underlying language model, it's a remarkable reduction.
This is where claims about optimising roughly a million times fewer parameters come from.
But the comparison needs a qualification: that ratio is against the size of the full model, not necessarily against the trainable parameter count of every LoRA-based baseline.
And there's another important detail.
ESSA doesn't begin with completely untrained adapters.
The researchers first perform a short supervised fine-tuning stage using ordinary training methods.
That gives the adapters a useful starting point.
ESSA replaces the subsequent online alignment stage, not the entire training pipeline.
An important distinction, particularly if someone is planning to announce the death of gradients on LinkedIn.
Evolution is surprisingly convenient when you have 128 GPUs
The nice thing about evolutionary search is that its experiments are largely independent.
One GPU can evaluate one candidate.
Another GPU can evaluate another.
A third can evaluate something completely different.
They don't have to exchange enormous gradient tensors after every training step.
Instead, the workers can communicate compact information such as random seeds and evaluation rewards.
The optimiser uses that feedback to update the distribution from which candidate parameter changes are sampled.
In ESSA, the researchers use an evolutionary strategy based on CMA-ES.
In other words, the method is more sophisticated than simply keeping the best model and making random copies of it.
It learns which regions of the small parameter space are promising.
And then searches those regions more effectively.
This makes the approach naturally parallel.
It can also work with quantised models using INT4 or INT8 representations, because the evolutionary evaluation only requires forward inference.
That's a significant engineering advantage.
It reduces the memory requirements of the alignment stage and makes it easier to distribute evaluations across devices.
The researchers tested the approach on models including Qwen2.5 and Llama, with experiments extending to models containing tens of billions of parameters.
The results were particularly interesting in mathematical reasoning.
In the reported comparisons, ESSA achieved performance competitive with gradient-based LoRA-GRPO, and in several settings performed better.
But the dramatic result was the training time.
On Qwen2.5-32B, aligned using PRM800K and evaluated at a fixed MATH500 accuracy, the published EMNLP results report that ESSA reached the target quality up to 7.8 times faster than LoRA-GRPO on 128 GPUs.
On 16 GPUs, the advantage was approximately twofold.
The improvement grew as the cluster became larger.
Which is exactly what you would expect from a method that requires far less inter-device coordination.
The paper’s measured breakdown also points to the fixed global batch becoming underfilled, additional policy and actor computations, and synchronization and idle time in LoRA-GRPO. Gradient all-reduce alone is not the whole explanation.
It's also why the result matters.
Not because somebody has proved that evolutionary algorithms are universally better than backpropagation.
They haven't.
But because the work shows that, for a sufficiently constrained adaptation problem, avoiding the backward pass can offer a substantial practical advantage.
The bottleneck isn't always how much mathematical work a GPU can perform.
Sometimes it's how much time the GPUs spend coordinating that work.
But what if the model is learning to become less imaginative?
The second paper tackles a different problem.
And I find it perhaps even more interesting.
Imagine asking a language model to solve a mathematical problem.
There might be several valid ways to approach it.
A short algebraic solution.
A geometric argument.
An unusual substitution.
A longer but perfectly correct derivation.
Initially, the model may have some probability of generating each approach.
Then we fine-tune it using reinforcement learning.
We reward correct solutions.
And the model becomes increasingly likely to produce successful answers.
Excellent.
Except there's a trap.
The model may learn to favour the correct solutions it already generates frequently, while neglecting rarer correct solutions.
It becomes better at doing what it already knows.
And potentially worse at exploring the alternatives.
This is particularly relevant to reinforcement learning with verifiable rewards.
In mathematics, we can often check whether a final answer is correct.
So the training process generates several candidate solutions, evaluates them and uses their relative rewards to update the model.
One popular algorithm for this is GRPO, Group Relative Policy Optimization.
The problem is that we can't sample every possible answer.
We typically work with relatively small groups of generated responses.
And if a rare but correct strategy doesn't appear in those samples, the training update has no direct positive evidence for it.
The common correct approaches receive reinforcement.
The rare ones may lose probability.
Even as the model's overall chance of generating a correct response increases.
That's a rather uncomfortable possibility.
We might be making the model more reliable while narrowing the set of solutions it can readily discover.
Don't reward the obvious quite so aggressively
The second method is called F-GRPO.
Its full paper title is wonderfully direct:
Don't Let Your Policy Learn the Obvious and Forget the Rare.
The proposed modification is relatively simple.
When the model already solves a particular problem frequently, reduce the strength of the corresponding training update.
If a sampled group has a high success rate, we don't need to reinforce that already-common behaviour quite so aggressively.
This leaves more room for learning from problems where successful solutions are less frequent.
The method draws inspiration from focal loss, an approach originally designed to reduce the relative influence of easy training examples.
The researchers introduce a difficulty-aware scaling coefficient that modifies the advantage signal in group-relative reinforcement learning.
Importantly, F-GRPO still uses gradient-based optimisation.
Unlike ESSA, it isn't an attempt to remove backpropagation.
It improves the learning objective rather than replacing the optimisation machinery.
And the idea is applicable beyond GRPO itself.
The researchers also evaluated related versions for DAPO and CISPO.
The principle remains the same.
Stop spending so much learning effort reinforcing what the model already does confidently.
Pay more attention to the parts of the solution space that are at risk of being neglected.
A correct answer among 256 attempts
The researchers evaluated F-GRPO using Qwen2.5-7B on mathematical reasoning benchmarks.
One of the most useful metrics here is pass@256.
Imagine giving the model a mathematical problem and sampling 256 possible solutions.
Pass@256 measures whether at least one of those attempts produces a correct answer, averaged over the evaluation problems.
It is not the probability that a single randomly generated answer will be correct.
That's closer to what pass@1 measures.
The distinction matters enormously.
A model might become very good at repeating one conventional method.
Its pass@1 could improve.
But if the task requires an unusual approach, it may struggle to find one even after many attempts.
A stronger pass@256 suggests better coverage of successful possibilities under that sampling budget.
And this is where F-GRPO produced its headline result.
- GRPO: average pass@256 increased from 64.1% to 70.3%.
- DAPO: average pass@256 increased from 69.3% to 72.5%.
- CISPO: average pass@256 increased from 73.2% to 76.8%.
These are the reported averages on mathematical benchmarks using Qwen2.5-7B, with a training group size of eight.
The GRPO improvement is 6.2 percentage points.
And according to the researchers, it required no increase in the number of sampled training rollouts or the computational budget.
The method also preserved or improved pass@1 in the tested configurations.
That's important.
The objective wasn't simply to make the model produce more varied nonsense.
It was to retain access to less common successful responses without sacrificing the reliability of ordinary answers.
There is still a caveat.
A higher pass@256 does not prove that a model has acquired completely new reasoning strategies.
It measures improved success coverage under repeated sampling.
The researchers' analysis offers an explanation involving rare correct trajectories, but the metric by itself doesn't tell us exactly which internal reasoning mechanisms changed.
Still, it's a meaningful result.
Sometimes making a model better doesn't mean teaching it something entirely new.
It means preventing the training process from making it less capable of using what it already knows.
Two papers, two very different ideas
What I like about these papers is that they attack separate limitations in the conventional training process.
ESSA asks:
Do we really need to compute gradients through the model for this stage of fine-tuning?
F-GRPO asks:
Even if we use gradients, are we reinforcing the right behaviours?
The first is about optimisation efficiency.
The second is about the distribution of learned behaviour.
One reduces the amount of coordination and parameter adjustment required.
The other tries to prevent training from overconcentrating probability on common successful solutions.
And both suggest that there is still substantial room for progress without simply increasing model size.
There's another interesting connection here.
Recently, I wrote about Q Labs' Dust, another attempt to train transformers without conventional backpropagation.
But Dust and ESSA solve the problem differently.
Dust perturbs internal activations and uses the resulting loss changes to estimate gradients.
ESSA searches directly over a compact set of adapter parameters using evolutionary strategies.
Both reduce dependence on a conventional backward pass during the stage they address.
But they aren't the same algorithm, and they shouldn't be treated as interchangeable demonstrations of a universal backpropagation replacement.
In fact, the growing variety of approaches is precisely what makes the field interesting.
For years, the dominant narrative around better AI has focused on scale.
Larger models.
More data.
More compute.
More reinforcement-learning rollouts.
And certainly those things matter.
But now we're seeing more attention paid to the structure of the learning process itself.
Where should updates happen?
How much information needs to move between devices?
Which behaviours deserve reinforcement?
And what useful capabilities might be lost while optimising the most immediately measurable reward?
These are algorithmic and engineering questions.
Not merely purchasing decisions.
The limitations are worth remembering
Neither paper establishes that conventional fine-tuning should be abandoned.
ESSA relies on a suitable supervised fine-tuning initialisation and a deliberately restricted optimisation space.
That restriction makes evolutionary search practical.
It also limits how much of the model can change.
The method isn't a general demonstration that a giant pretrained model can be trained from scratch using evolutionary search at a fraction of the normal cost.
Its reported speedups also depend on specific models, benchmarks, hardware arrangements and baseline implementations.
A 7.8-fold improvement in one distributed alignment experiment is not a promise that every fine-tuning job will become eight times faster.
F-GRPO has a different limitation.
Its results are particularly relevant to reinforcement learning with verifiable rewards, such as mathematics, where we can determine whether an answer is correct.
The approach is attractive because it can modify existing training algorithms without requiring more sampled responses.
But preserving rare correct outputs in mathematical benchmarks isn't automatically the same as improving general-purpose creativity, scientific discovery or open-ended agent behaviour.
Those would require further evaluation.
So the correct conclusion isn't that we've discovered two universal upgrades for every language model.
It's that two specific weaknesses in established training methods can be addressed with surprisingly focused interventions.
And that's still worth paying attention to.
Maybe the next breakthrough isn't another warehouse full of GPUs
What I find most appealing about these results is their underlying attitude.
Instead of accepting the existing training pipeline as something fixed, the researchers ask which parts of it are actually necessary.
Do we need the backward pass at every stage?
Do we need to update so many parameters?
Do we need to spend the same training effort on easy and difficult problems?
And do we really want to reward common correct answers so strongly that less common successful approaches become harder to discover?
These are the kinds of questions that can produce substantial improvements without requiring an entirely new generation of hardware.
Of course, better algorithms won't eliminate the appetite for more computation.
History suggests quite the opposite.
Make something cheaper and people usually find considerably more ambitious things to do with it.
But that's not a reason to ignore efficiency.
Especially when the efficiency improvement comes with a better understanding of what the learning process is actually doing.
ESSA makes a case for searching over a carefully chosen set of parameters rather than calculating gradients throughout the model.
F-GRPO makes a case for paying attention to the successful answers that conventional training can unintentionally leave behind.
One makes the training machinery lighter.
The other tries to make the resulting behaviour less narrow.
And together they offer a rather useful reminder.
Improving AI isn't always about giving it more resources.
Sometimes it's about asking whether we're using the resources, and the model's existing capabilities, particularly intelligently.