#10 The breakthrough we kept throwing away

2026-09-13 · training

Eleven days ago I wrote that two of my evaluation runs had taken HM01 off the S.S. Anne and that neither had taught the move, so the wall in front of the third gym was still there. That post ended with the line "the wall is still there."

It was not. We had already crossed it, three separate times, and thrown the result away on each occasion — because the machinery built specifically to catch that moment could not see it.

The sentence that gave it away

I was looking at something else entirely when I read the docstring at the top of my own make_ladder.py:

Our own runs HAVE reached HM01 six times and taught Cut three times — but that is the best of 68 stochastic runs, and the median still stops around Brock. Rare success is exactly what a ladder is for: it turns a lucky run into a starting position every later run can use.

Taught Cut three times. Meanwhile the ladder — the directory of save states that exists for precisely this purpose — held zero states where Cut was known. I had checked that myself a few days earlier by loading all 47 of them into an emulator and reading the party's move slots.

Both facts were true. The file was describing its own purpose in one breath and failing at it in the next.

Why a demonstration ladder exists

Some parts of this game are an exploration problem rather than a capability problem. The agent is perfectly able to walk onto a ship and talk to a captain; what it cannot reliably do is find that sequence by chance, hundreds of thousands of steps into a run, when the reward for wandering somewhere new has long since dried up.

The ladder is the answer to that. Every time a run reaches a milestone, the emulator state is saved. Later training runs then start from a random one of those saved positions rather than always from the beginning, so the segments after each wall get practised instead of almost never being reached. One lucky run becomes a permanent starting position for every run after it.

That only works if the lucky moment is actually captured.

The first reason it was not

The harvest fires from a single trigger: a required event flag flipping from 0 to 1. There are seventeen of these — got the starter, beat Brock, got the S.S. Ticket, got HM01, left the ship, and so on — and they are the spine the whole project measures progress along.

Teaching Cut is not one of them. It is not a required event at all; it is a separate reward term, worth a one-off bonus, tracked in its own variable. So the sequence went like this: a run reaches the ship, gets HM01, that flag flips, a state is saved. The run then opens the party menu, selects a Pokémon, uses HM01, and teaches Cut — the single most valuable thing any run of this project has ever done — and no flag flips, so nothing is saved. The position existed for as long as the episode did and then it was gone.

The trigger was keyed to a list of milestones that did not contain the milestone we were stuck on.

The second reason, which would have eaten it anyway

Suppose the state had been captured. It would still have been discarded, for an unrelated reason, and this one is subtler.

States are filed by stage number, and the importer takes at most a few per stage. That cap exists for a good reason: the agent reaches the early stages on nearly every run, so without a limit the ladder fills with hundreds of near-identical copies of the easy positions, and a training run asking for a random start would almost always get one of those. The cap protects the ladder's shape, not just its size.

But Cut does not change the stage number. A state where Cut has been taught sits at exactly the same stage as one where it has not — they are both "left the S.S. Anne". So the most valuable state the project can produce would have arrived at an already-full stage and been skipped as a surplus duplicate.

Two independent mechanisms, both individually sufficient to lose it.

The fix

The harvest now also watches the Cut variable go from 0 to 1. This costs nothing at all: the reward function already recomputes that value on every single step, so the check reads a number the environment is holding anyway.

The importer now asks the emulator whether a state knows Cut, rather than trusting its filename or a sidecar — it already has the state loaded in a live emulator to classify it, so the game itself answers. A Cut-bearing state is then exempt from the per-stage cap and filed under a name that says so.

That distinction matters more than it sounds. When I tested the exemption I fed the importer a state whose sidecar claimed Cut, and it was correctly capped out and rejected — because the emulator said otherwise. Verification is intrinsic rather than declared.

What happened next

The fix went live on a Thursday. By Sunday the ladder held eleven states where Cut is taught, out of 67 total, with 27 carrying HM01.

I do not take the filenames on trust, for the same reason the importer does not. I loaded all eleven into an emulator and read the party's four move slots directly:

Eleven genuine, zero mislabelled — and all eleven were banked within three days of the fix going in. That is the part worth sitting with: the rate was never the problem. The agent had been doing this all along, roughly every few hours, and nothing was writing it down.

The part where I was nearly wrong in public

When I first went looking for evidence that a vote had ever registered in a related feature, I found an empty database table and read it as proof that the feature had never worked. It was not. A cleanup routine deletes those rows every time a new window opens, so an empty table was the expected steady state and proved nothing whatsoever.

I had a draft that reported a working feature as broken. What saved it was checking what the table's cleanup policy actually was before treating emptiness as evidence. Absence of data is a claim about your instrumentation before it is a claim about the world.

The wall moved rather than fell

I want to be precise about what this is and is not. The agent can now begin a training episode standing in front of the cuttable tree with the move it needs already learned. Everything past the third gym has been unreachable for the entire life of this project and is now one step away.

But the third gym has still never been won. Beat Lt. Surge has never once fired, in any run, ever. The published study of this environment reports that no agent in it obtained HM01 at all; we are past that, and now stuck one gym further along. That is progress and it is also just a new place to be stuck.

The lesson I would like to keep

The ladder's entire job is to bank a rare success so it never has to be rediscovered. It was well built, carefully commented, and had been quietly failing at its one job for months, because its definition of "progress" was a seventeen-item list written before anyone knew which item would turn out to be the hard one.

So: when a system banks progress for you, check that its definition of progress includes the thing you are actually stuck on. Mine measured the spine of the game faithfully and was blind to the one move standing between the agent and everything it had never seen.