The model knew it was Austria. The resolver sent it to West Virginia.
Type Vienna, Austria into our geocoder and, until this week, it answered with a confident set of coordinates: 39.32, −81.54. That's Vienna, West Virginia — population about ten thousand, a few miles up the Ohio River from Parkersburg. The capital of Austria is 7,500 kilometers and one ocean away, and the geocoder had no doubt whatsoever.
It does this with Sydney too (a hamlet in Ohio), and Zurich (Kansas), and Toronto (also Ohio, as it happens). The pattern is always the same: you name a famous foreign city, you name the country it's in, and the geocoder hands you back its smallest American namesake like it's doing you a favor.
The fix that felt like progress
When you find a bug like this, the first idea is almost always the expensive one. Ours was: the country router doesn't know enough countries. Teach it that "Austria" means Austria — feed it more examples, retrain the model, ship a new set of weights. It's a satisfying plan. It feels like progress, because training a model is the kind of work that looks like work.
We had the GPU budget queued and the issue written up as a model gap. And then, because we've been burned before by fixing the wrong layer, we did the cheap thing first: we asked the model what it saw.
Here is Vienna, Austria straight out of the parser:
Vienna → locality
Austria → country
The model knew. It tagged Austria as a country, directly, every time — Australia, Switzerland, Canada, all of them. There was no model gap. The thing we were about to spend a retrain on was already working perfectly. The bug lived one layer down, in the part of the system that takes a correct parse and turns it into a place on Earth.
Forty-five Viennas and a popularity contest
The resolver's job is to take the word "Vienna" and decide which Vienna. Our gazetteer knows about forty-five of them in the United States, plus the real one in Austria, plus a few others. Faced with that pile, the resolver did the reasonable-sounding thing: it picked the most prominent match by population and name — and then it threw the country token away. It never asked the one question the address had already answered for it.
So Vienna resolved on its own merits, the populous American candidates outranked the Austrian one in the bag, and Austria sat there in the parse, ignored, like an unread return address. The model had said which country in plain text, and the resolver wasn't listening.
The direct version of the bug: the parse and the resolve each made a locally sensible decision, and no stage checked that they agreed.
The fix we didn't make
There's a tempting shortcut here, and we've watched it sink geocoders before. You could keep a list. "If the address contains Austria, force the country to AT." It would work for Vienna, and Sydney, and Zurich. It would also be the first paving stone on the road to a thousand-entry exception table that someone maintains forever and no user trusts — the same road that turned older rules-based geocoders into archaeology. We don't pin, and we don't keep lists of special cases the model is supposed to defer to.
The fix instead asks the parse and the resolve to be consistent with each other. When the address includes an explicit country and the locality it resolved to lives somewhere else, re-pick the locality to the one that sits inside the country the address states. The country code comes from the model's own country emission, normalized through an ISO table we already had. The gazetteer's own country column does the rest. No list. No prior. It generalizes to every country at once, because it isn't about any particular country — it's about the address agreeing with itself.
The fix has a guard rail, because this kind of fix typically goes wrong at that point. It only fires when the country is the most specific geographic evidence in the address. Write Springfield, IL, USA and the state is right there doing the disambiguating, so the new pass stays out of the way and Springfield stays in Illinois — not the largest Springfield in America. The fix warrants its keep precisely by knowing when not to act.
What it bought, and what it cost
We measure these things against coordinates rather than labels, because a name-match metric would have told us this bug didn't exist. The direct yardstick is a bare City, Country query — the exact thing a user types into the box — scored by how close the answer lands to the real place.
| before | after | |
|---|---|---|
City, Country resolves correctly | 54.2% | 77.9% |
| countries the country-name now rescues | — | 45 of 57 |
Forty-five countries that used to need a hint and now don't, a twenty-four-point jump on the query shape people use, and the cost on the GPU meter was zero. The thing we'd budgeted a retrain for was a hundred-odd lines in the resolver.
The fix does have a direct cost: it can only re-pick to a place the gazetteer has. Where the gazetteer lacks a city, the country in the address can't summon a place the gazetteer lacks, and the fix correctly declines to guess. That's a coverage problem, and it's a different change for a different day.
The lesson
We almost retrained a model to fix a bug the model didn't have. The diagnostic that saved us was thirty seconds of parsing seven addresses and reading the output — cheaper than the issue we'd written, never mind the GPU hours behind it.
The reflex is to train. The discipline is to look first. The model knew it was Austria the whole time. We had to teach the rest of the system to listen.
(One more for the honesty file: the test that proves Sydney now lands in Australia failed the first time I ran it — by 7,532 kilometers. The fix was fine. I'd dropped the minus sign on Sydney's latitude and parked it in the northern hemisphere. The check caught my typo before it caught a real regression. That is exactly the order you want.)