14 posts tagged with "Resolver / WOF"

Type Vienna, Austria into our geocoder and, until this week, it answered with a confident set of coordinates: 39.32, −81.54. That's Vienna, West Virginia — population about ten thousand, a few miles up the Ohio River from Parkersburg. The capital of Austria is 7,500 kilometers and one ocean away, and the geocoder had no doubt whatsoever.

You can point a Nominatim client at Mailwoman now. It serves the same endpoints (/search, /reverse, /status) with the same JSON shape, so your existing code does not change. The backend is different. It needs no PostgreSQL, no osm2pgsql import that takes hours and tens of gigabytes, and no separate server, so it can ship inside an app. It answers Nominatim's queries from a SQLite file.

Our geocoder had beaten Nominatim across the United States by fifteen points, so we ran the same benchmark on Europe and expected a similar result. Europe produced a double-digit loss.

A loss to Nominatim would have been acceptable, because Nominatim covers the whole planet on community-contributed data and is the standard to beat. We could not explain the loss, though. We knew our parser was not ten points worse in Europe than in America, so we investigated what the gap consisted of.

This post covers the cause, the fixes, and the problems the fixes did not solve. For the European comparison we added a third system: Pelias, through geocode.earth. geocode.earth hosts Pelias on Elasticsearch. Pelias turned out to be the stronger competitor, and this post states where it still beats us.

Open the demo and type an address in Kabul, and the demo returns no result. It does not place a pin a few streets off or fall back to the city center, because the gazetteer has no entry for Kabul. Hong Kong, Tirana, Chişinău, and every place in the Democratic Republic of the Congo return no result as well. We had been calling this gazetteer "world coverage" for weeks, and it covered 97 of the world's ~195 countries. The other ninety-odd countries were absent from the file.

The whole geocoder runs in your browser. You type an address, you get a rooftop coordinate, and no server ever sees your query — the gazetteer it resolves against is a SQLite database sitting on a CDN, and the page reads it with HTTP range requests, a few kilobytes at a time. It's a lovely trick. We were proud of it. Then we counted the requests it took to find a single city, and the number was 243.

So the questions for the day: why does looking up one name cost 243 round trips? What goes wrong when you search a database you can only read a byte range at a time? And how do you get a global gazetteer — every country, region, county, and city we resolve against — down to about a dozen reads without putting a server back in the loop?

Overnight I taught the geocoder Canada. Or I thought I did. I pulled 843,000 Canadian postcodes, computed a centroid for every one, spot-checked the result — M5H 2N2, downtown Toronto, 43.652, −79.382, dead on — and validated the database every way I could read it: every postcode present, every coordinate right, and no other data disturbed. Green, top to bottom. This morning, before flipping it live, I asked the demo to find a Toronto address. It dropped the pin in Ohio.

So, the questions for the morning. How does a database that passes every check still land the answer 600 kilometers wrong? What does it mean to "validate" a thing? And why is Toronto in Ohio?

We turned the demo into a real geocoder. You type an address and get a rooftop coordinate, computed in your browser without a server. The last feature was autocomplete. It completes the city name while you type. We had shipped autocomplete as a command-line tool days earlier and seen it rank San Francisco above San Diego, so we considered it done. We connected the same function to the search box, typed New Yor, and it suggested Denver.

This raised three questions. Why does a function that handles San correctly fail on New Yor? How does completing a word differ from completing the word a person is still typing? How does an autocomplete that knows ten thousand cities fail to complete one of them?

Our resolver scored 93.7% on the metric we'd been quoting for months. On the same addresses, its median answer was 326 kilometers from the truth.

Both numbers are correct. That's the awkward part.

There is a particular kind of engineering misery where you fix a bug three times and it never gets better, because the bug is in your ruler. This is that story.

Our neural parser handles German two ways. Native order — Hauptstraße 5, 10115 Berlin — is the layout real German feeds and real German people use. International order — 5 Hauptstraße, Berlin, 10115 — is the Americanized layout our evaluation set happens to ship. For months, international-order German "collapsed": locality accuracy sat around 44% while native cleared 80%. We had a story for it. The postcode anchor — a side-channel that feeds the model a country hint derived from the postcode — sits at the trailing postcode. In international order that postcode lands on the far side of the locality from where it's needed. Plausible. So we retrained.

Ask a geocoder for "Berlin" and it has to make a choice. There's the one in Germany, There's also Berlin, New Hampshire (population nine thousand and change), Berlin, Wisconsin, Berlin, Connecticut, and a dozen more scattered across the United States like the name was on sale. The parser hands you the word Berlin tagged as a locality. Something downstream has to decide which dot on the map that is. How would you even know if it picked right?

For a long time our answer was a scorecard that checked the name. Did the resolved place's name equal the expected name? Tick. Move on. It is a completely reasonable thing to measure, and it was lying to us for months.