14 posts tagged with "Resolver / WOF"

For about a month our resolver used one method. It takes a postcode, finds the city polygon that contains the postcode's centroid, and reads off the city. The method got the Netherlands to 95% and Germany to 93%. Then we ran it on Japan, and Japan's gazetteer data has no city polygons.

This post covers two countries where the gazetteer data is sparse, what a geocoder can still do there, and where it stops. We resolved Japan to 94% by replacing the polygon lookup with a postal-authority lookup. Korea had the opposite problem. We could place the coordinate correctly for every postcode, but we could not confirm the name of the place most of the time.

The work raised three questions:

• What do you do when the gazetteer gives you points where you expected shapes?
• Does the method that worked for Japan generalize, or did it work only once?
• What happens when the map is missing data for the part of a country you most need?

We set out to fix a small problem in our address parser and measured a number that told us not to build the larger change we had planned.

The small problem is this. When our postcode extractor sees a five-digit run, it has to decide whether the run is a real postcode or a house number that looks like one. It checks the neighboring words against every country's street vocabulary we know (American, German, and French) at once. This works with three countries. At twenty countries, a German street suffix starts matching an English word by coincidence. We looked for a way to tell the extractor which country's words to check.

That question leads to a larger one, and this post is mostly about the larger one.

Our neural address parser passes 20.7% of our test suite. The rule-based parser it's meant to replace passes 93.7%. By that scoreboard, we should delete the neural model and go home.

We shipped the neural model instead. Here's why both numbers are true — and why the one that matters points the opposite way.

Who's On First is the best open gazetteer we have, and one of the strangest datasets you'll encounter as a developer. This post covers what makes it hard to use, what makes it useful despite the effort, and the tooling we built inside Mailwoman to work with it.

If you've ever tried to answer "what city is this address in?" programmatically with open data instead of a paid geocoding API, you've probably already run into WOF, and it probably raised some questions.