The whole geocoder runs in your browser. You type an address, you get a rooftop coordinate, and no server ever sees your query — the gazetteer it resolves against is a SQLite database sitting on a CDN, and the page reads it with HTTP range requests, a few kilobytes at a time. It's a lovely trick. We were proud of it. Then we counted the requests it took to find a single city, and the number was 243.

So the questions for the day: why does looking up one name cost 243 round trips? What goes wrong when you search a database you can only read a byte range at a time? And how do you get a global gazetteer — every country, region, county, and city we resolve against — down to about a dozen reads without putting a server back in the loop?

Overnight I taught the geocoder Canada. Or I thought I did. I pulled 843,000 Canadian postcodes, computed a centroid for every one, spot-checked the result — M5H 2N2, downtown Toronto, 43.652, −79.382, dead on — and validated the database every way I could read it: every postcode present, every coordinate right, and no other data disturbed. Green, top to bottom. This morning, before flipping it live, I asked the demo to find a Toronto address. It dropped the pin in Ohio.

So, the questions for the morning. How does a database that passes every check still land the answer 600 kilometers wrong? What does it mean to "validate" a thing? And why is Toronto in Ohio?

Open the demo, open your browser's network tab, and type an address. It resolves to a rooftop coordinate: 1600 Pennsylvania Ave lands on the actual building, within about ten meters. The network tab shows no request carrying your address. The parser ran in the page. The gazetteer it resolved against is a file on a CDN, and the page read it a few kilobytes at a time. The query stayed on your machine.

That is unusual, because each of the three kinds of product on the market today requires you to give something up.

Every geocoder turns an address into a coordinate. Almost none of them will tell you where that coordinate came from. You get a latitude, a longitude, and a vague confidence enum, and when it's wrong you have no thread to pull — no way to know whether the point came from a federal data release, a county GIS office, or a straight line drawn down the middle of a street. Mailwoman keeps the source on every point. Here's New York, every dot colored by the open dataset it came from.

Three public datasets land on your desk. The national provider registry — NPPES, every NPI in the country. A federal telecom-funding file from the FCC's Rural Health Care program, one part of the Universal Service Fund. A state list of licensed nursing facilities from Texas HHSC. You want to know which records describe the same provider, and not one of the three shares an identifier with the other two. The NPI is internal to NPPES. The funding file keys on its own SPIN. The state list has its own facility ID. There's no crosswalk, because none of the three publishers ever built one.

So you do what everyone does: you start a spreadsheet, you sort by name, and you give up around row 400.

Suppose you have a set of records and no key to join them on. A clinic appears in the federal provider registry, again in a state licensing export, and a third time in a funding-program spreadsheet that someone typed by hand. The files share no identifier. The provider number is internal to one publisher, and the facility ID is internal to another. You must judge which records describe the same place for each pair, repeated a few million times.

The market offers two kinds of tools for this. Each solves part of the problem, and neither solves all of it.

We spend effort distrusting numbers that look too good. A validation score that jumps, an accuracy that rounds up suspiciously close to 100 — we've been burned by those, so we poke at them. The number that reports you failed gets a free pass. We assume it's right, because who lies to make themselves look bad?

Our evals did, three times this week. One of them nearly talked us out of a model we should ship. One invented a coverage problem we don't have. And one had us writing "3.3 km" into a model card for a geocoder that puts most addresses within a hundred meters. Each time the fix was the same, and embarrassingly cheap: stop reading the summary row and pull the actual records the summary is averaging over.

Here's a number that should make you nervous: our validation macro-F1 climbed from 0.71 to 0.73 on a retrain we were ready to ship. Instinct reads that as a win: the aggregate went up, so the model must be better, so publish the release.

It wasn't, and it took three probes to prove it. The same model that scored higher on the average had gotten worse at the most basic address there is — a town and a state. This post describes how the average lied to us, and how we caught it.

Mailwoman ships two address parsers in the same box.

We call the first one v0. It is our TypeScript port of the Pelias parser — a rules engine: tokenize, classify each token against dictionaries and patterns, solve for the most plausible arrangement under a pile of hand-written constraints. It is fast, deterministic, and the product of years of accumulated postal wisdom. It is also the thing we set out to beat.

The second is the neural classifier — a sequence labeler trained on a BIO-tagged corpus, the one this blog has spent most of its life arguing with.

So: a year in, did the neural net beat the rules parser? The answer is mostly a tie, until the address gets weird — and the weird is where it gets interesting.

Our failure taxonomy now ranks boundary instability first among the parser's weaknesses. We spent a night building training data to fix it and then chose not to retrain. Three separate checks each caught a step that looked reasonable and was wrong, and this post is about those checks more than the extract. The checks let us work quickly on the worst weakness without shipping a fix that improves the headline number while introducing hidden errors.