20 posts tagged with "Neural classifier"

Ask a geocoder for "Berlin" and it has to make a choice. There's the one in Germany, There's also Berlin, New Hampshire (population nine thousand and change), Berlin, Wisconsin, Berlin, Connecticut, and a dozen more scattered across the United States like the name was on sale. The parser hands you the word Berlin tagged as a locality. Something downstream has to decide which dot on the map that is. How would you even know if it picked right?

For a long time our answer was a scorecard that checked the name. Did the resolved place's name equal the expected name? Tick. Move on. It is a completely reasonable thing to measure, and it was lying to us for months.

The last postcode post concluded that the "which country is this" signal has to come from the trained model reading the whole string, because the postcode by itself settles the question less than half the time. That is the expensive version of the feature. In this post we built the country signal into the model. It improved German addresses in native word order by a large margin and did not help the same addresses in a different word order.

We took the postcode's gazetteer membership (the [us, de, fr] answer from last time) and injected it into the model at the postcode token instead of passing it to a regex. The injection is a small additive vector on the hidden state at the position of the five digits, encoding which countries the code could belong to. On German addresses in German word order, it added thirty-five points of locality accuracy and beat Pelias.

The international-order numbers were much worse. With the same model, the same anchor, and the same German cities written house-number-first with the postcode after the city (the way our test feed renders them), the model scored slightly above 50%. On those rows the anchor did slightly worse than no anchor.

The rest of this post addresses three questions:

• When a parser "collapses" on a test, is the parser wrong, or is the test?
• Can you train one model to read an address in any order, or does each order cost you the other?
• What does a learned anchor learn: the meaning you intended, or the position where it appeared in training? This one took three retrains to answer.

We set out to fix a small problem in our address parser and measured a number that told us not to build the larger change we had planned.

The small problem is this. When our postcode extractor sees a five-digit run, it has to decide whether the run is a real postcode or a house number that looks like one. It checks the neighboring words against every country's street vocabulary we know (American, German, and French) at once. This works with three countries. At twenty countries, a German street suffix starts matching an English word by coincidence. We looked for a way to tell the extractor which country's words to check.

That question leads to a larger one, and this post is mostly about the larger one.

Our neural address parser passes 20.7% of our test suite. The rule-based parser it's meant to replace passes 93.7%. By that scoreboard, we should delete the neural model and go home.

We shipped the neural model instead. Here's why both numbers are true — and why the one that matters points the opposite way.

We spent a night trying to make our neural address parser less overconfident. The diagnostics showed a different problem: the training data contained no examples of whole categories of address.

This post describes how we optimized the wrong metric and which diagnostics pointed at the right one.

The v0.5.0-a1 tokenizer was trained exclusively on US and French addresses. When it encountered Chinese, Japanese, Korean, Thai, or Arabic text, it fell back to encoding individual bytes — 50-75% of tokens for CJK scripts. Every byte-fallback token is a lost opportunity for the model to learn meaningful subword patterns.

Today we fixed that.

In Tokyo, the address of Tokyo Tower is 〒105-0011 東京都港区芝公園4-2-8.

In English: "4-2-8 Shibakōen, Minato City, Tokyo 105-0011".

The Japanese form runs right-to-left compared to the English form. The prefecture (都道府県) comes first, then the city or ward (市区町村), then a district (丁目) and a block-number-style locator. There's no street name — just a grid.

This is why every rule-based address parser written for Western addresses breaks on Japan.

For its first six versions, Mailwoman emitted ten BIO tags. The model could pick street out of a row but not street_prefix, street_suffix, unit, or po_box. Real addresses are messier than that. The golden eval set has known examples — 6220 SE Salmon St, Portland, OR 97215 (Stage 2 collapses prefix+name+suffix), 123 Main St Apt 4B, Springfield, IL 62701 (loses the apartment), PO Box 123, Burlington, VT 05401 (treats it as a malformed street).

v0.6.0 adds six tags: street_prefix, street_suffix, unit, po_box, intersection_a, intersection_b. The model is the same h384/6L/6H transformer. The recipe is the same v0.5.1 settings. The tokenizer is the same v0.6.0-a0 multi-script bundle. The only structural change is the output head: 21 BIO labels → 33.

The /demo page now loads a 9 MB FST (finite-state transducer) gazetteer alongside the 29 MB ONNX model. 94,000 US admin places with Wikipedia importance scores feed directly into the neural classifier's Viterbi decoder as emission priors — the same pipeline that runs server-side now runs entirely in the browser.

We shipped a browser-based address parser that runs a 29 MB ONNX model entirely client-side. The Playwright tests showed perfect results. Chrome desktop looked great. Then someone opened it on an iPhone.