Skip to main content

Add place autocomplete

Outcome. A search box returns ranked place completions as the user types. The completions come from a file you already have, and this page measures the cost of each keystroke.

This autocomplete completes place namestoponymA proper name for a geographic place. (localities, boroughsboroughAn administrative or historical division of a city — e.g. the five boroughs of New York City. May be postal, legal, or both, and complicates the locality hierarchy., counties, neighborhoods) from a finite-state transducerFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. built over the gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture.. It does not complete streetstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels. addresses or house numbers, and it uses a different index from forward geocodinggeocodingThe process of converting an address into geographic coordinates (latitude and longitude). Mailwoman geocodes in a multi-tier cascade: exact address-point match → street interpolation → locality centroid. Each tier is progressively coarser but more widely available..

Prerequisites​

  • The library install from Install and first parse. The FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. ships inside @mailwoman/neural-weights-en-us as fst-en-us.bin (21.8 MB), so you already have it.
  • @mailwoman/resolver-wof-sqlite, if you call autocomplete from code. The autocomplete functions are on deep subpaths of that package instead of its root. Step 2 shows the imports.
  • You do not need a gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture. database. The FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. is self-contained.

1. Try it from the command line​

Tell the CLI where the FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. is. Its built-in default is db/wof/fst-per-locale/fst-en-us.bin under the data root. That file exists only after mailwoman gazetteer build fst has run. Pass --fst or set $MAILWOMAN_FST_BIN:

export MAILWOMAN_FST_BIN="$PWD/node_modules/@mailwoman/neural-weights-en-us/fst-en-us.bin"
npx mailwoman autocomplete "san franc" --limit 5
 1. San Francisco [+francisco]  (locality, wof:85922583, imp:0.6901)
 2. San Francisco [+francisco]  (county, wof:102087579, imp:0.6901)
 3. San Francisco Township [+francisco]  (localadmin, wof:404511703, imp:0.0641)
 4. San Francisco [+francisco]  (locality, wof:1125799349, imp:0.0000)
 5. San Francisco Plaza [+francisco plaza]  (locality, wof:1226462397, imp:0.0000)

[+francisco] is the completion the FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. supplied for the partial tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. you typed. imp is the place's importance scoreimportance scoreA precomputed per-place prominence score (blending Wikipedia importance with population) used to rank same-name gazetteer candidates.. Results are sorted by importance, highest first.

--json emits the same list as an array of { name, placetype, wofID, importance, completionTokens }. Build a dropdown from that shape.

The tool's error text is wrong in one place. If the FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. is missing, it tells you to run mailwoman fst build, but that command does not exist. The builder is mailwoman gazetteer build fst, and you should not need it, because the shipped file is the same artifact.

The code uses three functions from deep subpaths of @mailwoman/resolver-wof-sqlite. The package root does not re-export them, so import them by subpath:

import { readFile } from "node:fs/promises"

import { resolveWeights } from "@mailwoman/neural/weights"
import { autocomplete, deserializeFST } from "@mailwoman/resolver-wof-sqlite/fst"

// resolveWeights finds the installed weights package, so you never hard-code a node_modules path.
const { fstPath } = await resolveWeights({ locale: "en-US" })

// fstPath is optional: it is only set when the index file is actually present in the resolved
// package, so a weights bundle without one gives you `undefined` rather than an error.
if (!fstPath) throw new Error("no FST index in the resolved weights package — autocomplete is unavailable")

// Deserialize once, at startup. The returned object IS the matcher — do not wrap it.
const fst = deserializeFST(await readFile(fstPath))

export function suggest(query) {
	return autocomplete(fst, query, { maxSuggestions: 10, dedupeByName: true }).suggestions
}

deserializeFST returns the matcher directly. If you wrap its result in new FSTMatcher(...), you produce an object that answers every query with zero suggestions and raises no error. That silent failure is hard to debug.

A dropdown needs dedupeByName: true, which is off by default. It collapses places with the same name to the one with the highest importance:

dedupeByName: false
  Brooklyn (borough, imp 0.806)
  Brookline (locality, imp 0.431)
  Brookline (localadmin, imp 0.429)
  Brooklyn (neighbourhood, imp 0.364)
  Brooklyn Center (locality, imp 0.359)
dedupeByName: true
  Brooklyn (borough, imp 0.806)
  Brookline (locality, imp 0.431)
  Brooklyn Center (locality, imp 0.359)
  Brooklyn Heights (neighbourhood, imp 0.318)
  Brookleigh (neighbourhood, imp 0.297)

Without deduplicationrecord matchingThe process of determining whether two database records refer to the same real-world entity. Mailwoman's matcher uses a geocode-first approach (match the resolved place, not the address string) with Fellegi-Sunter probabilistic scoring., five slots show three distinct places. With it, they show five. The CLI leaves deduplicationrecord matchingThe process of determining whether two database records refer to the same real-world entity. Mailwoman's matcher uses a geocode-first approach (match the resolved place, not the address string) with Fellegi-Sunter probabilistic scoring. off because a person auditing the gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture. wants to see that New York is both a citylocalityThe city / town / settlement component of an address: a populated place sitting between region and neighborhood in the hierarchy. and a county.

3. Measure the keystroke cost​

The matcher loads once at low cost. Query cost varies, and the most expensive keystroke costs about three orders of magnitude more than the cheapest. Measure it before you decide whether to debounce:

node autocomplete-bench.mjs
load 0.26 s   rss 265 MB
keystroke shape        n     p50 µs   p95 µs
complete token         760          8        67
partial first token    680       8370     10512
partial later token    400          6        19

Measured on an AMD Ryzen 9 8945HS, 16 logical cores, Node.js 26.2.0, mailwoman 8.7.0, over every keystroke of four typed queries (san francisco, brooklyn, washington dc, cambridge ma), 40 samples per prefix, after a warm-up pass. The machine was shared, so treat the absolute values as one host's figures. Your numbers will differ, but the ratios below should hold on other machines. Resident memory after load ranged from 241 MB to 293 MB across runs, so treat that figure as an order of magnitude.

The three keystroke shapes behave differently, and the partial first tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. dominates the cost:

  • Complete tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words.: the input is a whole word the index knows (san, brook). The walk reaches a state and reads its entries. 8 µs.
  • Partial later tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words.: a complete first word plus a fragment (san fr, washington d). The full walk fails, so the code walks the complete prefix and filters that state's continuation edges by prefix. That state has 100 edges. 6 µs.
  • Partial first tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words.: a fragment and no other tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. (sa, br, brookl). The same prefix filter runs, but on the root state. The root state has 53,904 continuation edges. The code scans every one of them. 8.4 ms.

The expensive keystrokes are therefore the first few characters of the first word. The cost drops as soon as the user types a space.

Compare these costs with a parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates. instead of with a clock, because the ratio holds on other machines. classifier.parse over the same twenty addresses, on the same host in the same session, measured p50 6,086 µs. A complete-tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. completion is roughly three orders of magnitude cheaper than one parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates., and a partial-first-tokentokenOne word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words. completion costs about as much as one parseaddress parsingThe process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates.. An index query takes 6 to 8 µs except at the root, where one query takes 8.4 ms, about as long as a modelneural classifierThe machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.' call.

The ratio has two consequences for the front end. First, debounce input instead of querying on every keydown, because a fast typist generates mostly root-scan keystrokes. Second, keep the query off the main threadthreadA parallel workstream within a release. Threads compose; they are not sequential milestones like phases. of a browser tab. On this host 8.4 ms is longer than a frame at 60 Hz, and a slower host exceeds the frame budget by more.

Verify​

npx mailwoman autocomplete "brookl" --limit 3 --json
[
  {
    "name": "Brooklyn",
    "placetype": "borough",
    "wofID": 421205765,
    "importance": 0.8064998388290405,
    "completionTokens": [
      "brooklyn"
    ]
  },
  {
    "name": "Brookline",
    "placetype": "locality",
    "wofID": 85950817,
    "importance": 0.4307776987552643,
    "completionTokens": [
      "brookline"
    ]
  },
  {
    "name": "Brookline",
    "placetype": "localadmin",
    "wofID": 404476429,
    "importance": 0.42851021885871887,
    "completionTokens": [
      "brookline"
    ]
  }
]

Every suggestion includes a wofID, the Who's On FirstWOF (Who's On First). An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations. identifier. Your code can resolve a selected suggestion to coordinates and a hierarchy without a second text query. That is the reason to build the dropdown on this index instead of on repeated geocodes.

Limits​

  • It completes places only. It does not complete streetsstreetThe named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels., house numbershouse numberThe numeric or alphanumeric identifier of a building on a street. Mailwoman's house_number component; its position relative to the street name flips between locales. or points of interest. A user typing 1275 Pennsylv gets no useful suggestion from this index.
  • Each file covers one localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for.. fst-en-us.bin ships with the en-US weightsparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values., and @mailwoman/neural-weights-fr-fr ships its own file. The library does not merge localeslocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for., so a search box that serves several markets loads several files and merges the result lists itself.
  • Resident memory is a few hundred megabytes after load (241 MB to 293 MB across the runs above). The file is read fully into memory instead of being memory-mapped. Plan for that cost once per localelocaleThe combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for. per process.
  • The ranker uses importance and ignores proximity. The API has no near parameterparameterA single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values.. A user in Boston typing brookl gets Brooklyn first because Brooklyn has higher importance, even though Brookline is closer. Re-rank on your side with the coordinates that the wofID resolves to.
  • The CLI's default FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. path is a build output under the data root. Set $MAILWOMAN_FST_BIN in any environment where you rely on the command and did not build the FSTFST (finite-state transducer). A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead. there.