Mailwoman Glossary
Key terms and concepts in the Mailwoman address parser and geocoder.
Domain26
The address-parsing problem space — why addresses are hard, how postal systems work, what users actually type.
- Address Management System (AMS)
- The USPS's authoritative database of every deliverable US address (~165 million points). Licensed to commercial mailers under strict terms, not openly available — part of why an open parser can't just look every address up.Also known as: USPS AMS
- address parsing
- The process of decomposing a free-text postal address string into structured components — house number, street name, locality, region, postcode, and country — so a geocoder can resolve them to coordinates.Also known as: parse, parsing
- APO/FPO (Army Post Office / Fleet Post Office)
- US military postal addresses for personnel stationed overseas or aboard ship. They route through military postal hubs rather than geographic locations, so they break geographic parsing assumptions.Also known as: APO, FPOReferenced by: PO boxes and other non-street addresses
- banchi
- In Japanese addressing, a block within a chōme. The middle level of the chōme / banchi / gō hierarchy.Referenced by: Component tags
- block face
- One side of a street between two intersections. Several postal systems specify delivery to block-face granularity — e.g. the USPS ZIP+4 extension.
- carrier route
- The delivery path one postal carrier covers on a shift. A ZIP code is fundamentally a set of carrier routes, not a geographic polygon, which is why ZIP boundaries are fuzzy and shift over time.
- chōme
- In Japanese addressing, a district-level subdivision in the block-based chōme / banchi / gō numbering scheme, which uses area-and-block numbers instead of street names.Referenced by: Addressing around the world, Component tags
- City State Product
- The USPS file mapping each ZIP to its 'preferred' and 'acceptable' city names. One ZIP can carry several city names because a post office serves multiple neighborhoods and adjacent municipalities.
- delivery point code (DPBC — Delivery Point Barcode)
- The full 11-digit USPS code (5-digit ZIP + 4-digit extension + 2 delivery-point digits) that uniquely identifies a single mailbox, encoded as the barcode on an envelope for automated sorting.
- diacritic
- An accent mark that modifies a letter (é, ñ, ç). Address normalization must fold diacritics for matching without discarding the information a user typed.Also known as: accent mark
- formula address
- A descriptive, relative address given by directions and landmarks ('two blocks past where the bakery used to be, half a block toward the lake') rather than a street and number. The hardest end of the parse-difficulty spectrum.Also known as: relational address, landmark-based address
- gō
- In Japanese addressing, a building or lot number within a banchi — the finest level of the chōme / banchi / gō block-based hierarchy.Referenced by: Component tags
- homonymy
- One surface, many referents: 'Georgia' is a US state in 'Atlanta, Georgia' and a country in 'Tbilisi, Georgia.' Handled by disambiguation, split across two stages — the parser resolves the tag from in-string context; the resolver late-binds the referent with geographic context.Also known as: homographReferenced by: What the model cannot do
- HPSA (Health Professional Shortage Area)
- A US federal designation identifying areas short of health providers relative to population and need, scored 0–25. An example downstream use of geocoded address data.
- IATA code (International Air Transport Association code)
- A three-letter airport code (JFK, LHR, CDG). Globally unique and machine-friendly, useful for resolving airport queries.Also known as: airport code
- IMU (Index of Medical Underservice)
- The composite score — provider-to-population ratio, poverty rate, infant mortality, and elderly-population share — behind the Medically Underserved Area designation.
- locale
- The combination of language and country an address comes from. en-US and fr-FR are the locales Mailwoman ships weights for.
- MUA (Medically Underserved Area)
- A US federal designation for inadequate primary-care access, scored using the Index of Medical Underservice. Another downstream consumer of accurate geocoding.
- parcel
- A property polygon or record carrying a situs (site) address and often a separate owner mailing address. County GIS parcel aggregations are a training source for address-point variety and situs-vs-owner divergence.
- PO box
- A numbered mailbox at a post office used as a delivery address instead of a physical street location. Mailwoman tags it as the po_box component; structurally the same family as a subpremise.Also known as: post office box, P.O. Box, po_box
- postal city
- The city name the postal service uses for a ZIP, which can differ from the legal municipality a resident lives in — e.g. many suburbs share a larger city's postal name. A source of locality ambiguity.Also known as: mailing city, USPS city name
- Postcode Address File (PAF)
- Royal Mail's authoritative database of valid UK postcodes and their delivery points. The UK equivalent of the USPS AMS.Also known as: UK postcode database
- sectional center facility (SCF)
- A USPS regional mail-processing hub. The first three digits of a 5-digit ZIP identify the SCF, of which there are roughly 900 nationwide.
- situs
- The physical site address of a property, as opposed to the owner's mailing address. Parcel records often carry both; the divergence is a real-world data-quality challenge.Referenced by: Improve geocode precision
- synonymy
- Many surfaces, one referent: '123 N Main St' and '123 North Main Street' are the same place. Handled by canonicalization, never by guessing which spelling is right. The dual of homonymy.
- toponym
- A proper name for a geographic place.Also known as: place name
Architecture5
Mailwoman's design — the staged pipeline, the Knowledge Ladder, schemas, and how the pieces compose.
- classification proposal
- The shared shape that every classifier (rule or neural) writes. The solver consumes proposals without knowing which kind of classifier produced them.
- component tag
- One of the 25 labels in Mailwoman's address schema — street, locality, region, postcode, house_number, unit, po_box, country, venue, intersection, and others. Each parsed span carries exactly one component tag.Also known as: ComponentTag, label
- parse tree
- The hierarchical address structure produced by decoding BIO labels into spans and nesting them per the schema's containment rules (house_number → street → locality → region → country).Also known as: AddressTree
- span
- A contiguous range of characters or tokens in the input string, tagged with an address component type (street, locality, postcode, etc.). Parsed addresses are represented as collections of spans, possibly nested in a tree.
- tree projection
- The process of converting a flat list of labeled spans into a nested tree representation — street contains street_prefix and street_suffix, locality contains region. Enables hierarchical address operations like containment checks.
Reference4
Glossaries, overviews, historical bridges, and implementation-plan material.
- iteration log
- The running record of each model release — the canonical 'what shipped when' for the neural classifier.
- Mermaid
- A Markdown-friendly diagram syntax used in these docs for flowcharts.
- phase
- A milestone in the implementation plan (Foundation, Corpus, Training, Integration, and forward-looking phases). Distinct from stage (runtime pipeline) and tier (model vocabulary).Referenced by: CLI, Deploy on a serverless runtime
- thread
- A parallel workstream within a release. Threads compose; they are not sequential milestones like phases.
Geocoding19
Turning parsed addresses into coordinates — cascades, precision tiers, and the reference datasets behind them.
- address ID
- A stable, parseable primary key for an address in the format <state>.<H3-cell>.<hash>. Content-addressed (derives from the data), jitter-stable (absorbs small geocode differences), and partitionable by state prefix.
- BAN (Base Adresse Nationale)
- France's authoritative open national address register — the highest-quality training source for French addresses, with full component structure.
- coverage
- The fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present.
- E911 (Enhanced 911)
- County- and state-level emergency-dispatch address-point databases with full component breakdown. A lineage source for the situs and address-point layers used in geocoding.
- G-NAF (Geocoded National Address File)
- Australia's authoritative open address register (CC-BY-licensed), used as a training source for Australian addresses.Referenced by: Build the planet, CLI
- geocode cascade
- Mailwoman's multi-tier coordinate resolution strategy: first try exact address-point match, then street interpolation, then locality centroid. Each tier is more widely available but progressively coarser.
- geocoding
- The process of converting an address into geographic coordinates (latitude and longitude). Mailwoman geocodes in a multi-tier cascade: exact address-point match → street interpolation → locality centroid. Each tier is progressively coarser but more widely available.Referenced by: Add place autocomplete, Build the US address dataset from public sources, Calibration and confidence, Capabilities, CLI, Compare, Cross-locale queries, Data products +32 more
- H3
- Uber's hexagonal hierarchical geospatial indexing system. Mailwoman uses H3 cells at resolution 9 (~0.03 km²) for geo-first blocking in the matcher and for stable address primary keys in @mailwoman/address-id.Also known as: H3 cellReferenced by: CLI
- interpolation
- A geocoding technique that estimates a coordinate along a street segment based on the house number range. Used as the middle tier of Mailwoman's geocode cascade when exact address-point data is unavailable.
- locality centroid
- The representative center point of a city or locality, used as a coarse coordinate when no exact address point is available — the coarsest tier of the geocode cascade.Also known as: city centroid
- NAD (National Address Database)
- A US Department of Transportation dataset of structured address points, added to the training corpus as a major source of real US addresses.Referenced by: Build the US address dataset from public sources, CLI
- NG911 (Next Generation 911)
- The modern 911 data standard. States and counties publish NG911 address-point layers that Mailwoman can ingest as address data.
- OpenAddresses (OA)
- A global open aggregation of address points collected from many official sources. A primary source of component-supervised training data outside proprietary registries.
- OpenStreetMap (OSM)
- A community-curated global map database (ODbL-licensed) with addr:* tagged features and place hierarchies. A secondary corpus source and a source of street names.Also known as: OSMReferenced by: Build the planet, Compare, Data products, Drop-in replacements, Gazetteers, Locales and tiers, Own what you look up, Security and compliance +4 more
- Overture Maps
- An open consortium (Meta, Microsoft, AWS, TomTom) publishing Places and Addresses themes with GERS entity IDs. A candidate gazetteer source for POIs and street-level addresses.
- rooftop
- Geocoding precision at the building or parcel level — coordinates within a few meters — the highest tier of the geocode cascade. Sourced from address-point and situs data.Also known as: rooftop-level, address-point geocoding
- situs data
- A dataset of exact address-point coordinates (rooftop-level). Mailwoman's geocoder uses a national situs layer (124.9M US points built from state address-point sources) as the highest-precision tier of the geocode cascade.Also known as: address points, rooftop data
- TIGER
- The US Census Topologically Integrated Geographic Encoding and Referencing database. Used as a corpus source for street-segment data.
- two-step floating catchment area (2SFCA)
- A spatial-accessibility metric that measures provider supply against population demand within a distance threshold, with distance decay (the E2SFCA variant). Appears in Mailwoman's spatial-demand analysis of geocoded points.Also known as: 2SFCA
Evaluation40
Measuring the parser — metrics, golden sets, parity scorecards, calibration, and eval discipline.
- Acc@1 (accuracy at 1)
- The share of eval rows whose top-ranked resolver candidate is the correct place, regardless of coordinate distance — 'did we pick the right place', independent of geocode precision.Also known as: top-1 accuracy
- acceptance battery
- The packaged promotion harness — golden floors with noise-honest margins, fragment boards with lowercase legs, invariance, gauntlet, canaries, evidence ablation, capability manifest — run identically for our own promotes and (with per-customer canaries + the EWC guarantee gate) at fine-tune-engagement delivery.Also known as: battery, promotion battery
- adversarial corpus
- Eval entries chosen specifically to break the parser. Mailwoman's adversarial set covers cases like 'Buffalo Buffalo' (a venue named after a city), 'St. Petersburg' (a multi-word locality), and prefix-honorific names.
- annotation noise
- Errors in human ground-truth labels (typically around 1%) that put an irreducible ceiling on eval metrics — no model can exceed the agreement of the labels it's graded against.Also known as: label noise
- arena
- A standardized test set probing one capability: libpostal (clean canonical), perturb (noisy and degraded), postal (edge formats). Each arena answers a different question about where rule vs neural wins.
- boundary instability
- A class of parser errors where the model wobbles on where one address component ends and the next begins — a street suffix swallowed into the street name, a French house number that trails instead of leads. Mailwoman's #1 weakness as of v1.5.0.
- canary fixture
- A named eval row pinning a specific past failure with its receipt, graded zero-flip at every promotion — the row must never regress. The customer-facing variant (rows from a customer's own stock) is the B11 acceptance battery's per-engagement layer.Also known as: canary
- confidence calibration
- The process of adjusting model confidence scores so that '0.6' actually means the model is right about 60% of the time. Mailwoman uses isotonic regression (PAVA) to calibrate per-span confidences against held-out data. Applied opt-in via createCalibrator.Referenced by: Footprints
- conformal calibration
- A calibration method that adjusts per-prediction uncertainty so a target coverage (e.g. 90%) is guaranteed on held-out data, without retraining the model.Also known as: split-conformal
- coord metric
- The primary evaluation metric: distance from the resolved coordinate to the true address point. Measured at percentiles (p50, p90) and as 'within X meters.' Prevents the label-F1 trap where a model scores higher on token labels but geocodes worse.Also known as: coordinate metric, assembled coordinate
- coordinate-acceptability
- The resolve-through read used as a probe gate: parse → resolve the predicted admin pair through the gazetteer → acceptable iff the resolved point lands within tolerance of the row's gold coordinate. Measures whether the parse reaches the right PLACE, not span cosmetics.Also known as: coord acceptability
- cross-pollution
- An eval tripwire for unwanted tag bleed across locales — predicting a German city as a postcode because multi-locale training let one locale's patterns hurt another.
- delta F1
- The F1 difference between two model versions for a tag, used to flag a regression or a gain step-over-step.Also known as: F1 delta
- ECE (Expected Calibration Error)
- A metric that measures how well a model's confidence scores align with its actual accuracy. Lower is better. Mailwoman's held-out ECE drops from 0.067 (raw) to 0.0035 (calibrated).
- eval
- Running the model against a held-out golden dataset and computing per-component F1, exact-match, calibration, and resolved-coordinate error.Also known as: evaluation
- exact match
- The share of eval items whose every component is correct (compared per-span or per-token). Stricter than per-tag F1, which credits partial correctness.
- F1 score
- The harmonic mean of precision and recall — a single number between 0 and 1 summarizing how good a classifier is at a class. F1 = 2·P·R / (P + R).
- flip census
- The per-row diff of two model verdicts on the same fixture (which rows flipped, which way), used to attribute a delta to a mechanism instead of trusting the aggregate. The instrument behind the four-law receipts.
- fragment board
- The standing targeted-failure-class eval: ~400 rows per pre-registered class (homonym, bare-street, bare-locality, housenumber variants, particles, date-names) sampled from a national source, graded with Wilson intervals, with the NEGATIVE class scored (a hallucinated street on a bare locality is visible by construction). The FR board (ban-fragments-fr) is the P0 instrument; the DE board (overture-fragments-de) is the v7-fold gate's leg.Also known as: P0 board, fragment fixture
- gated promotion
- A release discipline where a model ships to production only if it passes pre-registered metrics (e.g. 'DE locality ≥ 70%, no tag regresses > 2pp'). Failing models can be uploaded as experiments but not promoted to default.Also known as: promotion gate
- gauntlet
- The three-layer input-robustness gate: regression fixtures (named past failures, must-pass), metamorphic perturbations (label-preserving edits must not move the coordinate — INV/DIR/BAND relations), and a held-out fresh draw. Known failures are tracked xfails; a NEW failure blocks.Referenced by: resolver_score cannot reject garbage — a characterization
- golden set
- A hand-labeled evaluation dataset of US, French, and adversarial addresses used as ground truth for evals.
- ground truth
- The correct answer for an example, used as the standard a prediction is graded against. Mailwoman's ground truth is the hand-labeled golden set; its quality caps achievable accuracy.Also known as: gold labelReferenced by: Belgium — 30 addresses, three configurations
- grouper-audit
- A validation pass checking that phrase-grouper spans are internally consistent — no overlaps, no contradictions with BIO structural rules. Audit errors must be zero on a shipping model.
- harness pass rate
- Whether a full address parses end-to-end with every component correct and well-segmented and no orphan spans. Stricter than label F1 — a model can win on recall yet fail the harness if boundaries slip.Also known as: end-to-end accuracy
- honest eval
- Mailwoman's evaluation discipline: grade the assembled pipeline output (resolved coordinate) against real-world ground truth, not raw neural label-F1 in isolation. A model can win on labels while resolving to the wrong city.
- isotonic calibration
- A post-hoc calibration that fits a monotonic map from raw model scores to true probabilities without retraining (via PAVA). Mailwoman's confidence calibrator.Also known as: isotonic regressionReferenced by: Tune confidence thresholds
- leakage
- Train/test contamination that inflates reported accuracy when eval data has effectively been seen in training. Mailwoman guards it with held-out-geography evals and locality-aware splits.Also known as: data leakage, train-test leakage
- macro F1
- The unweighted average of per-class F1 scores — treats every class equally. Mailwoman's primary label-level eval metric.
- name-form variant
- A resolver 'miss' that is really a measurement artifact: the right place is found but its gazetteer-canonical name differs from the eval's expected literal — 'St. Johnsbury' vs 'Saint Johnsbury'.
- named watch
- A known below-target reading recorded at ship with an owner and a retirement condition — never a silent waiver. Example: fr.cedex shipped at 83.3 under the waived floor, named, and retired when the from-scratch base read 90.5.Also known as: watch
- noise-honest margin
- The floor-cutting rule: floor = reference reading − max(1.0pp, 2× the metric's 95% binomial half-width at its own support). The noise band varies ~19× across metrics, so a uniform margin either blesses real drift or fails on coin flips. Where support is not exposed the rule falls back to 2.0pp — a trap when the true support is small (the street_prefix lesson).
- oracle substitution
- A diagnostic that hands one pipeline stage the ground-truth answer and measures the end-to-end result, to find which stage an error actually lives in. Oracle the model's tags to measure the resolver's ceiling; oracle the gazetteer hints to measure the model's. Always an optimistic upper bound.Also known as: ceiling analysis
- p50 / p90 / p99
- Coordinate-error percentiles: p50 is the median error, p90 the worst 10%, p99 the tail. Reported together so a high p50 (systematic error) is distinguished from a high p90/p99 with low p50 (concentrated edge-case error).Also known as: percentile metrics
- parity campaign
- The systematic effort to close the gap between Mailwoman's neural parser and the legacy rule-based v0 parser on every address component tag. Tracked through per-tag parity tables and gated by honest-eval probes.
- parity scorecard
- The authoritative per-tag table tracking neural-vs-v0/Pelias F1 and resolver accuracy across head-to-head arenas. It answers 'where are we at parity, where do we still bleed?' and governs the parity campaign's priorities.
- per-locale F1
- Per-tag accuracy computed separately for each locale (US, FR, DE…) rather than aggregated, surfacing locale-specific regressions that macro F1 would mask under the dominant locale.
- precision
- Of the spans the model labeled as a given tag, the fraction it got right. High precision means few false positives. Paired with recall to compute F1.
- recall
- Of the spans whose gold label is a given tag, the fraction the model found. High recall means few misses. Paired with precision to compute F1.Also known as: sensitivity
- zero-shot
- Evaluating or predicting on a class or geography the model never saw in training, with no in-context examples — the strongest test of generalization.
Infrastructure9
The machinery around the model — compute, storage formats, runtimes, and tooling.
- Cartographer
- Mailwoman's mapping utilities. Composes MapLibre style specifications and builds vector source records for the demo's Protomaps basemap.
- execution provider fallback
- onnxruntime trying its requested backends in order (e.g. ['webgpu', 'wasm']) and silently falling back to the next on failure. Convenient, but it can mask a broken backend — headless Chromium lacks WebGPU, so tests silently ran on WASM.
- FTS5
- SQLite's built-in full-text-search module (with BM25 ranking). Mailwoman uses it for prefix and name matching against the gazetteer.Also known as: full-text search
- JSEP (JavaScript Execution Provider)
- onnxruntime-web's older WebGPU backend. A kernel bug on slice operations with axis reversal corrupted int8 dequantization in the browser; it has been superseded by the native WebGPU execution provider.
- Modal
- A cloud GPU platform (modal.com) where Mailwoman trains its neural models on NVIDIA A100 GPUs. Training runs are launched via scripts/modal/train_remote.py and typically complete in ~1 hour.
- ONNX (Open Neural Network Exchange)
- An open format for machine learning models that enables interoperability between training frameworks and inference runtimes. Mailwoman ships its trained model as an ONNX file so it can run in Node.js and the browser via onnxruntime.Also known as: ONNX runtime, onnxruntime
- R*Tree
- SQLite's spatial index of bounding boxes, enabling fast geographic range and nearest-neighbour queries in the resolver.Also known as: spatial index
- SQLite-wasm
- An open-source build of the SQLite library compiled to WebAssembly. Mailwoman uses it to run the WOF resolver inside the browser.
- WAL + Freeze
- The SQLite build pattern for the gazetteer: ingest under WAL (write-ahead logging) for concurrent writes, then checkpoint, set journal_mode=DELETE, add indexes, ANALYZE, and VACUUM INTO to emit a clean read-only artifact with no sidecar files.
Night shift1
The night shift debugging sessions — Problems. Process. Outcomes.
- night shift
- An autonomous overnight agent session — training launches, evals, publishing, issue triage — that ends with a structured postmortem (what shipped, regressions, open questions) committed for handoff.
Rule-based classifiers3
Regex, dictionary, and heuristic classifiers inherited from Pelias and Mailwoman v1.
- libpostal
- An open-source C address parser used by Pelias. Mailwoman's rule-based v0 and neural classifier supersede it.Referenced by: Compare, Component tags, Drop-in replacements, Footprints, HTTP APIs, mailfail — robustness on garbage, malformed and hostile input, Overview, Packages +5 more
- Pelias
- An open-source geocoder, Mailwoman's spiritual predecessor.
- rule-based classifier
- Mailwoman's legacy v0 parser — a library of deterministic token classifiers (house number, street suffix, postcode, place name, etc.) composed by priority. Now primarily used for corpus labeling, fallback classification, and arbitration diagnostics.Also known as: v0 parser, rules engine
Neural classifier50
The encoder-only transformer model — architecture, BIO labeling, decoding, and inference.
- additive bias
- A logit adjustment added before softmax from a prior or external knowledge source, typically soft-capped so it influences but never overrides the model's prediction. The mechanism behind shallow fusion.Also known as: emission biasReferenced by: Library API
- anchor inference
- A technique where structured knowledge (postcode locations, gazetteer place names) is injected into the model as soft input features — not as deterministic overrides. The model still decides the final labels, but the anchor signal biases it toward correct admin tags.Also known as: postcode anchor, soft anchor
- argmax decoding
- The simplest decoding strategy: pick the highest-scoring label independently for each token. Fast but can produce invalid BIO sequences (e.g., I-street without a preceding B-street). Mailwoman ships argmax as the default and Viterbi CRF as an option.
- attention
- The core mechanism inside a transformer encoder. Each token's representation is updated by looking at every other token, with learned weights deciding how much each one matters.
- attention head
- One of several parallel attention computations in a layer, each free to focus on a different kind of relationship between tokens. Their outputs are concatenated — 'multi-head attention'. Mailwoman uses 4 heads.Also known as: multi-head attention, head
- bidirectional context
- Letting each token attend to tokens on both its left and its right. Mailwoman's encoder is bidirectional — it reads the whole address at once, unlike a left-to-right language model.Also known as: bidirectional attention
- BIO tagging (Begin-Inside-Outside)
- A token-level labeling scheme where each token is tagged as B-X (beginning of an entity of type X), I-X (inside an entity of type X), or O (outside any entity). Mailwoman uses BIO over SentencePiece tokens to annotate address components.Also known as: BIO labels, BIO encodingReferenced by: Footprints, What ships today
- coarse-placer
- A lightweight int8 country classifier (~0.79 MB) that predicts which of a set of target countries an address belongs to, feeding a soft prior into resolver disambiguation.Referenced by: CLI
- context window
- The span of input a model can consider at once, bounded by its max sequence length. Everything in the window can influence every token's label via attention.
- CRF (Conditional Random Field)
- A statistical modeling method that predicts structured outputs by modeling dependencies between adjacent labels. Mailwoman uses a linear-chain CRF as the Viterbi decoder at inference time to enforce BIO label consistency — a B-street must be followed by I-street or O, never I-locality.Also known as: conditional random fieldReferenced by: Decoding and the best path, Pelias and libpostal
- decoder
- In a transformer encoder-decoder model, the part that produces output sequences. Mailwoman's classifier is encoder-only (no decoder); the 'CRF decoder' is a different thing — a structured-prediction layer that picks the best label sequence from the encoder's outputs.
- embedding
- A vector of numbers representing a token (or other item) so that similar items sit near each other in vector space. The first thing the model does is turn each token into an embedding.Also known as: embedding vectorReferenced by: Tokens and labels
- embedding space
- The high-dimensional space in which embeddings and hidden states live. Geometry there is meaningful — distance and direction encode similarity and relationships learned from data.Also known as: vector space, representation space
- emission logit
- The per-token, per-label logit the transformer encoder emits before any external prior is added — the model's own evidence for each label at each position.Also known as: emission score
- encoder
- The part of a transformer that turns input tokens into contextualized vector representations. Mailwoman's classifier is a small encoder-only transformer (~30M parameters).
- ensemble
- Combining several models' predictions to cut variance and improve robustness. A general ML technique, noted in the docs as a lever Mailwoman has not needed to pull.
- feature
- An input signal a model conditions on. Beyond the raw tokens, Mailwoman feeds soft features — gazetteer-membership channels and the postcode anchor — that inform predictions without overriding them.Also known as: input feature
- feed-forward network
- The per-token transformation inside each transformer layer that follows attention: a small two-layer network applied independently at every position.Also known as: FFN, MLP
- FiLM modulation (Feature-wise Linear Modulation)
- A parameter-efficient adaptation that scales and shifts feature vectors by learned per-example multipliers and biases. Used in self-conditioning to make parsing decisions locale-aware without retraining the whole model.
- fine label
- The Tier 2 labels added to the model vocabulary — venue, street, house_number — as opposed to the coarse labels (country, region, locality, postcode).Also known as: coarse label
- gazetteer anchor
- An input-layer feature channel that attaches a per-token candidate-tag set from the gazetteer/codex (e.g. 'this surface is a known country-and-region') so the model conditions on lexicon membership without being overruled by it. The knowledge lives outside the weights — extend the gazetteer, no retrain.
- inference
- Running the trained model on new input to get predictions, as opposed to training, which produces the model. In Mailwoman that means a small transformer encoder reads an address string and classifies every token — house number, street, locality, region, postcode, and the rest. A Who's On First gazetteer can feed soft location hints into the pass, but the model makes the final call on every label. Where a generative model writes text token by token, Mailwoman's output is a retrieval-augmented token classification: one label per input piece.
- int8 quantization
- A model compression technique that stores weights as 8-bit integers instead of 32-bit floats, reducing model size (~4×) with minimal accuracy loss. Mailwoman ships int8-quantized ONNX models (~30 MB) for fast loading in browsers.
- language model (LM)
- A model that assigns probabilities to sequences of tokens. Used here mostly as a prior — an FST or n-gram model that biases the decoder toward plausible sequences via shallow fusion.Referenced by: The gazetteer prior
- layer
- One transformer block — attention plus a feed-forward network, with normalization and residual connections — applied to every position. Stacking layers lets the model build up richer representations; Mailwoman's encoder has 6.Also known as: transformer block, encoder layer
- layer normalization
- A step that rescales a vector to a stable range before the next sublayer, keeping activations well-behaved and training stable. A standard ingredient of transformer layers.Also known as: layer norm, LayerNorm
- logit
- A raw, unnormalized per-label score the model outputs before softmax. Priors and biases are added in logit space, then softmax turns logits into probabilities.Also known as: raw score
- machine learning (ML)
- Building systems that learn patterns from examples instead of following hand-written rules. Mailwoman's neural classifier is trained on millions of labeled addresses rather than programmed with parsing rules.
- max sequence length
- The maximum number of tokens the model can process at once. Inputs longer than the cap are truncated, dropping tail context.
- model weights
- The learned parameters of the neural classifier, shipped as ONNX files in the @mailwoman/neural-weights-* packages. Weights are locale-specific bundles that include the model, tokenizer, and a model-card.json metadata file.Also known as: weights bundle
- n-gram
- A contiguous run of n tokens (a bigram is 2, a trigram is 3). Counting n-grams is a simple, pre-neural way to model which token sequences are common.
- neural classifier
- The machine learning model at the core of Mailwoman's parser — a transformer encoder (~30M parameters) trained from scratch to do BIO token classification over addresses. It learns the 'grammar' of address formats; the gazetteer supplies the 'atlas.'Also known as: model, transformer
- neural network
- A model made of layers of simple numeric units whose connection strengths (weights) are learned from data. The transformer encoder at Mailwoman's core is a neural network.Also known as: net
- one-hot encoding
- Representing a category as a vector that is 1 at the category's index and 0 everywhere else. The 'hard target' a classifier is trained toward — until label smoothing softens it.
- orphan-I
- A label-sequence bug where I-X appears without a matching preceding B-X (e.g. O, I-locality). Structurally invalid in BIO; the CRF prevents it.
- parameter
- A single learned number inside a model — one weight or bias. Mailwoman's encoder has roughly 30 million of them; training is the search for good values.Also known as: weight, model parameter
- perplexity
- How surprised a language model is by held-out text, expressed as the effective number of choices it was weighing per token; lower means better prediction. It is the standard health metric for generative models, with a classic trap: it is computed per token, so models with different tokenizers cannot be compared on it. Mailwoman does not gate on perplexity — a token classifier is graded on labels and, ultimately, coordinates — but the same trap governs our own metrics: parse-F1 is never compared across tokenizer versions.
- production scorer
- The canonical inference API in @mailwoman/neural. It reads the model-card.json requires block and fails closed if a declared channel (anchor, gazetteer) isn't fed — preventing silent degradation from running the model out-of-distribution.
- residual connection
- A shortcut that adds a sublayer's input to its output, so information and gradients flow easily through deep stacks. Part of why deep transformers train at all.Also known as: skip connection
- self-conditioning
- An auxiliary model head that predicts the input's locale (language/country) and then FiLM-modulates the encoder so parsing adapts to the detected locale. Structural facts like word order still need explicit training on both orders.Also known as: locale-conditional decoding
- sequence labeling
- A machine learning task where a model assigns a label to each element in a sequence. In Mailwoman, the sequence is the tokenized address string and the labels are address component types (street, locality, postcode, etc.).Also known as: token classification, token-level classification
- soft features
- Auxiliary input signals fed to the neural classifier that inform but never override the model's decisions. Mailwoman's soft features include the postcode anchor (a coordinate centroid for recognized postcodes) and gazetteer lexicon hits (known place names).
- softmax
- The function that converts a vector of logits into a probability distribution summing to 1, applied after priors and biases are added to the emission logits.
- tier
- Internal versioning of which label classes the model emits. Tier 1 is the coarse components (country, region, locality, postcode); Tier 2 adds venue, street, house_number; Tier 3 (future) would add attention, po_box, and POI venue subtyping. Historically called 'Stage 1/2/3' before the runtime-pipeline naming made that ambiguous.
- transformer
- A neural-network architecture introduced in 2017 ('Attention Is All You Need'), the basis of modern NLP models. Mailwoman uses a small, encoder-only transformer.
- transition matrix
- The CRF's learned table of per-label-pair scores. It encodes which BIO transitions are preferred or forbidden — e.g. that an I-tag must follow a matching B- or I-.Also known as: transition scoresReferenced by: Decoding and the best path
- Viterbi decoding
- A dynamic programming algorithm that finds the most likely sequence of hidden states (labels) given a sequence of observations (token emissions). Mailwoman uses Viterbi over a linear-chain CRF to produce globally coherent BIO label sequences from per-token model scores.Also known as: Viterbi, Viterbi algorithm
Rule + neural hybrid14
Rule classifiers and the neural model coexisting — the policy registry, FST priors, and shallow fusion.
- admin FST
- A finite-state transducer encoding Who's On First admin place names (countries, regions, localities, postcodes) for fast gazetteer lookup and emission priors at inference time. Ships to the browser (~9 MB for 94K US admin places).Also known as: gazetteer FST, FST gazetteerReferenced by: CLI
- beam search
- A decoding search that keeps the top-k partial candidates at each position. Used in Stage-5 reconciliation to explore the (phrase, tag, resolver) space with controlled pruning.
- conventions mask
- A decode-time constraint layer keyed by the model's own address-system detection (the exported locale head): tags that are ungrammatical in the detected system are removed from the Viterbi vocabulary, and the system's postcode shape arms a snap-only repair pass. The first slice forbids USPS street-affix decomposition for French. Same knowledge-outside-the-weights property as the gazetteer anchor — add a codex conventions row, no retrain.
- deterministic regex repair
- A post-parse pass that detects patterns the tokenizer fragmented — an alphanumeric postcode shredded into subword pieces, say — and snaps label spans onto regex-matched shapes, fixing zero-F1 cases the model can't recover without retraining.
- emission prior
- A log-probability bias injected into the model's emission logits from an external signal — gazetteer frequency, an FST, a Wikipedia-importance score — combined with the learned weights at decode time.Referenced by: Library API
- FST (finite-state transducer)
- A compact automaton that reads an input sequence and emits an output sequence. Mailwoman encodes gazetteer names and street affixes as FSTs for fast prefix matching and prior injection without search overhead.Also known as: finite-state transducerReferenced by: Add place autocomplete, Belgium — 30 addresses, three configurations, CLI, Library API
- inclusion bonus
- A per-word credit added in log space to accepted spans during reconciliation, so an empty parse can't win on multiplicative scoring just by asserting nothing.Also known as: empty-parse bonus
- joint decoding
- A decode strategy where the neural model's proposals and rule-based solver's output are reconciled into a single parse tree. Formerly the default (Route A Phase II), now retired in favor of argmax after it was found to break the street+house_number geocode precondition.Also known as: joint reconcile
- morphology FST
- A finite-state transducer encoding street-typing affixes (Avenue, rue, Calle) from libpostal dictionaries, for morphological street recognition.Also known as: street-affix FST
- policy registry
- The per-component table that decides which classifier (rule or neural) has authority for each address component. The Ship-of-Theseus dial.Referenced by: What ships today
- shallow fusion
- Blending an external knowledge source (a gazetteer, an FST prior) into a model's decision as a signal, rather than retraining the model or overriding its output. Mailwoman applies it at the input layer (the gazetteer anchor as membership features) and at decode time (an FST prior biasing emissions). RAG-shaped, but the retrieval lands as feature vectors, not prompt text.Referenced by: The gazetteer prior
- Ship of Theseus
- The migration pattern Mailwoman uses: replace rule classifiers with the neural classifier one component at a time, only when metrics justify it. Named for the philosophical thought experiment.
- soft prior
- Outside knowledge fed to the model or resolver as overridable evidence (a feature, a score term) rather than a hard filter. A focus-country hint becomes an anchor feature; a focus-point becomes a ranking term.
- street-supplement architecture
- The four-layer FST stack — morphology, candidacy, identity, schema — that fills the Who's On First street-level hierarchy gap, since WOF has no street node between locality and address.
Staged pipeline15
The six-stage runtime pipeline and the Knowledge Ladder decomposition.
- arbitration
- A pipeline stage that compares rule-based (v0) and neural classifier output, resolving disagreements via a policy registry. Built and merged but not promoted — the coordinate gate showed label-F1 gains came at the cost of worse geocoding.
- character class
- A token's character category — digit, alpha, CJK, Cyrillic, Arabic, mixed — used by the query-shape stage as a structural signal for locale and kind inference.Referenced by: Library API
- input-shape router
- An arbitration prior that reads kind-classifier, query-shape, and coarse-placer signals to set per-component defaults (rule_preferred, neural_preferred, or abstain) according to how clean or out-of-distribution the query is.
- kind classifier
- Stage 2.5 of the runtime pipeline: categorizes the input into one of eight query kinds (structured_address, postcode_only, locality_only, intersection, po_box, landmark, poi_query, vague) so the coordinator can route to the right parsing strategy.
- known format
- A recognized postcode or postal pattern (US ZIP, UK postcode, French code postal) detected by regex in the query-shape stage to arm fast paths and locale hints.
- locale gate
- Stage 2 of the runtime pipeline: rule-based locale detection from the query shape's script and known-format signals. Returns a LocaleHint with the top candidate and alternatives, surfacing disagreement with an explicit --locale flag.
- normalize
- Stage 1 of the runtime pipeline: deterministic input preprocessing (Unicode NFC, punctuation normalization, whitespace collapse). Returns a NormalizedInput with an offsetMap that maps normalized positions back to the raw input.
- offset map
- A data structure that tracks how each position in a normalized string maps back to the original raw input. Essential for reporting parse results (spans) in the user's original text, not the internally normalized form.Also known as: offset mapping
- phrase grouper
- Stage 2.7 of the runtime pipeline: proposes coherent input units (street phrase, locality phrase, postcode, etc.) with structural kind hypotheses. Decouples boundary discovery from type classification so the classifier answers 'what type?' not 'where?'
- phrase kind
- The structural category a phrase-grouper proposal carries — NUMERIC, STREET_PHRASE, LOCALITY_PHRASE, REGION_ABBREVIATION, POSTCODE, VENUE_PHRASE, HYPHENATED_COMPOUND — a 'where, not what' hypothesis.Also known as: PhraseKind
- query kind
- The coarse category the kind classifier assigns to the whole input — postcode_only, locality_only, structured_address, intersection, po_box, landmark, or vague — used to route processing.Also known as: QueryKindReferenced by: Library API
- query shape
- Stage 1.5 of the runtime pipeline: computes a structural fingerprint of the input — script class, segmentation, known-format hits (postcode regexes, state abbreviations) — in microseconds without ML. Used by downstream stages for locale detection and kind classification.Referenced by: Read everything a parse returns
- segment
- A punctuation-bounded chunk of the normalized input — the comma-separated parts of 'Portland, OR' — used to give downstream stages structural context.Referenced by: Build the planet, Build the US address dataset from public sources, Geocode a CSV of customer addresses, Handle PO boxes and other edge kinds, How close is close enough?, Improve geocode precision, mailfail — robustness on garbage, malformed and hostile input, Parse in the browser +1 more
- stage
- One of the dataflow stages in the runtime pipeline (normalize, locale gate, kind classify, phrase group, token classify, sequence correct, reconcile, resolve). Distinct from tier (model vocabulary) and phase (plan milestone).Also known as: runtime stage
- staged pipeline
- Mailwoman's runtime architecture: a sequence of pure-function stages (normalize → query-shape → locale-gate → kind-classifier → phrase-grouper → classifier → decoder) connected by typed handoffs. Each stage is published as its own npm package.Also known as: runtime pipeline, pipeline
Resolver / WOF22
The Who's On First gazetteer, concordance scoring, and resolver candidate ranking.
- coherence
- The property of a parse whose resolved places form a consistent geographic hierarchy — the resolved locality really does sit inside the resolved region.
- coincident roles
- A precomputed Who's On First relation that pairs a region with its same-named locality (Berlin city ↔ Berlin state) so the resolver can complete the hierarchy at resolution time.Also known as: coincident-roles relation
- concordance
- A joint-decode signal that rewards a parse whose spans form a consistent Who's On First parent-child chain and vetoes contradictory ones — a hard veto for conflicts, a log-space bonus for full agreement.Also known as: concordance bonus
- country posterior
- A country → probability map (derived from postcodes or the coarse-placer) that re-ranks resolver candidates as a soft prior, never a hard filter.Also known as: anchorPosterior
- dual-role place
- A place that is two placetypes at once — Berlin as both a city and a state, Washington DC as city and district. Resolved using the coincident-roles relation plus hierarchy completion.Also known as: city-state
- epsilon floor
- A small cutoff (ε) below which per-country posterior entries are dropped before the resolver sees them, so implausible tail probabilities cannot influence ranking. Domain: [0, 1]; 0 disables it (the full distribution passes through — the shipped default); higher values keep only stronger beliefs, at the extreme leaving just the argmax (a one-hot). Mailwoman defaults to 0 because no nonzero value has measured benefit on the misroute battery (the 2026-07-03 floor sweep was null at 0.05–0.30); the knob exists for distribution-mode experiments (--posterior-floor).Also known as: posterior floor, ε-floor
- exact-match tiering
- A resolver ranking strategy that groups candidates by match quality (exact name/alias match vs partial) and ranks within each tier by secondary signals, so a strong population prior can't promote a poor name match.
- gazetteer
- A geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture.
- GeoNames
- A free global gazetteer combining administrative, postal, and POI data across 200+ countries. Supplements Who's On First for postcode centroids and places where WOF has gaps.Referenced by: Build the planet, Gazetteers
- hierarchy completion
- The post-resolution step that synthesizes a dropped locality when a dual-role region resolves but the parser omitted the city (e.g. supplying 'Berlin' the city under 'Berlin' the state).
- importance score
- A precomputed per-place prominence score (blending Wikipedia importance with population) used to rank same-name gazetteer candidates.Referenced by: Add place autocomplete
- parent chain
- The sequence of administrative parents above a place — Springfield → Sangamon County → Illinois → United States. Used to check the geographic coherence of a parse.Also known as: ancestor chainReferenced by: Reverse geocode a coordinate
- parent fallback
- Retrying a zero-result parent-constrained resolver lookup ('find this locality inside this region') without the parent constraint but keeping the country, so a slightly-wrong parent doesn't produce silent zero results.
- placetype
- The Who's On First hierarchical classification of places: planet → continent → country → region → county → locality → neighborhood. The resolver uses placetype to rank candidates — an exact locality match outranks a county-level match.
- posterior (country posterior)
- A per-country probability map — e.g. {GB: 0.8, FR: 0.06} — expressing how likely an input belongs to each country, produced by a model (the coarse placer's softmax, or a postcode anchor's lookup) AFTER seeing the input (hence 'posterior', from Bayesian usage: the updated belief). The resolver consumes it as a soft ranking boost (anchorPosterior): each candidate's score gains weight × posterior[candidate.country]. Values lie in [0, 1] and need not sum to 1 (in-map marginals exclude the OTHER class).Also known as: anchorPosterior, in-map posterior, country posterior
- prominence (candidate prominence)
- A candidate place's combined importance for within-tier ranking: the population term (log-scaled, capped at populationBoost, default 4.0) plus the best proximity-bias term (distance-decayed, capped at biasBoost, default 4.0). Domain: [0, populationBoost + biasBoost], typically [0, 8]; higher = more prominent = ranked earlier WITHIN an exact-match tier. It exists because raw text-match scores (bm25) are length-poisoned for famous places, so ties among equally-exact matches break by prominence instead of score.Also known as: candidate prominence
- prominence signal
- A ranking input that favors the better-known place among same-named candidates — the original Eiffel Tower in Paris over a replica in Las Vegas. Closely related to the importance score.Also known as: prominence score
- resolver
- The component that converts parsed address components (locality, region, postcode) into coordinates by looking them up in the gazetteer. The resolver ranks candidates by name match, population, and proximity, and returns the best-matching place with its centroid or polygon.
- soft signal
- An informative but non-authoritative input to ranking — user location, a language prior, a coarse-country guess — that biases the resolver but never hard-filters candidates. The resolver-side cousin of a soft prior.
- Wikipedia importance
- A place-notability score derived from the count and language spread of a place's Wikipedia articles, normalized to [0, 1]. A resolver ranking prior that helps pick the prominent same-named place.Also known as: Wikipedia rank
- WOF (Who's On First)
- An open-source gazetteer of places maintained by Mapzen/whosonfirst. Mailwoman builds a custom SQLite database from WOF GeoJSON repos, extended with postcode data, importance scores, and coincident-role relations.Also known as: Who's On First
- WOF GeoJSON
- The raw one-feature-per-file GeoJSON distributed by Who's On First repositories — the input that Mailwoman's WOF SQLite build consumes.Also known as: raw WOF
Record matching10
The geocode-first record matcher — blocking, scoring, clustering, and entity resolution.
- blocking
- The first stage of entity resolution: generate candidate record pairs using cheap, high-recall keys (geo cell, canonical address, phone) instead of comparing every record to every other (O(n²)). The matcher only scores pairs that survive blocking.
- canonical key
- A deterministic, normalized string representation of an address produced by @mailwoman/formatter. Lowercase, abbreviation-expanded, punctuation-stripped — so '123 Main St' and '123 MAIN STREET' produce the same key. Used for blocking in the matcher.Also known as: match key
- canonicalization
- Mapping a span to its canonical value: 'St' and 'Street' both become the USPS suffix ST; 'USA' and 'United States' both become ISO US. The operation that handles synonymy. Canonical values come from @mailwoman/codex, not a hand-grown alias list.
- clustering
- The final stage of entity resolution: resolve non-transitive pairwise match decisions (A↔B, B↔C, but not A↔C) into canonical entities via union-find with path compression. Each cluster of records becomes one resolved entity.Referenced by: Packages
- edit distance
- The minimum number of single-character insertions, deletions, or substitutions needed to turn one string into another (Levenshtein distance). Used in corpus alignment and record matching.Also known as: Levenshtein distance
- expectation-maximization
- An iterative algorithm that estimates model parameters when some variables are unobserved. In Mailwoman's matcher, EM learns the Fellegi-Sunter m and u parameters from unlabeled data — no training labels needed.Also known as: EM, EM estimationReferenced by: CLI
- Fellegi-Sunter
- A probabilistic record linkage model that computes match probability from agreement-level log-likelihood ratios: log₂(m/u) where m is the probability of agreement given a true match and u is the probability of agreement by chance. Mailwoman learns m and u label-free via expectation-maximization.Also known as: FS model, Fellegi-Sunter modelReferenced by: CLI, Match messy records to one entity each
- GBT (Gradient Boosted Trees)
- A non-linear machine learning model that combines many weak decision trees into a strong predictor. Mailwoman uses a GBT as an optional learned scorer for single-dataset dedup, improving F1 by 5–7 percentage points over the Fellegi-Sunter baseline.Also known as: GBM, gradient boostingReferenced by: CLI
- geocode-first
- The matcher's core design principle: resolve addresses to geographic coordinates first, then compare the resolved places — not the raw address strings. Two records at the same coordinates match even if one says '123 Main St' and the other says '123 MAIN STREET.'Referenced by: Match messy records to one entity each
- record matching
- The process of determining whether two database records refer to the same real-world entity. Mailwoman's matcher uses a geocode-first approach (match the resolved place, not the address string) with Fellegi-Sunter probabilistic scoring.Also known as: entity resolution, deduplication, record linkage
Corpus pipeline12
The training-data pipeline — adapters, alignment, synthesis, Parquet shards.
- alignment
- The step in the corpus pipeline that takes a (raw, components) pair from an adapter and produces a (raw, tokens, BIO labels) row by finding each component's text inside the raw string and labeling the matching tokens.Referenced by: Training and the corpus
- corpus
- The BIO-labeled training data used to train Mailwoman's neural classifier. Assembled from real sources (OpenAddresses, National Address Database) and synthetic shards (boundary stress, order variants, negative space). Managed by @mailwoman/corpus.
- label vacuum
- A (token, label) pairing introduced by a synthetic shard with zero support in the existing corpus — a corpus-poisoning risk, since the model has no other evidence to anchor it.
- LLM-as-corpus-generator
- Using a large language model to synthesize annotated training rows for low-data or multilingual cases, gated by alignment validation that rejects rows whose components don't substring-match the surface.
- locality-aware split
- Partitioning the corpus so all rows from one locality land in the same train/val/test split, preventing the model from memorizing a place in training and being scored on it in test.
- multi-order training
- Synthesizing the same address in multiple word orders — native postcode-first vs international house-number-first — so the model learns to read both layouts rather than overfitting one direction.
- negative space
- Address formats the neural model was never trained on because they're absent from the base training corpus — PO boxes, CEDEX, intersections, units. The parity campaign targets negative space through synthetic shards and corpus augmentation.
- Parquet
- The open columnar file format the corpus is written and streamed in. The training pipeline reads shards row-by-row from Parquet.Referenced by: Build the planet, Build the US address dataset from public sources
- reject rate
- The fraction of generated or harvested rows that fail validation and are excluded from the corpus — a quality gauge for a synthesis source.
- shard
- A partial output file of the corpus build, written in Parquet format. The training pipeline streams shards row by row.
- source weighting
- Multiplicative per-source weights applied during training to oversample underrepresented corpus sources, balancing a corpus dominated by a few large datasets.Referenced by: Training and the corpus
- synthetic shard
- A machine-generated training dataset that augments the real-address corpus with targeted variations — reversed word order, missing punctuation, all-caps, boundary-stress patterns. Teaches the model to handle edge cases not present in the reference data.
Model training38
Training runs, loss recipes, verdict smokes, and divergence diagnostics.
- absence curriculum
- Training-time ramped per-row zero-out of an evidence channel's confidence so the model keeps label competence with AND without the clue — the anti-over-reliance leg. The ablation gate (evidence-zeroed parse ≥ baseline) is its eval-side check.Also known as: evidence curriculum, anti-over-trust curriculum
- backpropagation
- The algorithm that computes the gradient of the loss with respect to every parameter by applying the chain rule backward through the network. The engine behind neural-network training.Also known as: backprop
- batch size
- How many examples the model processes before each parameter update. Larger batches give smoother gradients but cost more memory; gradient accumulation simulates a big batch on a small GPU.Also known as: batch, minibatch
- bf16 (bfloat16)
- A 16-bit floating-point format used during training. Half the memory of fp32 with similar numeric range, which lets the training batch fit on the lab's GPU.Also known as: bfloat16
- checkpoint
- A saved snapshot of the model weights and optimizer state during training. Mailwoman saves a checkpoint periodically so training can resume after a GPU hang.Referenced by: Training and the corpus
- cross-entropy
- The standard classification loss: it penalizes a model for putting low probability on the correct label. Per-token negative log-likelihood is the cross-entropy of each token's label.Also known as: cross-entropy loss, CE lossReferenced by: Decoding and the best path, Training and the corpus
- cumulative dilution
- The effect where stacking many synthetic shards into one run spreads the step budget thin and weakens each shard's target tag. Shards are best proven solo, then consolidated with a larger step budget.
- divergence
- A training failure where loss descends through warmup, plateaus low for a while, then climbs catastrophically back to its starting magnitude — the model unlearns everything despite no obvious component failure.Also known as: training divergence
- dropout
- A regularization trick that randomly zeroes a fraction of activations during training, forcing the model not to rely on any single feature. Disabled at inference.
- EWC (elastic weight consolidation)
- The fine-tune forgetting brake: add λ/2·ΣF_i(θ_i−θ*_i)² to the loss, penalizing movement of parameters the base's Fisher says carried capability. Fresh heads are unpenalized (new capability trains freely; base capability is braked). The parameter-space half of the fine-tune guarantee; the acceptance battery is the behavior-space half.Also known as: EWC brake
- false-evidence noise
- The collision-robustness training leg the absence curriculum lacks: with small probability, a row's evidence features are corrupted so the model also learns to override WRONG evidence, not just survive missing evidence.Also known as: evidence noise
- Fisher artifact
- The diagonal empirical Fisher information captured over the final ~2k steps of a base run (mean squared gradient per parameter) and shipped beside the checkpoint (fisher-diag-v1.npz + provenance sidecar). Capturable cheaply ONLY during the base run; the input to the EWC brake.Also known as: consolidation artifact, fisher-diag
- fp32 / fp16
- 32-bit and 16-bit floating-point formats. Mailwoman trains in bf16 (a 16-bit variant) and exports the ONNX model in int8 for size.Also known as: fp32, fp16Referenced by: Footprints, What ships today
- generalization
- A trained model's performance on data unlike its training set — new regions, new input distributions. The property honest eval is designed to measure.
- gradient
- The direction and rate at which the loss would change if each parameter were nudged. Training follows the gradient downhill to reduce error. Huge gradients are tamed by gradient clipping.Referenced by: Match messy records to one entity each, The landscape
- gradient accumulation
- Summing gradients over several minibatches before taking an optimizer step, to simulate a larger effective batch size under limited GPU memory.
- gradient clipping
- A training trick: if the gradient norm exceeds a threshold, scale it down. Stops a single bad batch from blowing up the model weights.
- gradient descent
- The optimization method behind training: repeatedly compute the gradient on a batch and step the parameters a small amount in the downhill direction. 'Stochastic' gradient descent uses one minibatch at a time.Also known as: SGD, stochastic gradient descent
- hyperparameter
- A training setting chosen by the engineer rather than learned by the model — learning rate, batch size, number of layers, label-smoothing strength. Tuning hyperparameters is much of the craft of training.
- label smoothing
- A training regularization technique: instead of putting 1.0 probability on the correct label, train to put 1 − ε on it and ε/(N−1) on each wrong label. Improves calibration; disabled in some Mailwoman runs for stability.
- learning rate (LR)
- How big a step training takes along the gradient each update. Too high and training diverges; too low and it crawls. Mailwoman warms it up, then decays it on a cosine schedule.
- learning-rate schedule
- A plan for changing the learning rate over training — Mailwoman's is linear warmup to a peak, then cosine decay toward zero. Smooths early instability and lets the model settle at the end.Also known as: LR schedule, cosine decay
- lever-shape taxonomy
- A classification of parity-gap fixes by mechanism: distributional tags (street_prefix, unit, locality) need synthetic shards plus retraining; closed-vocab tags (country, po_box, cedex) respond better to deterministic matchers as proposal sources.
- loss function
- A number measuring how wrong the model's predictions are on a batch of examples. Training minimizes it. Mailwoman's loss combines per-token negative log-likelihood with the CRF sequence loss.Also known as: training objective, loss
- model card
- A JSON metadata file (model-card.json) shipped with each weights bundle. It declares the model version, lineage, label set, required inference channels (anchor, gazetteer), calibration data, and training provenance.Referenced by: CLI, Component tags, Decoding and the best path, Footprints, Library API, Locales and tiers, Packages, Runtime flags +3 more
- NaN (not a number)
- A floating-point result for an undefined operation (log of a negative, 0/0). Appearing in the training loss usually halts the run; recovering from it follows the NaN protocol.
- NaN protocol
- The NaN-recovery discipline: stop, change one variable, retry, document the hypothesis — never adjust two knobs at once during recovery, or you can't tell which change fixed it.
- NLL (Negative Log Likelihood)
- The standard form of a loss function in probability-based models: loss = −log P(correct label | input). The CRF uses the NLL of the entire label sequence.
- optimizer
- The component that decides how to update parameters from the gradient — Adam/AdamW being the common choice, adding momentum and per-parameter scaling on top of plain gradient descent.Also known as: Adam, AdamW
- over-trust decay
- The mid-run peak-then-decline an evidence-channel FINE-TUNE shows as the model progressively over-relies on an always-on clue (peak ~4-6k, decay by 8k under constant LR). Absent in the from-scratch cosine regime — a recipe-class artifact, not a property of the bundle.
- overfitting
- When a model memorizes quirks of its training set instead of learning general patterns, so it scores well in training but poorly on new data. Guarded against with held-out evals and regularization.Also known as: memorization
- rare-class forgetting
- The structural fine-tune failure mode: a class too small to matter in the gradient bulk erodes under any fine-tune objective (fr.cedex, ~10/430 rows, in every bundle-era recipe). Mirrored by rare-class UNDER-learning: a short low-LR window also cannot TEACH a rare class the base never knew (date-name, 0.133 through every increment, 0.762 from scratch).Also known as: rare-class under-learning
- regularization
- Techniques that discourage a model from overfitting — label smoothing, dropout, weight decay. They trade a little training accuracy for better generalization.
- starved
- A tag with too little training representation to learn — near-zero F1 — because the corpus adapter never emits examples of it (intersection tags sat at 0% until an intersection synthesizer existed).
- training
- The process of adjusting a model's parameters so its predictions match labeled examples, by repeatedly measuring error and nudging the weights to reduce it. Distinct from inference, when the trained model is run on new input.
- underfitting
- When a model is too simple or undertrained to capture the patterns in its data, so it performs poorly even on the training set. The opposite failure from overfitting.
- verdict-smoke
- A short diagnostic training run (≈1,500–3,000 steps) that must match the full run's gradient-noise profile, used to catch recipe instability before committing to a full 50k-step launch.Also known as: smoke test
- warmup
- The early phase of training where the learning rate ramps up from 0 to its peak value before cosine decay. Mailwoman uses a linear warmup.
Tokenization10
SentencePiece tokenization, byte-fallback, and multi-script handling.
- byte fallback
- A tokenization strategy where characters not seen during training are encoded as raw UTF-8 byte sequences rather than mapped to an unknown token. Mailwoman's tokenizer uses byte fallback so non-Latin scripts (CJK, Cyrillic) produce real token sequences even though training data was Latin-dominant.
- fertility
- How many tokenizer pieces it takes to spell a word: fertility 1 means whole words survive tokenization, fertility 3 means they arrive in fragments. High fertility hurts sequence labeling directly — the model has to reassemble a word before it can label it, and fragment boundaries are where labels break. Mailwoman measured this on French street names: accented words splitting at the accent ('René' → '▁Ren' + 'é') caused real geocoding misses, and lowering French fertility was worth a tokenizer revision plus a retrain.
- padding token
- A filler token added to short sequences to reach a fixed length, masked out during attention so it carries no meaning.Also known as: PAD
- SentencePiece
- A language-independent subword tokenizer that splits text into pieces using a unigram language model. Mailwoman uses a SentencePiece tokenizer with a 48,000-token vocabulary and byte-fallback, trained on address data rather than general text.
- subword tokenization
- Splitting words into smaller learned pieces so a fixed vocabulary can spell any word — common words stay whole, rare ones break into parts. The approach SentencePiece implements.
- token
- One word or subword in the tokenized input. For the neural classifier, tokens come from SentencePiece (subword units); for the rule classifiers, tokens are whitespace- and punctuation-separated words.
- tokenizer
- The component that converts a raw address string into a sequence of numeric token IDs the model can process. Mailwoman's tokenizer is a SentencePiece unigram model trained specifically on postal addresses.
- truncation
- Cutting an input down to the max sequence length (or an LLM response to its token limit), discarding everything past the cap.
- unknown token
- A placeholder for input not in the tokenizer's vocabulary. Byte fallback largely removes the need for one by encoding unseen characters as raw bytes instead.Also known as: UNK, OOV
- vocabulary
- The fixed set of tokens a tokenizer can produce. Mailwoman's SentencePiece vocabulary is tens of thousands of subword pieces, with byte fallback for anything outside it.Also known as: vocab, token inventoryReferenced by: Compare, Footprints, Install the Claude Code skill, Pelias and libpostal, Read everything a parse returns, The gazetteer prior, The landscape, Tokens and labels +6 more
Street components7
Street names, building numbers, street suffixes, and intersection addresses.
- dependent street
- A secondary street name required for delivery in some postal systems (notably Royal Mail), as in '6 Elm Avenue, Runcorn Road, Birmingham' where Runcorn Road is the dependent street.
- designator
- The closed-vocabulary leading word of a secondary-address phrase — 'Apt', 'Suite', 'Floor', 'PO Box', 'Level' — paired with an identifier to form a complete subpremise.Referenced by: Component tags
- house number
- The numeric or alphanumeric identifier of a building on a street. Mailwoman's house_number component; its position relative to the street name flips between locales.Also known as: street number
- street
- The named linear feature along which house numbers are ordered. Decomposes into a name plus street affixes; one of the Tier 2 fine labels.
- street affix
- A modifier on a street name indicating type or direction — Street, Avenue, rue, Calle, N, East. Mailwoman tags these as street_prefix / street_suffix, recognized via a morphology FST.Also known as: street suffix, street prefix
- subpremise
- A secondary address unit inside a single street address (apartment, suite, floor), typically a designator plus an identifier such as 'Apt 4B'. The same proposer family as PO box.
- unit
- A subdivision of a building — apartment, suite, floor — that refines a street address. Mailwoman's unit component; a designator plus identifier forms a subpremise.
Locality components4
Cities, towns, neighborhoods, boroughs, and administrative place names.
- borough
- An administrative or historical division of a city — e.g. the five boroughs of New York City. May be postal, legal, or both, and complicates the locality hierarchy.
- dependent locality
- A sub-locality (neighborhood or borough) hierarchically inside a larger locality — e.g. Brooklyn within New York City. Provides finer geographic specification below the primary locality.Also known as: sub-locality
- locality
- The city / town / settlement component of an address: a populated place sitting between region and neighborhood in the hierarchy.Also known as: city, municipalityReferenced by: Add place autocomplete, Belgium — 30 addresses, three configurations, Build the planet, Calibration and confidence, Capabilities, CLI, Compare, Component tags +37 more
- microhood
- A Who's On First placetype for a very fine-grained neighborhood subdivision, sitting below neighborhood in the hierarchy.
Postcode components4
ZIP codes, postcodes, postal routing codes, and their formats.
- CEDEX (Courrier d'Entreprise à Distribution Exceptionnelle)
- A French postal routing for high-volume business mail: a CEDEX code delivers directly from a sorting center, bypassing the local post office. A common negative-space format Mailwoman must parse.Referenced by: Component tags
- check digit
- A digit appended to a postcode or identifier and derived by modular arithmetic, used to detect transcription errors.
- postcode
- The country-specific postcode (US ZIP, French code postal, etc.). Mailwoman handles postcode parsing entirely by rule classifier — a regex problem, not an ML one.Referenced by: Addressing around the world, Belgium — 30 addresses, three configurations, Benchmarks, Build the planet, Calibration and confidence, Capabilities, CLI, Compare +39 more
- ZCTA (ZIP Code Tabulation Area)
- The US Census Bureau's polygon approximation of a ZIP code, generalized from census blocks rather than USPS delivery routes. Explicitly not USPS ground truth and often wrong in rural areas.Referenced by: Postcodes and ZIP Codes
Region components1
States, provinces, départements, and first-level administrative divisions.
- region
- The first-level administrative subdivision of a country — a US state, a French region, a province. The component between country and locality.Also known as: state, provinceReferenced by: Build the planet, Calibration and confidence, Capabilities, CLI, Component tags, Decoding and the best path, Footprints, Gazetteers +25 more
Country components1
Sovereign states, territories, and country-level addressing conventions.
- country
- The top-level address component (an ISO country). Closed-vocabulary, so it is best handled by a deterministic matcher feeding a proposal rather than a retrained model head.
Venue / POI7
Venue names, points of interest, organizations, and the venue-vs-address ambiguity problem.
- amenity
- A point of interest referenced by category ('gas station', 'pharmacy', 'ATM') rather than by name. Resolved by mapping the query onto a category taxonomy.Also known as: POI category
- franchise query
- A point-of-interest query naming a brand or chain ('nearest Starbucks') rather than a category or an address.Also known as: brand query
- point of interest (POI)
- A named place that is not strictly an address — landmark, transit stop, venue, amenity, or franchise. Mailwoman tags these as venue and resolves them through the gazetteer.Also known as: POIReferenced by: Build the planet, CLI, Deployment options, Handle PO boxes and other edge kinds, Library API, Locales and tiers, Packages, Runtime flags +3 more
- regional variant
- A local term for an amenity or brand differing from the global name — 'servo' for a gas station in Australia, 'bodega' for a corner store in NYC.Also known as: localism
- station entrance
- One of several access points to a large transit station. Entrances at stations like Tokyo or Shibuya can be hundreds of meters apart, so entrance-level precision matters for navigation.
- sub-brand
- A branded variant within a larger franchise — Walmart Supercenter vs Walmart Neighborhood Market vs Sam's Club, all under Walmart.Also known as: brand variant
- venue
- A named, non-address place — a business, building, park, or stadium. Mailwoman's free-text point-of-interest component, added as a Tier 2 fine label.Also known as: named place
Intersection addresses2
Crossing-street addresses and street-pair location descriptions.
- intersection
- An address that names a location by two crossing streets ('5th & Main') rather than a number and street. Mailwoman tags the two streets as intersection_a and intersection_b — a negative-space format that starved the early model.Also known as: cross-street, corner address
- named intersection
- A crossing that is itself a named place — common in Japan, e.g. the Shibuya Station-front intersection — rather than a pair of street names.
Multilingual2
Addresses in multiple languages, mixed-language input, and locale detection.
- code-switching
- Mixing two languages in a single query, e.g. 'スタバ near Shibuya Station' (a Japanese brand name with an English location constraint).
- transliteration
- Converting a name from one writing system to another while preserving pronunciation (Cyrillic → Latin, for instance). Needed for multilingual address handling and corpus synthesis.Also known as: script conversion
Project motivation1
The case for Mailwoman — why neural, why TypeScript, why a staged pipeline.
- bitter lesson
- Sutton's observation that general methods which scale with compute and data ultimately beat hand-engineered systems. The rationale behind Mailwoman's Ship-of-Theseus migration from rule classifiers to a learned model.
Model7
The trained artifact itself — weights, versions, lineage, bases vs fine-tunes.
- char path
- The CharCNN input route (v8 CJK): per-unit char_ids (S,W) — S label units, W a positional composition window — composed into token embeddings by a multi-width CNN, replacing SentencePiece entirely. Latin char-word mode (unit=whitespace token, ctx 0) and CJK char mode (unit=one character, ctx 3 → W 7) share one code route.Also known as: CharCNN path, char mode
- composition window
- The W dimension of the char path's (S,W) contract: the characters describing one label unit — a whole word for Latin char-word mode, the character ± ctx neighbors for CJK (ctx 3 → W 7, giving the CNN local n-gram context for designators like 丁目).
- consistent random marker
- What an UNTRAINED evidence channel becomes at inference: real painted features flowing through a frozen random-init projection — a fixed random direction injected at hit tokens. Real, reproducible behavior (it measurably helped homonyms in the lowercase register) that is an accident of initialization, not a learned mechanism.Also known as: random marker
- evidence bundle
- The pair of retrieval-augmented input channels (street-type + locality-surface) that feed lexicon membership as soft per-token evidence alongside the text. Shipped in 6.7.0; trained natively from step 0 in the from-scratch base line.Also known as: bundle
- evidence channel
- A dedicated model input that injects externally computed per-token features (confidence-scaled, own projection) at the embedding layer: postcode anchor, gazetteer, country, street-type, locality-surface. The clue informs; the model decides (model-first).Also known as: soft-feed channel
- golden master
- The from-scratch base model every capability comes from in ONE recipe with one provenance and zero named watches at ship — the starting line every fine-tune engagement inherits, with its Fisher artifact beside it.Also known as: base model
- label set
- A model's BIO vocabulary, per-config since CJK Phase 2: stage3 (the Latin 33), stage3-jp (STAGE3 + the seven JP tags = 47, the JP char head), stage4 (the secondary-address family — numerically also 47, a coincidence). A checkpoint persists its own id→label map; label-space mismatches raise instead of collapsing silently.
Evidence channels2
The evidence-bundle inputs — lexicon-painted channels, presence calibration, absence curricula.
- digit guard
- A lexicon rule (carried in the artifact so both painters apply it): a matched span paints NO evidence when any span word or nearest non-empty neighbor carries a digit. Turned the bundle's alnum-housenumber damage from −0.325 into +0.013.
- painter fold
- The word normalization BOTH evidence painters apply at lookup: per whitespace word, strip leading/trailing non-letter/digit characters (keep internal), lowercase, single-space join. Lexicon keys must use this fold or they are unreachable at paint time. Deliberately different from the FST fold.Also known as: painting
Lexicons3
The painted evidence lexicons — street-type, locality-surface, the four-law selectivity rules.
- city-state exception
- The DE carve-out in law-4 region-vocabulary exclusion: Berlin/Hamburg/Bremen stay locality evidence because Land and Stadt are one coextensive place whose dominant reading is the locality — unlike the Washington class (state and city are DIFFERENT places), which is excluded. Brandenburg is the Washington class and is excluded.
- four-law selectivity
- The curation laws deciding which surfaces may carry locality evidence: (1) degenerate exclusion (function words, street-type words, directionals), (2) prominence floor (importance ≥ 0.25 for 1-token surfaces), (3) person-name tier (≥ 0.45), (4) region-vocabulary exclusion (state/Land names, with the DE city-state exception). Each law was bought with a falsified training run.Also known as: three-law selectivity
- parent vouch
- The lexicon prominence rule for neighborhoods, which structurally lack population rows: a neighborhood surface's prominence-floor input is max(own importance, parent locality importance) — Montmartre is prominent BECAUSE Paris is. Person-name surfaces are exempt (only OWN metropolis-tier importance clears them).
Process discipline2
How model work is run — pre-registration, named watches, envelopes, stop rules.
- D-rule
- Iron rule 6: no default-on mechanism (model or decode-time) ships with a known regression vs the shipped model on any tier-1 locale — fix, per-locale-gate, or opt-in. The gate every cut passes through.Also known as: iron rule 6Referenced by: resolver_score cannot reject garbage — a characterization
- pre-registration
- Writing the full gate sheet — every bar, baseline identity, and stop rule — before any GPU spend. Bars are bars: a below-bar number is a miss to confront (re-baseline with a stated reason, or iterate), never a quiet pass; deviations are recorded, never silently re-read.Also known as: bars are bars
Methods1
Named diagnostic techniques — fingerprints, probes, ablations, and attribution tools.
- zeros-init fingerprint
- The cheap decisive test for 'did channel X ever train': a zeros-initialized parameter moves on ANY gradient, so an exactly-zero tensor in the exported graph proves the channel never received one. How the untrained locality channel (#1349) was proven in the shipped artifact.Also known as: fingerprint
Doctrine1
The standing principles behind design calls — user register, positive evidence only, model-first.
- user register
- The doctrine that lowercase, comma-light, fragmentary input is the PRIMARY register real users type (Google Maps trained them). Every eval gets a lowercase leg; no mechanism may key on case. Evidence value peaks in this register because the case cue is dead.Also known as: lowercase register
Runtime behavior4
Decode-time and serving-time mechanisms — input modes, gates, riders, and routers.
- input mode
- The Decision-A register switch: 'fragmented' (human-typed fragments — feeds the evidence channels) vs 'formatted' (complete records — runs the trained absence identity). Explicit on CLI/API; per-endpoint defaults (batch→formatted, autocomplete→fragmented); kind-derived otherwise.Also known as: register, fragmented, formatted
- retry rider
- The geocode fallback: a zero-hit result whose register was DERIVED (not explicit) triggers one re-parse in the flipped register before giving up. Measured 3/26 zero-hit recovery on golden; regression-free by construction.Also known as: alternate-register retry
- script router
- The query-shape dispatch for the sibling-model architecture: a Unicode-block histogram routes CJK-dominant input to the char model and everything else to the SentencePiece model — making Latin non-regression PROVABLE (byte-identical routing) rather than measured.
- street-context gate
- The decode-time seam that zeroes locality evidence on inputs with no street-painter hits — bare lookups must not have city names painted into street structure. Every measured evidence win class carries street words, so the gate costs nothing.