Keep your data current

Outcome. You can tell which vintage of each data artifact is on disk, refresh one that has changed, and switch a long-running process to a new version without restarting it.

One thing to know before the steps: mailwoman data pull checks whether a file is there rather than whether it is current. mailwoman data status is the freshness check. When you run data pull on a stale file, the command reports it as already present and performs no download.

Prerequisites

• A data root, and $MAILWOMAN_DATA_ROOT pointing at it. Every transcript below uses /tmp/mw-fresh-demo.
• At least one bundleevidence bundleThe pair of retrieval-augmented input channels (street-type + locality-surface) that feed lexicon membership as soft per-token evidence alongside the text. Shipped in 6.7.0; trained natively from step 0 in the from-scratch base line. already pulled. Your first ten minutes pulls candidate; Improve geocode precision pulls the per-state databases.
• mailwoman 8.7.0 or later. data pull and data status do not exist before it.

1. Read what you have

mailwoman data status candidate

✓ candidate: wof/candidate.db: present (1652.9 MB) at /tmp/mw-fresh-demo/wof/candidate.db
PASS (1/1 checks)

When you pass no bundleevidence bundleThe pair of retrieval-augmented input channels (street-type + locality-surface) that feed lexicon membership as soft per-token evidence alongside the text. Shipped in 6.7.0; trained natively from step 0 in the from-scratch base line., data status reports on all four (candidate, poi, fr, us) and prints one line per artifact — 106 lines, because the us bundleevidence bundleThe pair of retrieval-augmented input channels (street-type + locality-surface) that feed lexicon membership as soft per-token evidence alongside the text. Shipped in 6.7.0; trained natively from step 0 in the from-scratch base line. is 103 files. Name the bundleevidence bundleThe pair of retrieval-augmented input channels (street-type + locality-surface) that feed lexicon membership as soft per-token evidence alongside the text. Shipped in 6.7.0; trained natively from step 0 in the from-scratch base line. you care about.

The check is a byte-count comparison against the size recorded for the published artifact, so it catches a truncated or half-copied file:

mailwoman data status candidate

✗ candidate: wof/candidate.db: stale — 900.0 MB on disk vs 1652.9 MB (recorded) at
/tmp/mw-fresh-demo/wof/candidate.db
FAIL (0/1 checks)

The exit code is 0 only when every artifact is present and matches. That makes it usable as a check that blocks a deploy.

2. Refresh what has moved

A stale artifact is the case data pull will not fix on its own:

mailwoman data pull candidate --dry-run

✓ candidate: gazetteer/2026-07-07a/candidate.db: already present (/tmp/mw-fresh-demo/wof/candidate.db) —
skipped
PASS (1/1 checks)

That transcript was produced against the 900 MB truncated file from step 1. The skip is a filesystem existsSync and no more. --force is what makes the download happen:

mailwoman data pull candidate --dry-run --force

✓ candidate: gazetteer/2026-07-07a/candidate.db: [dry-run] 1652.9 MB
https://public.mailwoman.ai/mailwoman/gazetteer/2026-07-07a/candidate.db →
/tmp/mw-fresh-demo/wof/candidate.db

Drop --dry-run to run it. The download streams to <data-root>/tmp/, is verified against the Content-Length the server sent, is sealed, and only then moves into place — so a failure partway through leaves the old file untouched rather than a corrupt new one. --dry-run makes no network call at all: the plan comes from the registry and a filesystem stat.

For the us bundleevidence bundleThe pair of retrieval-augmented input channels (street-type + locality-surface) that feed lexicon membership as soft per-token evidence alongside the text. Shipped in 6.7.0; trained natively from step 0 in the from-scratch base line., --only scopes the refresh to one state rather than all 103 files:

mailwoman data pull us --only dc --force

3. Ask the bucket instead of the registry

The size data status compares against is the one recorded when the bundleevidence bundleThe pair of retrieval-augmented input channels (street-type + locality-surface) that feed lexicon membership as soft per-token evidence alongside the text. Shipped in 6.7.0; trained natively from step 0 in the from-scratch base line. was surveyed. --check-remote replaces it with a live HEAD request per artifact:

mailwoman data status candidate --check-remote

[DEBUG] (mailwoman data status): HEAD: https://public.mailwoman.ai/mailwoman/gazetteer/2026-07-07a/candidate.db
[DEBUG] (mailwoman data status): 200 (uncached) HEAD: https://public.mailwoman.ai/mailwoman/gazetteer/2026-07-07a/candidate.db
✓ candidate: wof/candidate.db: present (1652.9 MB) at /tmp/mw-fresh-demo/wof/candidate.db
PASS (1/1 checks)

The visible difference is one word in the stale line — (live) where the offline run prints (recorded). Only artifacts already on disk are probed, so the flag stays cheap even against the us bundleevidence bundleThe pair of retrieval-augmented input channels (street-type + locality-surface) that feed lexicon membership as soft per-token evidence alongside the text. Shipped in 6.7.0; trained natively from step 0 in the from-scratch base line.: a missing file is reported from the registry without a request.

Two limits on what this proves. No bundleevidence bundleThe pair of retrieval-augmented input channels (street-type + locality-surface) that feed lexicon membership as soft per-token evidence alongside the text. Shipped in 6.7.0; trained natively from step 0 in the from-scratch base line. publishes an md5 sidecar today, so the comparison is byte count, with no stronger check. A rebuild that landed at the same size would pass. And the remote path is dated and immutable (the transcripts above show gazetteer/2026-07-07a/candidate.db; the current build is gazetteer/2026-08-25b/candidate.db at 2.88 GB), so a live HEAD confirms your copy of that artifact rather than that a newer artifact exists. A new build is a new path. That path arrives in a new release of the mailwoman package.

4. There is no refresh cadence

No cadence is committed. The gazetteergazetteerA geographical index that maps place names and postcodes to real-world coordinates. Mailwoman uses a custom-built Who's On First (WOF) SQLite database as its gazetteer — the 'atlas' half of the grammar/atlas architecture. and layerlayerOne transformer block — attention plus a feed-forward network, with normalization and residual connections — applied to every position. Stacking layers lets the model build up richer representations; Mailwoman's encoder has 6. artifacts are rebuilt when coveragecoverageThe fraction of a population or region for which a data source has real, non-placeholder entries — e.g. 47% rooftop coverage on Texas addresses. Distinct from accuracy on the rows that are present. changes or an ingest bug is fixed rather than on a schedule, and no schedule is promised. This is the same statement the pricing page makes, and it is the one to plan against: do not build a pipelinestaged pipelineMailwoman's runtime architecture: a sequence of pure-function stages (normalize → query-shape → locale-hint → kind-classifier → phrase-grouper → classifier → decoder) connected by typed handoffs. Each stage is published as its own npm package. that assumes a weekly or monthly artifact.

What you get instead is that every artifact is sealed and dated. The vintage travels in the remote path, so mailwoman data pull candidate --dry-run always tells you which build your install is pinned to without downloading anything.

5. Pin a version, and switch without a restart

The per-state databases support a second addressing scheme. Instead of one file per state, publish <family>/<family>-us-<slug>-<version>.db alongside the old one and put a releases.json at the data root naming the current version per family:

{ "address-points": "2026-05-20.0", "interpolation": "TIGER2023" }

USStateDatabaseProvider.create reads that manifest before it returns the provider:

import { AddressPointSqliteLookup, StreetInterpolator } from "@mailwoman/resolver-wof-sqlite"
import { USStateDatabaseProvider } from "mailwoman/geocode"

const extracts = await USStateDatabaseProvider.create(
	{ AddressPointSqliteLookup, StreetInterpolator },
	"/tmp/mw-releases-demo"
)

console.log("versions:", JSON.stringify(extracts.versions()))
console.log("dc address points open:", Boolean(extracts.for("dc").addressPoints))

With no manifest, it runs in legacy mode against the unversioned filenames:

versions: null
dc address points open: true

With a manifest naming a version whose file exists, it opens that file:

versions: {"address-points":"2026-05-20.0"}
dc address points open: true

A pinned version whose file is absent falls back to the unversioned name and gives no warning. Above, a run against a data root holding both address-points-us-dc.db and address-points-us-dc-2026-05-20.0.db, pinned to a third version that was never published:

versions: {"address-points":"2026-06-01.0"}
dc address points open: true

The database opened. It is the legacy file rather than the pinned one. Remove the legacy file and the same pin resolves to no file at all:

versions: {"address-points":"2026-06-01.0"}
dc address points open: false

So a typo in releases.json degrades to the old data rather than to an error. Assert on versions() after deploy rather than assuming the pin took.

The reason to run the manifest at all is reload(). Write the new database files alongside the old, then write releases.json — one atomic file write — then await extracts.reload(). Any database whose resolved path changed is reopened. Any database whose path is unchanged keeps its open handle. Retired handles close on the next reload, so in-flight reads have one reload generation to finish.

// A long-running server, on a signal or a poll.
process.on("SIGHUP", async () => {
	console.log("switched to", JSON.stringify(await extracts.reload()))
})

Verify

A deploy check that fails when any artifact you depend on has drifted:

mailwoman data status candidate --check-remote && echo "data OK"

✓ candidate: wof/candidate.db: present (1652.9 MB) at /tmp/mw-fresh-demo/wof/candidate.db
PASS (1/1 checks)
data OK

Limits

• data status compares against the published artifact rather than against validity. A database you built yourself from the US dataset tutorial fails the check by construction, because its byte count is not the published one. That page explains the case.
• The four bundlesevidence bundleThe pair of retrieval-augmented input channels (street-type + locality-surface) that feed lexicon membership as soft per-token evidence alongside the text. Shipped in 6.7.0; trained natively from step 0 in the from-scratch base line. are the whole published set. There is no timezone, NUTS or UN/LOCODE artifact in the bucket. Those packages build their data locally. mailwoman data pull with any other name exits 1 and lists the four.
• releases.json addresses the per-state families only. The versioned scheme covers address-points and interpolation. candidate.db, poi.db and the French rooftoprooftopGeocoding precision at the building or parcel level — coordinates within a few meters — the highest tier of the geocode cascade. Sourced from address-point and situs data. registerinput modeThe Decision-A register switch: 'fragmented' (human-typed fragments — feeds the evidence channels) vs 'formatted' (complete records — runs the trained absence identity). Explicit on CLI/API; per-endpoint defaults (batch→formatted, autocomplete→fragmented); kind-derived otherwise. are addressed by their dated remote path instead, so refreshing one of those means upgrading mailwoman to a release that registersinput modeThe Decision-A register switch: 'fragmented' (human-typed fragments — feeds the evidence channels) vs 'formatted' (complete records — runs the trained absence identity). Explicit on CLI/API; per-endpoint defaults (batch→formatted, autocomplete→fragmented); kind-derived otherwise. a newer path, then pulling again.
• --check-remote needs network egress to public.mailwoman.ai. In an air-gapped deploy the offline comparison is what you have, and it still catches truncationtruncationReducing an input down to the max sequence length (or an LLM response to its token limit), discarding everything past the cap..

Related

• Improve geocode precision — which bundleevidence bundleThe pair of retrieval-augmented input channels (street-type + locality-surface) that feed lexicon membership as soft per-token evidence alongside the text. Shipped in 6.7.0; trained natively from step 0 in the from-scratch base line. provides which resolution tier.
• Build the US address dataset from public sources — what to do instead when you build the databases yourself rather than pull them.
• Deploy on a serverless runtime — how the same artifacts behave on a read-only, short-lived filesystem.