FST read-path and search optimizations

[capemox]

Overview

A set of allocation-conscious performance optimizations to vellum's FST read and search paths. The headline change (this branch's newest commit) speeds up lookups and automaton-guided search with no on-disk format change — existing FSTs benefit without a rebuild. Two earlier commits on the branch trim allocations and table sizes in the regexp/levenshtein automatons.

All changes are covered by the existing test suite plus new benchmarks, and pass under -race.

Contributions

1. Cache root state in Reader and skip redundant key lookups in the iterator (4f29369)

Two format-compatible, allocation-free read speedups:

• Reader root-state cache — the root state is traversed on every Get, so it is now parsed once at Reader construction and the immutable parsed form is reused, letting each lookup skip re-decoding the root.
• Iterator offset accessor — the automaton-guided iterator already knows a transition's sorted offset from TransitionAt, but then re-searched for the byte via TransitionFor (a bytes.IndexByte scan) to recover the destination/output. A new TransitionDestForOffset resolves destination/output directly by offset, and only after the automaton accepts the transition, so selective searches don't decode transitions they discard.

Benchmarks (Apple M4 Pro, benchstat, n=6, all p=0.002; allocations unchanged at 0 for Get):

Benchmark	Before	After	Δ
GetWords	99.2µs	88.1µs	−11.1%
GetWide (8-byte ids)	5.91ms	5.49ms	−7.1%
GetWide2 (2-byte, high-fanout)	2.50ms	1.74ms	−30.4%
ScanWords	87.3µs	81.9µs	−6.2%
ScanWide	7.15ms	6.71ms	−6.1%
FuzzyWords1	2.97µs	2.73µs	−8.0%
FuzzyWords2	10.6µs	9.6µs	−9.3%
RegexWords	80.6µs	74.0µs	−8.2%
geomean			−11.2%

2. Shrink regexp DFA tables and pool FST automaton states (c953a0a)

Reduces the regexp DFA table footprint and pools the transient fstState instances used by the Automaton/Transducer interface methods, avoiding a per-call allocation.

3. Fix swallowed build error and reduce levenshtein allocations (0469d8d)

Surfaces a previously swallowed build error and trims allocations in the levenshtein automaton construction path.

Notes

• No FST on-disk format change; the encoding version is unchanged and existing FSTs are read faster as-is.
• A more invasive, format-changing optimization (a fat-node transition index, format v2) was evaluated and deliberately not included here — it helps only high-fanout exact lookups (~+27%), does little for search, and costs file size plus a format migration. It is preserved on a separate WIP branch for later consideration.