Finish the CRLF fix, and make the guard one that cannot drift (#1041)

[realmarcin]

Reopens and completes #1041. #1042 was incomplete and I called it done.

What was still shipping CRLF

11 files, five sources — found only by enumerating every TSV instead of looping over guessed names:

file	why the first pass missed it
prego_habitat/nodes.tsv, edges.tsv, unmapped_associations.tsv	the survey looked in data/transformed/prego/ — PREGO writes to prego_habitat (PREGO_SHAPES=habitat changes the output dir, #885's exact confusion), so the source was skipped entirely
rhea_mappings/edges.tsv	the loop tested nodes.tsv and stopped; that one happens to be LF
metatraits/, metatraits_gtdb/ × 3 reports, microbedecoder/unmapped_labels.tsv	never in the hand-written module list

Two of those are graph files that feed the merge.

Fix

Every csv.writer/DictWriter under transform_utils/ and merge_utils/ now goes through tsv_io — 37 call sites across 14 modules. (scripts/ is out of scope and untouched.)

The guard no longer carries a list

That was the real defect. GRAPH_WRITERS was seven hand-written paths; a list someone must remember to extend is the failure mode #1035 was about, and it drifted within a day. It now walks both trees — 68 modules — with a companion test asserting the walk found >30 and specifically reaches prego, rhea_mappings, metatraits, merge_kg, gtdb, so a glob that silently matched nothing can't make every check vacuous.

The acceptance test enumerates too

for tsv in sorted(TRANSFORMED.rglob("*.tsv")):   # not a source-name loop

It fails today, naming all 11 offenders — correct behaviour: the code is fixed, the output needs regenerating. It skips where data/transformed is absent, so CI is unaffected.

Verification

• 68/68 modules pass the AST guard; pytest -k "prego or rhea or metatraits or microbedecoder or mediadive or bactotraits or madin or stubs or gtdb or merge" → 482 passed, 4 skipped.
• Not blocking the merged artifact: proven empirically that _normalize_nodes_tsv turns a CRLF input into pure LF with no stray \r in any field, so merged-kg.tar.gz is LF regardless of what transforms hand it.

Reruns

prego, rhea_mappings, metatraits, metatraits_gtdb, microbedecoder, bactotraits, madin_etal, mediadive — ~64 min, dominated by metatraits at 42. No shared code moved, so nothing else goes stale and no dependents cascade.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TPpUDtT57QRMteXw7dLvRo

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[realmarcin]

Adversarial review

Read-only pass over all 37 converted call sites plus four probes.

• No call lost a real kwarg. Grepping the removed lines for anything other than delimiter="\t" / lineterminator="\n" / a bare open-paren returns nothing — no quoting, quotechar or escapechar was dropped by the regex. The one multi-line DictWriter (microbedecoder.py:794) kept its fieldnames=[...] list intact; all five tsv_dict_writer calls still pass fieldnames.
• ontologies_stubs was already correct (it passed lineterminator="\n" explicitly) and the conversion is a no-op there — verified the helper still emits 'a\tb\n'.
• The guard is genuinely exhaustive now, not just longer: 68 modules walked, and test_the_scan_actually_covers_the_transforms fails if the walk returns fewer than 30 or misses any of prego, rhea_mappings, metatraits, merge_kg, gtdb. That second test is the one that would have caught Write LF, not CRLF: csv.writer's default terminator reached the shipped graph (#1041) #1042's list drifting.
• Found while reviewing, not fixed here: 22 of the converted handles are opened "w" without newline="". On POSIX that is harmless — the text layer passes \n through unchanged, which is why every canary shows LF — but on Windows Python would translate it back to \r\n, undoing the fix. The repo is POSIX-only in practice (macOS dev, Linux CI, and #871 pins the platform matrix), so this is latent rather than live. Filing it separately rather than widening this diff by 22 more edits.
• Honest limit: verified by unit test and AST scan, not by rerunning the eight affected transforms — that is the 64-minute batch, deliberately deferred to the next quiet window (the machine is at load 115). The rglob acceptance test is what will confirm it; it fails today naming exactly the 11 known files, which is the state I want recorded rather than a green tick.