RFC: Make fact-extraction prompts easier to edit and test

[gaperton]

Problem

Fact-extraction prompts are assembled at runtime from Python constants, formatting calls, conditionals, and appended sections.

A prompt engineer cannot open one file and see the complete prompt sent to the model. Reviewing or changing it requires tracing Python code, and shared instructions can drift between extraction modes. Issues such as #745 and #181 show this happening in practice.

Proposal

Move the prompts to Jinja templates with:

• one readable template per extraction mode
• shared fragments for repeated instructions
• strict validation of template variables
• golden tests for complete rendered prompts
• a simple way to render prompts without calling an LLM

I propose doing this incrementally:

1. Split fact_extraction.py into focused modules without behavioral changes.
2. Add prompt-rendering compatibility tests.
3. Migrate prompts to Jinja while preserving their rendered output.
4. Deduplicate shared prompt sections in a separate change.

I volunteer to implement these changes, add the tests, and address review feedback. I already have a tested prototype of the first module-splitting step, but will wait for agreement on the direction before opening a PR.

Would maintainers be interested in this direction? If so, would you prefer the module split as one mechanical PR or several smaller PRs?

[nicoloboschi]

shipped in 0.10.0!