Skip to content

Add AI translation scripts - #6273

Open
j0ntz wants to merge 4 commits into
developfrom
jon/ai-translations
Open

j0ntz wants to merge 4 commits into
developfrom
jon/ai-translations

Conversation

@j0ntz

@j0ntz j0ntz commented Oct 17, 2026

Copy link
Copy Markdown
Contributor

Technical Design Document

ai-translations.md

Description

Asana task

Three scripts replace Crowdin's human translation step with model translations. The app is unchanged: src/locales/strings.ts still loads the human files in src/locales/strings/, and switching it to the AI files waits on the human review of the compare reports.

  • localize:context writes src/locales/context/enUS.context.json: for each of the 2,119 enUS keys, what it means, where it shows, what each placeholder stands for, a length hint and its usage files. Claude writes the entries and Jev screens for keys reused with different meanings.
  • localize:translate fills src/locales/strings-ai/<locale>.json for all 12 locales, 40 keys per Claude batch, with the batch sent as a cached prompt prefix. A value that loses a placeholder or line break is retried once. Jev scores meaning and language for every value and reports, without rejecting.
  • localize:compare puts each AI value next to the human one and opus judges every differing pair blind. A Jev pre-filter that skips the judge for pairs scored 0.9 or higher is opt-in (--prefilter): on the full run the judge called only 336 of those 732 pairs equivalent, and 20 of them existing better.
  • npm run localize:ai chains enUS.json, context and translate for new or changed strings. The project skill .claude/skills/localize-strings/SKILL.md, linked from AGENTS.md, tells agents to run it after any en_US.ts change and commit the generated files with the change.

All Claude calls go through claude -p on the subscription. Jev token counts are reported apart from Claude's.

Context store:

  • 2,119 of 2,119 keys have an entry; a rerun makes 0 calls.
  • Unused keys (no usage site in src): 299.
  • Reuse warnings: fragment_request_subtitle (receive action vs. received status) and fio_reject_request_title (reject confirmation title vs. send-request action label).
  • lengthHint: label 1,338, paragraph 595, button 186.

Translation:

  • All 12 files have 2,119 keys in enUS order, 0 empty values, 0 placeholder or line-break failures.
  • Jev gate flags per locale range from 14 (ko) to 320 (pt). A read of all es (50) and it (186) flags found 1 real error, currency_label_SYP in it ("Sterlina siriana" for "Lira siriana"), left as is and reported.

Compare (judge opus):

Locale Different Existing better AI better Equivalent Both wrong Weak human (Jev) Placeholder problems
it 1,185 60 777 348 0 210 4
esMX 931 40 592 299 0 102 20
es 568 21 317 230 0 68 2

Every compare placeholder problem is in the existing human file. Of the weak human values, the judge called 315 AI better, 29 equivalent and 11 existing better. Judging every pair instead of pre-filtering at 0.9 cost 18 more Claude calls and 67,911 more output tokens.

Tokens. Claude runs on the subscription through claude -p. Jev is a separate service, and its counts are kept apart.

Step Claude model Claude calls Claude input Claude output Claude cache write Claude cache read
Context sonnet 54 106 230,298 856,096 27,719
Translate sonnet 639 1,278 841,729 702,844 3,335,431
Compare opus 69 138 260,820 434,633 77,148
Step Jev calls Jev input Jev output
Context 3,190 1,784,936 102,518
Translate 25,428 10,712,276 2,820,027
Compare 4,526 1,616,844 156,936

Claude input counts only uncached prompt tokens; the rest of each prompt is in the cache columns. The context input of 106 is the 53-call full run. The one reuse-review call did not log its uncached input.

The full reports are on the Asana compare subtask.

CHANGELOG

Does this branch warrant an entry to the CHANGELOG?

  • Yes
  • No

Dependencies

none

Requirements

If you have made any visual changes to the GUI. Make sure you have:

  • Tested on iOS device
  • Tested on Android device
  • Tested on small-screen device (iPod Touch)
  • Tested on large-screen device (tablet)

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-10-17T02:38:31.203799Z a6c4ce8 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@cursor

cursor Bot commented Oct 17, 2026

Copy link
Copy Markdown

This PR is too large for Bugbot to review. It changes 56,880 lines and 4,130,318 characters. Split the change into smaller pull requests to get a review.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

j0ntz added 4 commits October 16, 2026 19:41
localize:context documents every enUS string for translators: what it
means, where it shows, what each placeholder stands for and a length
hint, from its usage sites in src. Claude (claude -p on the subscription,
or the Anthropic API when ANTHROPIC_API_KEY is set) writes the entries.
Jev screens every key for reuse with different meanings and Claude
confirms the flagged ones. Entries carry a hash of the English value and
usage files, so a rerun only processes new or changed keys.
localize:translate fills src/locales/strings-ai/<locale>.json from enUS
and the context store, one Claude batch of 40 keys at a time. The batch
is sent as a cached system prompt so each locale after the first reads
it from cache. Every value must keep its placeholders and line breaks or
it is retried once with the problem, then left empty. Jev scores
meaning and language for each value; the gate reports by default, since
every rejection in the pilot was a good translation. A state file keyed
by locale and source hash skips unchanged keys on the next run.
localize:compare --locale it puts each AI value next to the existing
human one. Pairs that match after normalizing case, quotes and trailing
punctuation are bucketed without a judge. Jev pre-filters pairs it scores
0.9 or higher as equivalent and flags human translations whose meaning
scores under 0.5. Opus judges the rest blind (A/B order set by a key
hash) as existing better, AI better, equivalent or both wrong, with a
one-line reason. The report also lists placeholder mismatches in either
file and the tokens spent, Jev apart from Claude.
Orchestrated runs need one way to fill the AI context and translations
after an en_US.ts change. Add npm run localize:ai, which chains the
language, context and translate scripts, and a project skill that tells
agents to run it and commit the generated files with the change.
AGENTS.md points to the skill.
@j0ntz
j0ntz force-pushed the jon/ai-translations branch from a6c4ce8 to 49ed5c4 Compare October 17, 2026 02:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant