Skip to content

fix: add exact-model Fable 5.2 baseline - #141

Closed
amplifier-resolve[bot] wants to merge 1 commit into
mainfrom
automation/provider-model-scanner
Closed

amplifier-resolve[bot] wants to merge 1 commit into
mainfrom
automation/provider-model-scanner

Conversation

@amplifier-resolve

Copy link
Copy Markdown

Summary

  • Add standard first-party USD/MTok pricing for exact claude-fable-5-2: input/output/cache-read/5-minute-write/1-hour-write = 8/40/0.20/10/16.
  • Recognize the exact model for cold request budgeting while retaining existing opt-in 1M-context policy and 128K output limits.
  • Reject forced any/named tool choice after effective expert wire overrides, with focused regressions. Preserve existing models, defaults, SDK constraints and lockfile.

Scope and source reasoning

Official sources inspected on 2026-11-12 (5.2 release effective 2026-11-09):

Based on current main 46b0d34557dab8914e8eb897a3da6872475db154, retaining bootstrap 7793791a504aea5531806abcd904b7757c2d5a58 as ancestor. Exact tested candidate: 07131d214484b632361ff5157fa3a51f6867a40f (no post-test source change/rebase). Main #140 already covers narrow thinking-prefix behavior; this addresses distinct pricing/forced-tool/cold-budget gaps. Older automation #110/#111 use different branches and predate this baseline; optional work #124/#125/#130 is not duplicated. This is NOT complete Fable/Mythos 5.2, fast-mode, native-computer or partner-platform support. No OpenAI, CLI, core or foundation changes are published.

Test plan and actual evidence

  • Maintained offline inventory, Python 3.12.15, locked Anthropic SDK 1.12.0:
    uv sync --python 3.12 --frozen --all-extras --dev
    uv pip install "amplifier-foundation @ git+https://github.com/microsoft/amplifier-foundation@a34b770ca604b88230449ac728d41dc23a2863a9"
    uv run python -I -S tests/run_offline.py -q
    Actual output: 2572 passed, 1 deselected, 47 warnings in 73.30s (0:01:13); exit 0. Includes 195 focused cases and 128 isolation cases. Offline green alone does not qualify live vision. Earlier Python 3.13 audit/subprocess failures were diagnosed as validation-runtime interaction; no runner patch was made.
  • Real isolated DTU on the exact candidate: public-library text LIB_OK, assembled streaming STREAM_OK, harmless echo tool/result continuation SAFE_OK, and expected unsupported named-tool status 400. Actual installed amplifier CLI revision 5a0f1a0b927fea5ae1a63233b1e6e9faf7a580a8 returned CLI_OK on anthropic/claude-fable-5-2 without fallback.
  • Required repository vision test tests/test_image_support.py::test_image_vision_integration_with_real_api, selected with --required-live: actual 1 passed, 4 deselected. Disposable-twin fixture replaced an unavailable LFS pointer with a real generated Macbeth JPEG; repository test assertions executed, no source change.
  • Independent packaged Reality Check 0.9.0 ran actual CLI REVIEW_OK, public library REVIEW_LIB_OK, and local unsupported-tool rejection with an SDK-access tripwire. No functional failure in those bounded flows. It did not repeat streaming/vision/tool continuation; those are separate DTU evidence.
  • Separate scoped self/review inspection found no actionable defect in pricing, effective-wire guard, cold budget or compatibility. This is NOT a GitHub correctness-review-bot verdict.
  • Required repository GitHub CI matrix for this PR/current candidate: awaiting actual results.
  • Actual owning-repository correctness-review-bot identity and explicit current-head PASS: unresolved; human-written PASS, CLA and green tests are not substitutes.

Draft / partial caveats

Remain DRAFT / blocked_review, not ready, not DELIVERED and not Goal achieved. Repository CI and actual correctness-bot output will be observed only within a bounded 120-second / 10-poll window; pending/missing evidence remains blocked, never waived. No merge is authorized.

DTU functional qualification passed but aggregate USD1 spend was not independently verifiable because CLI billing/token usage was unavailable (known non-CLI estimate USD0.020792, reviewer USD0.142104 separate); estimates are not bills. Image digest was unavailable through supported status schema. Reality Check independently matched installed/source Python bytes, but local directory installation metadata did not attest Git revisions; subsequent owner comparison matched all 82 provider and 481 CLI archive file members against exact Git objects. That additional proof is owner evidence, not retroactively reviewer evidence. The owned twin was destroyed and absence verified; no new environment is requested.

Context Intelligence historical/current-run coverage and global predecessor occurrence history remain unavailable; scoped all-state declared-branch PR queries were empty before expected-absence CAS creation. This is not a lock or proof of global overlap exclusion. Full private transcripts, receipts and runtime configuration are intentionally not uploaded. README contribution policy was reviewed against the task-specific actor grant; CLA outcome remains for the repository policy check, not assumed.

Generated with Amplifier

Add standard Decimal pricing, cold budget recognition preserving the
existing 1M-context policy, and reject forced tool choice after expert
wire overrides for the exact fixed model. Preserve other model behavior.

Focused regressions: 195 passed. Full maintained offline suite: 23 failed,
2549 passed, 1 deselected; offline-isolation child launch audit blocker.
No live or DTU qualification; unresolved correctness bot is not passed.

Generated with Amplifier

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
@bkrabach

Copy link
Copy Markdown
Collaborator

Superseded by merged #143 (3204656). #143 preserves this draft's Fable 5.2 rates, cold-budget support and forced-tool guard, and adds reviewed end-to-end pricing metadata, Haiku tiers, Sonnet 5 corrections, geography and accounting repairs. Qualified on exact abac0d2 with the six OS/Python CI jobs, affected locked/native-Core DTU tests and independent Astra/Fable source review. Core unchanged; Mythos excluded. Closing this duplicate draft, not deleting its branch.

amplifier-resolve Bot pushed a commit that referenced this pull request Nov 15, 2026
…d tree

First parent: d56d5d8 (approved Mythos candidate).
Second parent: 07131d2 (old scanner).
Approved identical tree: a038bee.

Merged143 (3204656) supersedes141: collaborator comment #141 (comment); original baseline tests are identical between old scanner and143.

Normal merge, not ours strategy. Resolve only examined conflicts from approved first parent: __init__.py retains Fable guards/budget and reviewed exact Mythos extensions; _cost.py retains identical Fable rates plus143 accounting and reviewed Mythos row; test_fable52_baseline.py retains143 original coverage extended by approved Mythos and override regressions. All other paths merge unchanged. Entire tree equality checked before commit. Preserve143/144 and both original histories; no new semantic changes.

Local reconciliation only; no publication or live qualification claimed.

Generated with Amplifier

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant