⚡ Copilot Token Optimization2026-07-17 — test-coverage-improver

[github-actions]

Target Workflow: test-coverage-improver

Source report: Most recent token-usage-report issue (label: token-usage-report)
Estimated cost per run: ~$2.17 (measured via AIC pricing at claude-sonnet-4.6 rates)
Total tokens per run: ~30K (+ 4.7M ambient cache-read tokens)
Cache hit rate: N/A (1 LLM request; ambient context provides 4.7M cached tokens)
LLM turns: 1
Action minutes: 15

Current Configuration

Setting	Value
Model configured in .md	claude-haiku-4-5
Model actually used	claude-sonnet-4.6 (overridden by org variable)
Tools loaded	github (repos toolset), bash (5 patterns)
Tools actually used	bash (jest rerun), create-pull-request
Network groups	github only
Pre-agent steps	Yes (6 steps incl. npm ci, npm run build, npm run test:coverage)
Post-agent steps	None
Prompt body size	~7,410 chars
Cache write tokens	91,663/run
Output tokens	26,563/run (test file written in single turn)

Root Cause Analysis

The dominant cost driver is a model mismatch: the workflow sets COPILOT_MODEL: claude-haiku-4-5 in its frontmatter, but the org variable GH_AW_MODEL_AGENT_COPILOT (or GH_AW_DEFAULT_MODEL_COPILOT) overrides it to claude-sonnet-4.6 at runtime. This is visible in the compiled lock file:

GH_AW_INFO_MODEL: ${{ vars.GH_AW_MODEL_AGENT_COPILOT || vars.GH_AW_DEFAULT_MODEL_COPILOT || 'claude-sonnet-4.6' }}
# and in the agent step:
COPILOT_MODEL: ${{ vars.GH_AW_MODEL_AGENT_COPILOT || vars.GH_AW_DEFAULT_MODEL_COPILOT || 'claude-sonnet-4.6' }}

The secondary cost driver is 15 action minutes per run from npm ci + npm run build + npm run test:coverage running without caching.

Recommendations

1. Fix Model Override — Enforce claude-haiku-4-5

Estimated savings: ~$1.45/run (~67%)

This workflow writes tests and verifies them in 1 LLM turn — well within Haiku's capability. The model mismatch results in paying Sonnet rates for a Haiku task.

Cost comparison using actual token volumes from this run:

Model	Input	Output	Cache Read	Cache Write	Total
claude-sonnet-4.6 (actual)	$0.010	$0.398	$1.417	$0.344	$2.17
claude-haiku-4-5 (intended)	$0.003	$0.133	$0.472	$0.115	$0.72

Actions:

• Check the org variable GH_AW_MODEL_AGENT_COPILOT — if it is set to a Sonnet model, it is overriding all workflow-level COPILOT_MODEL settings.
• If the org variable must stay at Sonnet for other workflows, open a gh-aw issue requesting per-workflow model priority that takes precedence over the org variable.
• Alternatively, if the task quality is acceptable on Haiku, change the org default.

2. Cache node_modules and Build Output in Pre-Steps

Estimated savings: ~5–8 action minutes/run (~33–53% of runner time)

The pre-steps run npm ci, npm run build, and npm run test:coverage unconditionally every run. Add caching for node_modules and the TypeScript build:

steps:
  - name: Restore node_modules cache
    id: node-cache
    uses: actions/cache/restore@v6
    with:
      path: node_modules
      key: node-modules-${{ hashFiles('package-lock.json') }}

  - name: Install dependencies
    if: steps.node-cache.outputs.cache-hit != 'true'
    run: npm ci

  - name: Save node_modules cache
    if: steps.node-cache.outputs.cache-hit != 'true'
    uses: actions/cache/save@v6
    with:
      path: node_modules
      key: node-modules-${{ hashFiles('package-lock.json') }}

  - name: Restore build cache
    id: build-cache
    uses: actions/cache/restore@v6
    with:
      path: dist
      key: build-${{ hashFiles('src/**/*.ts', 'tsconfig.json') }}

  - name: Build
    if: steps.build-cache.outputs.cache-hit != 'true'
    run: npm run build

  - name: Save build cache
    if: steps.build-cache.outputs.cache-hit != 'true'
    uses: actions/cache/save@v6
    with:
      path: dist
      key: build-${{ hashFiles('src/**/*.ts', 'tsconfig.json') }}

  - name: Run coverage
    run: npm run test:coverage 2>&1 | tail -10
    id: coverage

npm run test:coverage cannot be cached (it reads current source), but skipping npm ci and npm run build on cache hits saves ~3–5 minutes.

3. Remove Unused eslint Tool from tools:

Estimated savings: ~600 tokens/turn (~0.7% of cache-write tokens)

The eslint tool is loaded but the prompt's Turn Budget (≤6 calls) prioritizes write-test → verify → PR. In a 1-turn workflow, eslint is nearly never exercised. Remove it from tools: and add an unconditional post-steps: lint instead:

In .github/workflows/test-coverage-improver.md:

tools:
  github:
    toolsets: [repos]
  bash:
    - "node:*"
    - "./node_modules/.bin/jest:*"
    # removed: ./node_modules/.bin/eslint:*
    - "cat:jest.config.js"
    - "cat:jest.config.ts"

post-steps:
  - name: Lint generated test file
    run: |
      TEST_FILE="$(cat /tmp/generated-test-path.txt 2>/dev/null || echo '')"
      if [ -n "$TEST_FILE" ] && [ -f "$TEST_FILE" ]; then
        ./node_modules/.bin/eslint "$TEST_FILE" --max-warnings=0
      fi

4. Cap Injected Source Content Size

Estimated savings: ~10–20K cache-write tokens/run on large target files (~10–20%)

The Select target file and inject content step injects full source and test files into the prompt via GITHUB_OUTPUT. Larger files like src/cli-options.ts (18KB) would balloon the cache-write token count significantly. Add a size cap:

- name: Select target file and inject content
  id: target
  run: |
    # ... existing target selection logic ...
    {
      echo "SOURCE_CONTENT<<EOF"
      cat "$TARGET" 2>/dev/null | head -c 8000 || echo "(not found)"
      echo "EOF"
    } >> "$GITHUB_OUTPUT"
    {
      echo "TEST_CONTENT<<EOF"
      cat "$TEST_FILE" 2>/dev/null | head -c 4000 || echo "(test file does not exist yet)"
      echo "EOF"
    } >> "$GITHUB_OUTPUT"

This limits injected context to ≤12KB, sufficient for the agent to understand the structure.

5. Add post-steps: Test Validation

Estimated savings: Prevents broken PRs (reliability, not direct token savings)

Currently there are no post-steps:. If the agent writes syntactically broken tests, this is only discovered when CI runs against the PR. Add a targeted Jest check:

post-steps:
  - name: Validate generated tests pass
    run: |
      TEST_FILE="${{ steps.target.outputs.TARGET_TEST_FILE }}"
      if [ -f "$TEST_FILE" ]; then
        ./node_modules/.bin/jest --testPathPattern="$TEST_FILE" --no-coverage 2>&1 | tail -20
      fi

Expected Impact

Metric	Current	Projected	Savings
Cost/run	~$2.17	~$0.75–0.80	~63–65%
AIC/run	215.9	~80–85	~61–63%
Action minutes	15	~8–10	~33–47%
Cache-write tokens	91,663	~70,000	~24%
Output tokens	26,563	~26,563	0% (model-independent)

Primary savings from model fix (#1). Secondary savings from npm/build caching (#2). Items #3–5 are lower-impact but improve reliability and reduce unnecessary tool loading.

Implementation Checklist

• Check org variable GH_AW_MODEL_AGENT_COPILOT value — determine why claude-haiku-4-5 is not taking effect
• Fix model: either update org variable, or open gh-aw issue for per-workflow model priority
• Add actions/cache/restore + actions/cache/save steps for node_modules in .github/workflows/test-coverage-improver.md
• Add actions/cache/restore + actions/cache/save steps for dist build output
• Remove ./node_modules/.bin/eslint:* from tools: section
• Add post-steps: for lint and test validation
• Add head -c 8000 guard on SOURCE_CONTENT injection
• Recompile: gh aw compile .github/workflows/test-coverage-improver.md
• Verify CI passes on PR
• Compare AIC on next scheduled run (target: <85 AIC vs baseline 215.9)

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by Daily Copilot Token Optimization Advisor · 94.4 AIC · ⊞ 6.4K · ◷