Skip to content

fix(gestures): fail pre-removal gesture forms at .ad parse time (#1216)#1393

Open
thymikee wants to merge 3 commits into
mainfrom
claude/agent-device-1216-a79985
Open

fix(gestures): fail pre-removal gesture forms at .ad parse time (#1216)#1393
thymikee wants to merge 3 commits into
mainfrom
claude/agent-device-1216-a79985

Conversation

@thymikee

Copy link
Copy Markdown
Member

Closes #1216.

Why

#1315 removed the timed forms of swipe, gesture fling, and gesture swipe, plus gesture rotate's velocity — but shipped without the migration guide, the repository sweep, or the parse-time error that #1216's own "before removal" checklist gates a removal on.

The sweep this PR performs finds what that left behind:

  • both 06-swipe-gestures.ad integration fixtures still carried the 5-argument swipe and would fail at replay today;
  • export-flow.test.ts and script.test.ts still asserted the removed shapes;
  • readSwipeGeometry (Maestro export) and a swipe-duration fallback in recording-gestures.ts still read the retired positional from a slot that can no longer be populated.

What changed

One arity table. Positional arity for every public gesture syntax now lives in PUBLIC_GESTURE_SYNTAX in src/contracts/gesture-normalization.ts, keyed off the canonical GESTURE_KINDS via a template-literal type — a new gesture kind cannot skip it, and a form removed from the CLI is removed from .ad in the same edit. Both callers read it: the CLI argv parse and a new .ad preflight.

Stale scripts fail at parse, not at step N. A .ad script carrying a removed positional now fails when the script is parsed — before the replay executes any device action — naming the line and computing the rewrite:

Error (INVALID_ARGS): swipe accepts 4 arguments: x1 y1 x2 y2 (line 6). The trailing durationMs
positional was removed: use "gesture pan 197 650 0 -350 300" for the same timed drag, or
"swipe 197 650 197 300" for a default-duration swipe.

Previously the script ran up to that step and then failed as a repairable REPLAY_DIVERGENCE, which is wrong: it is a script syntax error, not a UI divergence. #1216 allows "an upgrader or an actionable parse error"; this is the latter, and the message already contains the exact replacement line.

The preflight checks arity only${VAR} tokens resolve after planning and interpolation never splits a token, so the count is decidable at parse time while the values are not. A single extra numeric argument is reported as the migration; anything else stays a plain usage error, so an unknown flag is not misreported as a retired duration.

Migration guide published. website/docs/docs/migrating-gestures.md covers CLI, Node.js, MCP, and saved .ad recordings (plus why Maestro flows need no migration), and documents the five-step deprecation policy the next such removal has to clear — announce, warn for one minor, publish the migration, prove the repository is clean, then remove the branch and its tests together.

Reviewer notes

One judgment call. Deleting the dead duration read in readSwipeGeometry would have left replay export emitting no duration at all, handing Maestro's own 400ms default to a gesture the .ad script runs at 100ms. The export now states duration: 100 (the canonical fling duration) explicitly. Maestro flows replayed by agent-device are untouched — a timed Maestro swipe still normalizes to gesture pan with the endpoint-hold profile.

.ad positional parsing is deliberately NOT removed. #1216 listed it as the second shim, conditional on "recordings and migration tooling having a structured replacement". Its only remaining callers are the CLI argv parse and the .ad line parse — both the current public syntax, not a bridge to an older one — so there is nothing to migrate off, and a structured payload would make recordings unreadable and ungreppable for no behavioral gain. ADR 0013 already decided to keep it; this PR makes that explicit and drops the "compatibility" framing that made it read as debt. If you would rather keep that half of the issue open, the guide's Positional .ad syntax section is the piece to revert.

Fixture migration. Both 06-swipe-gestures.ad files drop the trailing duration and keep swipe (rather than converting to gesture pan), preserving each suite's coverage of the swipe command. That does change the motion — a 300ms drag becomes a 100ms fling, which travels further on a scrollable list — so it was verified rather than assumed.

Verification

  • Both migrated fixtures pass on real devices, driven by the repo's own CLI: iOS simulator 34.9s, Android emulator 45.9s (ad test <fixture> --platform ... --state-dir ...).
  • lint, typecheck, format:check, check:layering, check:production-exports, check:fallow, maestro:conformance, test:smoke — all clean.
  • Full unit suite passed end-to-end. Two later runs under heavy load (parallel worktree integration tests + three emulators, load avg ~15) showed a changing set of unrelated failures — screenshot-diff, request-router-open, android input-actions — all of which pass in isolation; this is the known contention flake, not a regression.

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Jul 24, 2026

Copy link
Copy Markdown
PR Preview Action v1.8.1

QR code for preview link

🚀 View preview at
https://callstack.github.io/agent-device/pr-preview/pr-1393/

Built to branch gh-pages at 2026-07-25 10:26 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

@github-actions

github-actions Bot commented Jul 24, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
JS raw 1.84 MB 1.84 MB +1.8 kB
JS gzip 587.5 kB 588.2 kB +702 B
npm tarball 701.9 kB 702.6 kB +626 B
npm unpacked 2.46 MB 2.46 MB +1.8 kB

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 29.8 ms 28.8 ms -1.1 ms
CLI --help 61.9 ms 62.5 ms +0.6 ms

Top changed chunks:

Chunk Raw diff Gzip diff
dist/src/runtime.js -5.1 kB -1.7 kB
dist/src/recording-export-quality.js +1.8 kB +503 B
dist/src/interaction.js -94 B -25 B
dist/src/cli.js -33 B -20 B
dist/src/registry.js +16 B +17 B

@thymikee

Copy link
Copy Markdown
Member Author

Reviewed exact head d88c6ed8. Not ready.

P1 — the Node swipe migration still silently drops durationMs. The guide promises a runtime INVALID_ARGS, but interactionDaemonWriters.swipe projects only from, to, count, pauseMs, and pattern. A JavaScript or old-compiled caller’s durationMs therefore never reaches the handler rejection and the default 100ms fling runs instead. Reject removed keys at the Node/MCP boundary with an SDK regression, or correct the guide and resolve the silent behavior explicitly.

P2 — variable-backed stale .ad durations miss the actionable migration. The preflight calls itself arity-only and variables resolve later, yet migration guidance requires the extra token to be numeric. swipe 197 650 197 300 ${DURATION} gets only usage text. Treat one extra interpolation token as the retired slot and test it.

P2 — correct the release version. The migration page says 0.21 removed these forms, but removal commit 6d99914f4 is contained in tag v0.20.0.

Literal stale forms do fail before dispatch; ADR alignment, explicit Maestro 100ms export, green CI, and reported iOS/Android fixture runs otherwise look sound.

@thymikee

Copy link
Copy Markdown
Member Author

All three addressed in 328bad8e3.

P1 — Node swipe silently dropped durationMs. Confirmed and fixed. gesture was already safe (its writer runs readGestureInputreadGesturePayload, which rejects the removed keys); swipe was the one public gesture surface with no structured reader of its own, so the writer's five-field projection dropped the key before the daemon's readSwipeInput could reject it. The rejection now lives in contracts/gesture-normalization.ts (assertNoRemovedSwipeInput) and is called from both the client writer and the daemon handler — one rule, one message, rather than a second copy.

The SDK regression in client.test.ts covers all four removed keys (swipe/fling/gesture swipe durationMs, rotate velocity) through client.interactions.* and asserts the transport is never reached. I checked it has teeth: removing the writer call fails it with AssertionError: swipe durationMs.

P2 — variable-backed durations missed the migration. Fixed. The numeric-only guard existed to stop a stray --unknown-flag being reported as a retired duration; an unresolved ${VAR} now also counts as the retired slot and is carried through into the rewrite:

swipe accepts 4 arguments: x1 y1 x2 y2 (line 6). The trailing durationMs positional was removed:
use "gesture pan 197 650 0 -350 ${DURATION}" for the same timed drag, or "swipe 197 650 197 300"
for a default-duration swipe.

A stray flag or word still gets plain usage text — both are covered by tests.

P2 — release version. You're right, and my source was wrong: git tag --contains 6d99914f4v0.20.0. I took "0.21" from the CHANGELOG's Unreleased heading, but git show v0.20.0:CHANGELOG.md shows the removal was already filed under Unreleased at tag time — headings lag several releases repo-wide. The guide now says 0.20.0. I did not restructure the CHANGELOG: the lag predates this PR and moving other PRs' entries mid-review invites conflicts. Flagging it in case you want it cut separately.

Also updated: the guide's Node section now states the rejection is client-side (so a JS caller gets the error, not a retimed gesture), and the .ad grep recipe matches variable-backed durations.

Re-verified: full unit suite, lint, typecheck, format:check, check:layering, check:production-exports, check:fallow, check:bundle-owner-files, maestro:conformance, test:smoke — all green. Residual unit failures across runs are the known contention flake (changing failure set on untouched files, all passing in isolation, load avg ~10 with a parallel worktree suite running).

🤖 Addressed by Claude Code

@thymikee

Copy link
Copy Markdown
Member Author

Re-reviewed exact head 328bad8e. The three prior code findings are resolved, but two migration-guide claims remain inaccurate.

P2 — the saved-script sweep does not find every retired form. The documented grep matches only five-argument swipe; it misses timed gesture fling, timed gesture swipe, and gesture rotate ... velocity. Provide patterns for all four forms so the repository-cleanliness step is usable.

P2 — MCP does not return the CLI’s concrete replacement command. The guide promises the same contextual rewrite, but structured readers return generic messages such as gesture fling does not accept durationMs; use gesture pan for timed movement. Correct the guide or implement the claimed rewrite.

Current checks are green and the reported iOS/Android fixture evidence is present.

thymikee added 3 commits July 25, 2026 12:22
#1315 removed the timed forms of `swipe`, `gesture fling`, and `gesture swipe`
and `gesture rotate`'s `velocity`, but shipped without the migration guide, the
repository sweep, or the parse-time error that issue #1216's own checklist
gates a removal on. The sweep finds what that left behind: both
`06-swipe-gestures.ad` integration fixtures still carried the 5-argument swipe
and would fail at replay, two tests still asserted the removed shapes, and two
branches still read the retired positional.

Argument arity for every public gesture syntax now lives in one table keyed off
the canonical `GESTURE_KINDS`, so a new kind cannot skip it and a form removed
from the CLI is removed from `.ad` in the same edit. Both callers read it: the
CLI argv parse, and a new `.ad` preflight. A stale script now fails when it is
parsed — before the replay executes any device action — naming the line and
computing its rewrite, instead of running up to that step and failing as a
repairable divergence. The preflight checks arity only: `${VAR}` tokens resolve
after planning, and interpolation never splits a token, so the count is
decidable while the values are not.

Deleting the dead duration read in `readSwipeGeometry` would have left
`replay export` emitting no duration, handing Maestro's 400ms default to a
gesture the script runs at 100ms, so the export now states `duration: 100`.

`.ad` positional gesture parsing is NOT removed. Its only remaining callers are
the CLI argv parse and the `.ad` line parse, both the current public syntax
rather than a bridge to an older one, so there is nothing to migrate off. ADR
0013 records that and drops the "compatibility" framing that made it read as
debt.

Both migrated fixtures verified on real devices with the repo's own CLI: iOS
simulator 34.9s, Android emulator 45.9s.
Review findings on d88c6ed.

P1: `interactionDaemonWriters.swipe` hand-projects five fields, so a JavaScript
caller's `durationMs` was dropped before the daemon's `readSwipeInput` could
reject it and a default-duration fling ran instead — the exact silent
reinterpretation the guide promises does not happen. `gesture` was already safe
because its writer runs `readGestureInput` -> `readGesturePayload`, which
rejects the removed keys; `swipe` was the one surface with no reader of its own.
The rejection now lives in contracts and is shared by the client writer and the
daemon handler, so there is one rule and one message. The SDK regression covers
all four removed keys and asserts the transport is never reached; reverting the
writer call fails it on `swipe durationMs`.

P2: the preflight's retired-slot test required a numeric token, so
`swipe 197 650 197 300 ${DURATION}` fell back to bare usage text. An unresolved
`${VAR}` now counts as the retired slot and is carried into the pan rewrite,
while a stray flag or word stays a plain usage error.

P2: the removal shipped in 0.20.0, not 0.21 — removal commit 6d99914 is
contained in tag v0.20.0. The guide said 0.21 because the CHANGELOG still files
it under `Unreleased`; the tag is the truth (headings lag several releases
repo-wide, so that is pre-existing and left alone). The `.ad` grep recipe now
matches variable-backed durations too.
Re-review findings on 328bad8 (both migration-guide accuracy).

The saved-script sweep matched only the five-argument `swipe` form; it missed
timed `gesture fling`, timed `gesture swipe`, and `gesture rotate ... velocity`,
so the repository-cleanliness step was incomplete. Provide one grep per retired
form (number-or-`${VAR}` token), each verified to flag the removed shape and
skip the valid one.

The MCP section claimed the structured rejection carries the CLI's concrete
replacement command. It does not: `readGesturePayload` and
`assertNoRemovedSwipeInput` return a message that names the removed key and the
replacement command (e.g. `gesture fling does not accept durationMs; use gesture
pan for timed movement`) but not the fully-substituted coordinate rewrite,
because the structured request carries no positional string to rewrite. Describe
what the structured path actually returns.
@thymikee
thymikee force-pushed the claude/agent-device-1216-a79985 branch from 328bad8 to 4fd5d39 Compare July 25, 2026 10:25
@thymikee

Copy link
Copy Markdown
Member Author

Both re-review findings addressed in 4fd5d39e, and the branch is rebased onto main (now at 14be01b78).

P2 — the saved-script sweep missed three of the four retired forms. Correct — it only matched five-argument swipe. Replaced with one grep per retired form (a token is a number or a ${VAR}), each verified against a fixture to flag the removed shape and skip the valid one:

# swipe x1 y1 x2 y2 durationMs
grep -rnE '\bswipe( +([-0-9.]+|\$\{[^}]*\})){5}' --include='*.ad' .
# gesture fling <direction> x y distance durationMs
grep -rnE '\bgesture +fling +[a-z]+( +([-0-9.]+|\$\{[^}]*\})){4}' --include='*.ad' .
# gesture swipe <preset> durationMs
grep -rnE '\bgesture +swipe +[a-z-]+ +([-0-9.]+|\$\{[^}]*\})' --include='*.ad' .
# gesture rotate degrees x y velocity
grep -rnE '\bgesture +rotate( +([-0-9.]+|\$\{[^}]*\})){4}' --include='*.ad' .

P2 — MCP does not return the CLI's concrete rewrite. Correct, and the honest fix is to describe what the structured path actually returns rather than implement a rewrite it can't meaningfully produce. readGesturePayload / assertNoRemovedSwipeInput name the removed key and the replacement command (e.g. gesture fling does not accept durationMs; use gesture pan for timed movement) but not the fully-substituted coordinate rewrite, because a structured request carries no positional string to substitute into. The guide now says exactly that. (Implementing the coordinate rewrite would only be possible for swipe, which has from/to; fling/rotate/gesture swipe structured inputs have nothing to compute a gesture pan from — so a partial rewrite would be more confusing than the clear generic message.)

Rebase. Onto origin/main. Four files overlapped with #1369's parameterized-recorded-inputs work (interactions.ts, replay/script.ts, its test, commands.md); all in different functions/regions, applied cleanly, verified both sides' changes coexist (the coordinate-fill parsing sits beside the gesture preflight, recordAs beside the swipe writer guard).

Re-verified post-rebase: typecheck, lint, format:check, check:layering, check:production-exports, check:fallow, check:bundle-owner-files, maestro:conformance (46/46), test:smoke — all green; the 6 gesture-affected unit files pass together. Residual full-suite failures are the same contention flake (router / install-source / screenshot-diff / runner-client, none touched here, all passing in isolation under load).

🤖 Addressed by Claude Code

@thymikee

Copy link
Copy Markdown
Member Author

Review at 4fd5d39: one actionable migration-guide issue remains. The MCP wording is now accurate, but the documented sweep still does not find every stale syntax accepted by the .ad parser. Its regexes require literal spaces and unquoted [-0-9.]+ tokens, while replay tokenization accepts arbitrary whitespace and quoted numeric tokens; examples such as tab-separated swipe ... 300 or gesture fling ... "500" are rejected by the parser as retired forms but missed by the sweep. Please make the scan parser-aligned (or narrow the “every affected line” claim) and cover accepted stale encodings plus valid-form exclusion. Current checks are green and the head is mergeable; do not apply labels until this is fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

chore: plan removal of legacy gesture compatibility shims

1 participant