Skip to content

test: fix flaky test Shield entry modal in shield-plan-subscription - #47608

Open
LeVinhGithub wants to merge 1 commit into
mainfrom
harry/fix-flaky-shield-entry-modal
Open

LeVinhGithub wants to merge 1 commit into
mainfrom
harry/fix-flaky-shield-entry-modal

Conversation

@LeVinhGithub

@LeVinhGithub LeVinhGithub commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Description

Shield subscription E2E tests sometimes click Get started while the Shield entry modal is still sliding up. The click misses, the modal stays open, and the test times out waiting for the Shield plan page.

Flake type: Race Conditions with React Re-renders

Root cause:

  • The test opens Settings, then Transaction Shield, which opens the Shield entry modal.
  • The modal slides up for about 400ms, and a large animation canvas loads inside it while the buttons are still moving.
  • The test sees the modal and clicks Get started about 95ms later.
  • The click hits while the button is still moving, so it misses.
  • The modal stays open on top of Settings.
  • The test then waits for the Shield plan page and times out after 10 seconds.
  • The same miss fails the monthly crypto subscription, the annual card subscription, and the settings back-button test.
  • The navigation tests also click Back before the Shield plan page has finished opening, so Back is pressed while the wallet is still on the modal.

Solution:

  • Get started and Skip now wait until the button can be clicked and has stopped moving, then click and wait for the button to disappear, so the click lands after the slide-up animation.
  • The navigation tests wait until the Shield plan page is loaded before clicking Back, so Back is only pressed once that page is on screen.

Slack: https://consensys.slack.com/archives/CTQAGKY5V/p1793775998824649?thread_ts=1793693705.182669&cid=CTQAGKY5V

Changelog

CHANGELOG entry: null

Manual testing steps

  1. Start a test build with yarn start:test.
  2. Run the Shield subscription spec:
SELENIUM_BROWSER=chrome yarn test:e2e:single test/e2e/tests/shield/shield-plan-subscription.spec.ts --browser=chrome
  1. Confirm these cases pass:
    • Shield Settings Subscription, monthly plan, crypto payment
    • Shield Settings Subscription, annual plan, card payment
    • Navigation test that returns to Settings when Back is clicked

Screenshots/Recordings

Before

CI failure screenshot (should subscribe to the shield plan from the settings > shield - monthly plan, should subscribe to the shield plan from the settings > shield - annual plan, and should shield-plan page redirect to settings page when user clicks back button): Shows the Settings page in the background (Preferences, Notifications, Security and privacy, Privacy, Backup and security, Transaction Shield) with the "Introducing Transaction Shield" modal ([data-testid="shield-entry-modal"]) still open on top of Settings. Inside the modal body, the heading, subtitle, and 252px Rive shield illustration fill almost the entire modal height (395px), leaving only a ~2px white sliver of the "Start 14-day free trial" button ([data-testid="shield-entry-modal-get-started-button"]) visible at the very bottom edge because the click fired while the modal and Rive canvas were still animating/mounting and missed the button.

After

No product UI change. Get started and Skip wait until the button is clickable and has stopped moving before clicking and waiting for it to disappear, and the navigation tests wait for the Shield plan page to load before clicking Back.

Pre-merge author checklist

Pre-merge reviewer checklist

  • I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

Note

Low Risk
E2E page-object and spec timing only; no application or production code changes.

Overview
Fixes flaky Shield subscription E2E failures caused by clicking Get started / Skip while the entry modal is still animating (slide-up + Rive canvas), which left the modal open and timed out waiting for the plan page.

HomePage shield modal actions now wait for the button to be clickable and stationary, then click and wait for the control to disappear—matching patterns used elsewhere in page objects.

Navigation tests in shield-plan-subscription.spec.ts now assert the Shield plan page is loaded before Back, so back navigation isn’t fired while the wallet is still transitioning off the modal.

Reviewed by Cursor Bugbot for commit adbbf32. Bugbot is set up for automated code reviews on this repo. Configure here.

Co-authored-by: Cursor <cursoragent@cursor.com>
@LeVinhGithub LeVinhGithub self-assigned this Nov 4, 2026
@github-actions github-actions Bot added the size-S label Nov 4, 2026
@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

@metamask-ci

metamask-ci Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

✨ Files requiring CODEOWNER review ✨

🧪 @MetaMask/qa (1 files, +10 -2)
  • 📁 test/
    • 📁 e2e/
      • 📁 page-objects/
        • 📁 pages/
          • 📁 home/
            • 📄 homepage.ts +10 -2

@metamask-ci metamask-ci Bot added the team-qa QA team label Nov 4, 2026
@metamask-ci

metamask-ci Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor
Builds ready [adbbf32] [reused from 2338170]
⚡ Performance Benchmarks (Total: 🟢 8 pass · 🟡 2 warn · 🔴 3 fail)

Baseline (latest main): f693520 | Date: 11/3/2026 | Pipeline: 41793052297 | Baseline logs

Metricschrome-webpackfirefox-webpack
loadNewAccount
[Sentry log · main/release]
🔴 load_new_account(p95) [CI log]🔴 load_new_account(p95) [CI log]
onboardingNewWallet
[Sentry log · main/release]
🟢 [CI log]🔴 [CI log]

Regressions (🔴 3 failures)

Interaction Benchmarks · Samples: 5 🔴 2
Benchmarkchrome-webpackfirefox-webpack
loadNewAccount
[Sentry log · main/release]
🔴 [CI log]
🔴 load_new_account
🔴 [CI log]
🔴 load_new_account
confirmTx
[Sentry log · main/release]
🟢 [CI log]🟢 [CI log]
bridgeUserActions
[Sentry log · main/release]
🟢 [CI log]🟢 [CI log]

📈 Results compared to the previous 5 runs on main

  • ↑ loadNewAccount/load_new_account: +177%
  • ↑ loadNewAccount/total: +177%
  • ↓ loadNewAccount/inp: -33%
  • ↓ loadNewAccount/fcp: -35%
  • ↑ loadNewAccount/cls: +282%
  • ↓ confirmTx/longTaskTotalDuration: -15%
  • ↓ confirmTx/tbt: -23%
  • ↓ confirmTx/inp: -44%
  • ↓ confirmTx/fcp: -34%
  • ↓ confirmTx/lcp: -28%
  • ↓ bridgeUserActions/bridge_load_page: -29%
  • ↓ bridgeUserActions/bridge_load_asset_picker: -33%
  • ↓ bridgeUserActions/bridge_search_token: -10%
  • ↓ bridgeUserActions/longTaskTotalDuration: -42%
  • ↓ bridgeUserActions/longTaskMaxDuration: -46%
  • ↓ bridgeUserActions/tbt: -92%
  • ↓ bridgeUserActions/total: -16%
  • ↓ bridgeUserActions/inp: -26%
  • ↓ bridgeUserActions/fcp: -33%
  • ↓ bridgeUserActions/lcp: -24%
  • ↑ bridgeUserActions/cls: +281%
  • ↑ loadNewAccount/load_new_account: +184%
  • ↑ loadNewAccount/total: +184%
  • ↓ loadNewAccount/inp: -18%
  • ↓ loadNewAccount/fcp: -36%
  • ↑ loadNewAccount/lcp: +1715%
  • ↓ confirmTx/longTaskCount: -100%
  • ↓ confirmTx/longTaskTotalDuration: -100%
  • ↓ confirmTx/longTaskMaxDuration: -100%
  • ↓ confirmTx/tbt: -100%
  • ↓ confirmTx/inp: -13%
  • ↓ confirmTx/fcp: -46%
  • ↑ confirmTx/lcp: +1331%
  • ↑ bridgeUserActions/bridge_load_page: +249%
  • ↑ bridgeUserActions/bridge_load_asset_picker: +115%
  • ↓ bridgeUserActions/longTaskCount: -100%
  • ↓ bridgeUserActions/longTaskTotalDuration: -100%
  • ↓ bridgeUserActions/longTaskMaxDuration: -100%
  • ↓ bridgeUserActions/tbt: -100%
  • ↑ bridgeUserActions/total: +54%
  • ↑ bridgeUserActions/inp: +76%
  • ↓ bridgeUserActions/fcp: -34%
  • ↑ bridgeUserActions/lcp: +1237%

🌐 Core Web Vitals — 🟢 good · 🟡 needs improvement · 🔴 poor (web.dev thresholds)

  • 🟡 loadNewAccount/LCP: p75 3.2s
Startup Benchmarks · Samples: 100
Benchmarkchrome-webpackfirefox-webpack
startupStandardHome
[Sentry log · main/release]
🟢 [CI log]🟡 [CI log]
🟡 loadScripts

📈 Results compared to the previous 5 runs on main

  • ↑ startupStandardHome/uiStartup: +15%
  • ↑ startupStandardHome/load: +11%
  • ↑ startupStandardHome/domContentLoaded: +11%
  • ↑ startupStandardHome/domInteractive: +22%
  • ↓ startupStandardHome/firstReactRender: -98%
  • ↓ startupStandardHome/initialActions: -29%
  • ↑ startupStandardHome/loadScripts: +11%
  • ↑ startupStandardHome/setupStore: +166%
  • ↑ startupStandardHome/longTaskTotalDuration: +10%
  • ↑ startupStandardHome/longTaskMaxDuration: +10%
  • ↑ startupStandardHome/tbt: +15%
  • ↑ startupStandardHome/fcp: +11%
  • ↑ startupStandardHome/cls: +400%
  • ↑ startupStandardHome/domInteractive: +45%
  • ↓ startupStandardHome/firstReactRender: -98%
  • ↓ startupStandardHome/initialActions: -50%
  • ↑ startupStandardHome/setupStore: +149%
  • ↑ startupStandardHome/inp: +150%
  • ↑ startupStandardHome/fcp: +46%
User Journey Benchmarks · Samples: 5 · mock API 🔴 1
Benchmarkchrome-webpackfirefox-webpack
onboardingImportWallet
[Sentry log · main/release]
🟢 [CI log]🟡 [CI log]
🟡 total
onboardingNewWallet
[Sentry log · main/release]
🟢 [CI log]🔴 [CI log]
🔴 total

📈 Results compared to the previous 5 runs on main

  • ↑ onboardingImportWallet/srpButtonToSrpForm: +38%
  • ↑ onboardingImportWallet/confirmSrpToPwForm: +41%
  • ↑ onboardingImportWallet/pwFormToMetricsScreen: +31%
  • ↑ onboardingImportWallet/metricsToWalletReadyScreen: +23%
  • ↓ onboardingImportWallet/doneButtonToHomeScreen: -92%
  • ↓ onboardingImportWallet/openAccountMenuToAccountListLoaded: -73%
  • ↓ onboardingImportWallet/longTaskCount: -100%
  • ↓ onboardingImportWallet/longTaskTotalDuration: -100%
  • ↓ onboardingImportWallet/longTaskMaxDuration: -100%
  • ↓ onboardingImportWallet/tbt: -100%
  • ↓ onboardingImportWallet/total: -85%
  • ↓ onboardingNewWallet/createPwToRecoveryScreen: -16%
  • ↓ onboardingNewWallet/agreeButtonToOnboardingSuccess: -33%
  • ↓ onboardingNewWallet/doneButtonToAssetList: -77%
  • ↓ onboardingNewWallet/longTaskCount: -100%
  • ↓ onboardingNewWallet/longTaskTotalDuration: -100%
  • ↓ onboardingNewWallet/longTaskMaxDuration: -100%
  • ↓ onboardingNewWallet/tbt: -100%
  • ↓ onboardingNewWallet/total: -73%
Dapp Page Load Benchmarks · Samples: 100
Benchmarkchrome-webpack
dappPageLoad
[Sentry log · main/release]
🟢 [CI log]

📈 Results compared to the previous 5 runs on main

  • ↓ dappPageLoad/pageLoadTime: -49%
  • ↓ dappPageLoad/firstPaint: -33%
  • ↓ dappPageLoad/firstContentfulPaint: -33%
Bundle Size Diffs
Status Bundle Total Diff Change
✅ background 15.26 MiB 0 Bytes 0.00%
✅ ui 19.33 MiB 0 Bytes 0.00%
✅ common 0 Bytes 0 Bytes 0.00%
✅ other 4.44 MiB 0 Bytes 0.00%
✅ content scripts 3.23 MiB 0 Bytes 0.00%
✅ unzipped 79.64 MiB 0 Bytes 0.00%
✅ zip 23.81 MiB 0 Bytes 0.00%

@LeVinhGithub
LeVinhGithub marked this pull request as ready for review November 4, 2026 08:17
@LeVinhGithub
LeVinhGithub requested a review from a team as a code owner November 4, 2026 08:17

This branch was successfully deployed

1 active deployment
pr-comment — adbbf321 Deployed Nov 4, 2026 by LeVinhGithub via identify-codeowners #88417
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants