Skip to content

CI: run the backend suite in parallel (pytest-xdist) and fix the warnings that hide test bugs #1408

Open

Description

@plauzy

Scope narrowed (2026-11-18). #1387 already removes the duplicate backend run: the coverage ratchet reads the Unit Tests (3.12) report, and CAO MCP Apps took 55 s on its run. Thanks @fanhongy. This issue now covers only what #1387 does not, and the work starts after #1387 and the #1350 test-stability series (#1389, #1394, #1401, #1402, #1405, #1406) merge, so nothing edits the same job or tests twice.

Problem

The backend suite runs in one process: pytest-xdist is a dev dependency, but no job passes -n. On #1400 (run 44269686813) it reported 28251 passed, 148 skipped, 22 deselected, 8 xfailed, 361 warnings in 2715.61s (0:45:15). Unit Tests (3.12) has grown from 16 min (2026-10-19) to 45 min (2026-11-16) on main, against its 60-minute timeout.

Proposal

  1. Parallel tests. Use pytest -n auto --dist loadgroup in both Unit Tests legs.
  2. Measure. Record --durations=25 and the job's wall-clock time on main and on the head in the PR body.
  3. Warnings that point at test bugs:
    • RuntimeWarning: coroutine 'AsyncMockMixin._execute_mock_call' was never awaited at services/herdr_inbox_service.py:1470, a mocked writer that is never awaited;
    • the HTTP_422_UNPROCESSABLE_ENTITY and Click isolated_filesystem deprecations.

Done when

  • Unit Tests (3.12) wall-clock time drops measurably with -n auto, with three consecutive green runs on the PR head, or the PR reports why parallel runs are unsafe and lands only the warning fixes.
  • No blanket filterwarnings ignores.

Activity

  1. fanhongy commented on Nov 18, 2026

    @fanhongy
    Collaborator

    Thanks @plauzy. #1387 already makes this change: the coverage ratchet reads the Unit Tests (3.12) coverage report, and CAO MCP Apps no longer runs the backend suite (that job took 55 s on #1387's run). haofeif approved it, and it's waiting for his re-approval after a comment-only P3 fix.

    If #1387 lands first, #1400 won't need a ci.yml change for #1408, so the two PRs won't conflict.

    🤖 Generated with Claude Code

  2. changed the title [-]CI: the backend test suite runs twice per push (Unit Tests and CAO MCP Apps), about 45 minutes each[/-] [+]CI: run the backend suite in parallel (pytest-xdist) and fix the warnings that hide test bugs[/+] on Nov 18, 2026
  3. plauzy commented on Nov 18, 2026

    @plauzy
    CollaboratorAuthor

    Thanks @fanhongy, agreed. I narrowed this issue to the parallel-test trial and the warning fixes, and it now waits for #1387 and the #1350 test series. #1400 won't touch ci.yml for this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions