Skip to content

1.2.0

Choose a tag to compare

@jochen-testingbot jochen-testingbot released this 02 Sep 12:45
· 78 commits to main since this release
3c322a6

Highlights

This release makes testingbot maestro a first-class citizen in CI pipelines: machine-readable results, meaningful exit codes, commands to come back for async runs, app reuse, device matrices, and the plumbing the new TestingBot Maestro GitHub Action builds on.

Machine-readable output and exit codes

  • --json prints one JSON document on stdout (logs move to stderr); --json-file / --json-file-name write it to disk while keeping console output. Every flow attempt is listed with attempt, latest, passed, device, timings and errors.
  • Exit codes now distinguish test failures (2) from CLI or infrastructure errors (1) for maestro, espresso and xcuitest. With --json-file a failed run still exits 0 so the file is the contract.
  • JSON reports the device that actually ran, not a wildcard request. (#31, #36)

New commands for async runs

  • testingbot status --id <projectId> [--wait]: snapshot of every run and flow; --wait attaches with the live flow table. Ctrl-C detaches without cancelling.
  • testingbot artifacts --id <projectId>: download --report and/or --download-artifacts for a finished project.
  • testingbot list [--count] [--offset]: recent projects, newest first.
  • testingbot upload <app>: upload an app once and get a project ID. (#32, #33)

Reuse an uploaded app

  • --app-binary-id <projectId> runs new flows against an app uploaded earlier without re-uploading it; platform is taken from the stored app. Every run prints its project ID after the upload. Requires the matching TestingBot API deployment. (#33)

Device matrix

  • --device-matrix "<device>[:<version>][:real]" (comma-separated or repeatable) runs every flow on each listed device in one command, each device with its own run, results and retries. The CLI prints devices × flows before submitting. (#35)

Drop-in flags and reports

  • --branch, --pr-url and repeatable -m/--metadata KEY=VALUE run metadata.
  • --exclude-flows to leave files, directories or globs out of the bundle.
  • --report allure writes Allure result files next to the existing junit/html formats.
  • Alternative spellings for common flags are accepted as hidden aliases (--app-file, --flows, --apiKey, --device-model, --device-os, --format, --output, --test-suite-name), so command lines written for other Maestro runners work unchanged. (#34)

GitHub PR checks

  • When a run carries repository and commit metadata and the TestingBot GitHub App is installed, the server posts a TestingBot / tests check on the pull request; the CLI logs when that happened. (#37)

Fixes and maintenance

  • Maestro: fail instead of uploading a bundle with no runnable flow.
  • Repeatable options no longer leak values between invocations.
  • Tests: the --exclude-flows directory test is path-separator agnostic (fixes Windows CI).
  • Dependency and code-scanning updates (js-yaml, socket.io-parser, brace-expansion; workflow permissions).

Full Changelog: 1.1.1...1.2.0