Playwright Test Suites on GitHub Actions

To run Playwright tests on GitHub Actions, split the suite across a shard matrix, cache the browser binaries under ~/.cache/ms-playwright so every job stops downloading them, and merge the per-shard blob reports into one HTML report at the end. On WarpBuild runners the shape stays the same and two things change: the runs-on label, and the cache action, where WarpBuilds/cache is a drop-in replacement for actions/cache@v4 that is enabled by default on Linux runners.

Sharding fixes wall clock, caching fixes the browser download, and worker count decides whether the vCPUs you pay for are doing anything. This page covers the workflow configuration for a sharded Playwright run, the sizing call between 4, 8, and 16 vCPU runners, the four bottlenecks that dominate Playwright pipelines, the cost math against GitHub-hosted list prices, and how to read shard duration and queue time out of the reports once it is running.

Overview

Every GitHub Actions job starts on a fresh virtual machine. For Playwright that means two empty directories at the start of every job: node_modules, and the browser cache that npx playwright install writes to ~/.cache/ms-playwright on Linux. The browser install pulls a full browser build for each configured project plus the system libraries those builds link against, and it repeats on every job in the matrix unless a cache restores it.

The second default that shapes the run is worker count. The config that npm init playwright generates sets workers: process.env.CI ? 1 : undefined, so the suite that used half your laptop's cores locally runs one test at a time on GitHub Actions. Playwright's own default when workers is unset is half the logical CPU count. A 4 vCPU runner executing one worker is a machine paid for and idle.

Sharding is the third piece. --shard=<n>/<total> splits the test list across jobs, each shard writes a blob report, and merge-reports reassembles them afterwards. Wall clock then tracks the slowest shard rather than the total suite time, which is what most teams actually want from a pull-request check.

Playwright work belongs on Linux x64 for two reasons: the browser builds and system libraries are best supported there, and the caching documentation states that WarpBuild Cache is not supported on Windows runners, which would put the browser cache back on a cold path.

One boundary before the configuration. This page covers Playwright. A Cypress suite has a different runner model, a different cache directory, and a different parallelization mechanism, and it is covered on the Cypress on GitHub Actions page.

Configuration

This workflow shards a Playwright suite four ways, restores the browser binaries by Playwright version, writes a blob report per shard, and merges the blobs in a job that runs after the matrix:

name: playwright
on:
  push:
    branches: [main]
  pull_request:

jobs:
  test:
    runs-on: warp-ubuntu-latest-x64-4x
    strategy:
      fail-fast: false
      matrix:
        shard: [1, 2, 3, 4]
    steps:
      - uses: actions/checkout@v4

      - uses: WarpBuilds/setup-node@v6
        with:
          node-version: 22
          cache: npm

      - run: npm ci

      - name: Read the Playwright version
        id: pw
        run: |
          echo "version=$(node -p "require('@playwright/test/package.json').version")" >> "$GITHUB_OUTPUT"

      - name: Restore Playwright browsers
        id: browsers
        uses: WarpBuilds/cache@v1
        with:
          path: ~/.cache/ms-playwright
          key: ${{ runner.os }}-playwright-${{ steps.pw.outputs.version }}

      - name: Install browsers and system libraries
        if: steps.browsers.outputs.cache-hit != 'true'
        run: npx playwright install --with-deps chromium

      - name: Install system libraries only
        if: steps.browsers.outputs.cache-hit == 'true'
        run: npx playwright install-deps chromium

      - name: Run shard ${{ matrix.shard }}
        env:
          PLAYWRIGHT_BLOB_OUTPUT_FILE: blob-report/report-${{ matrix.shard }}.zip
        run: npx playwright test --shard=${{ matrix.shard }}/4 --workers=3 --reporter=blob

      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: blob-report-${{ matrix.shard }}
          path: blob-report/report-${{ matrix.shard }}.zip
          retention-days: 3

  report:
    if: always()
    needs: test
    runs-on: warp-ubuntu-latest-x64-2x
    steps:
      - uses: actions/checkout@v4
      - uses: WarpBuilds/setup-node@v6
        with:
          node-version: 22
          cache: npm
      - run: npm ci
      - uses: actions/download-artifact@v4
        with:
          path: all-blob-reports
          pattern: blob-report-*
          merge-multiple: true
      - run: npx playwright merge-reports --reporter=html ./all-blob-reports
      - uses: actions/upload-artifact@v4
        with:
          name: playwright-report
          path: playwright-report
          retention-days: 7

Four details in that file carry the weight.

The cache key hashes the Playwright version. Browser builds are pinned to the Playwright release, so 1.56.0 and 1.57.0 need different binaries while an unrelated dependency bump needs none. A key built from the lockfile hash would roll on every dependency change and throw away a perfectly good browser cache. Reading the version out of @playwright/test/package.json into a step output keeps the key rolling exactly when the browsers change.

A cache hit still needs install-deps. The cache covers the browser binaries under ~/.cache/ms-playwright. It does not cover the apt packages the browsers link against, which live in system directories outside that path. Running npx playwright install-deps on the hit branch keeps the restore correct without re-downloading browsers.

The blob reporter carries the shard index. Each shard writes its own zip, the artifact name repeats the index so uploads never collide, and merge-reports reassembles a single HTML report in the report job. fail-fast: false keeps the other shards running when one fails, which is the only way the merged report describes the whole suite.

WarpBuilds/setup-node routes dependency caching through WarpBuild Cache. It is a drop-in for actions/setup-node, and on WarpBuild runners the npm cache it manages is served by WarpBuild Cache with no extra configuration.

Two behaviors from the caching documentation matter before you rely on the browser cache. Entries are scoped to key, version, and branch, so seed the cache on main and let branch builds restore from it. The version hash covers the compression tool and the cached paths, which means a cache written on a macOS runner cannot restore on a Linux runner even with an identical key. Entries expire 7 days after last use, so an active repository stays warm and an abandoned branch stops costing storage on its own.

Cache usage is metered at $0.20 per GB-month of storage and $0.0001 per write or restore operation on hosted runners, and it is free on BYOC.

Sizing

Worker count is the number that connects Playwright to the runner you picked. Each worker is a separate process driving its own browser context, so workers scale with vCPU until memory or the application under test becomes the limit. These are the Linux x64 rows that matter for a sharded suite:

Pick the size from what the suite can keep busy:

4x for a small suite. Four vCPU and 16GB run 3 workers with headroom for the application under test in the same job. This is the default for a few hundred tests split into three or four shards, and it is where most Playwright pipelines should start. The merge job needs almost nothing, so send it to 2x at $0.004 per minute.

8x when workers saturate. If every worker on a 4x shard stays busy and the shard is still the long pole, 8 vCPU and 32GB carry 6 or 7 workers. This is the size for suites where each test drives a heavy single-page application, or where the shard also runs a dev server, a database, and a queue alongside the browsers.

16x for suites that saturate 8 workers. Sixteen vCPU and 64GB earns its rate when a shard runs a full application stack next to a dozen browser processes and memory becomes the limit before CPU does. Check utilization before buying it: a shard that never passes 50 percent CPU on 8x gains nothing from 16x, and the fix there is more shards.

Set --workers explicitly on every size. Leaving it at the scaffolded 1 wastes the machine, and leaving it unset gives you half the cores, which is conservative when the application under test lives outside the job.

Worked cost model

GitHub publishes per-minute list prices for its hosted runners on the GitHub Actions minute multipliers reference. Checked on 2026-08-13, the standard ubuntu-latest runner for private repositories (2 vCPU) is $0.006 per minute, and the larger Linux runners are $0.012 for 4 vCPU, $0.022 for 8 vCPU, and $0.042 for 16 vCPU.

Take a concrete team: a Playwright suite that runs 550 times a month across pull requests and pushes to main, sharded four ways, with each shard taking 9 minutes on a 4 vCPU machine. That is 19,800 shard minutes a month, plus a 2 minute merge job per run.

The WarpBuild column totals $163.45 a month against $237.60 for the same minutes on GitHub-hosted 4-core runners, before any change to how long a shard takes.

Stated as list-price arithmetic: warp-ubuntu-latest-x64-4x (4 vCPU, 16 GB) costs $0.008 per minute against $0.012 per minute for the 4-core Linux larger runner (4 vCPU, 16 GB): 33 percent lower list price. GitHub list price checked on 2026-08-13. The same arithmetic at the next size up: warp-ubuntu-latest-x64-8x (8 vCPU, 32 GB) costs $0.016 per minute against $0.022 per minute for the 8-core Linux larger runner (8 vCPU, 32 GB): 27 percent lower list price. GitHub list price checked on 2026-08-13.

Every size and platform is listed on the pricing page.

Bottlenecks

Four things account for most slow Playwright pipelines on GitHub Actions.

Browser downloads on every run. Without a restored ~/.cache/ms-playwright, every shard downloads a browser build per configured project before the first test starts, and a three-browser project multiplies that by three across every job in the matrix. The cache step above removes it. Confirm the restore worked by reading steps.browsers.outputs.cache-hit in the job log rather than assuming it: a mistyped path or a version key that moved will silently fall through to a full download.

One worker per job. The scaffolded workers: process.env.CI ? 1 : undefined is the single most common reason a Playwright suite runs slowly on GitHub Actions. The suite executes serially while the other cores sit idle, and adding shards papers over it while multiplying the runner minutes. Set the worker count per runner size first, then decide how many shards you need.

Video and trace artifacts. trace: 'on' and video: 'on' write a file per test, and the upload step then compresses and ships gigabytes of them. The scaffolded trace: 'on-first-retry' is the setting worth keeping, with video: 'retain-on-failure' when a video actually helps triage. Setting a short retention-days on the artifact keeps the storage from accumulating across every pull request.

Flaky retries inflating wall clock. With retries: 2, one flaky test costs up to three times its runtime, and it lands entirely on the shard that owns it, so the merged run waits for a single misbehaving spec. Retries hide the flake instead of fixing it, and the shard duration is where the cost shows up. The playbook for finding and removing them is in the guide on fixing flaky GitHub Actions jobs.

Reading shard duration and queue time

Guessing which shard is the long pole wastes more time than measuring it. The reports documentation covers three sections, and two of them answer this directly.

The Jobs report aggregates every run of a unique repository, workflow, and job name combination, and gives Duration P75 and P90, Queue Time P75 and P90, plus CPU and Memory at the same two percentiles when observability is enabled. Because a sharded matrix produces one row per shard, the spread between the fastest and slowest shard row shows up immediately, and an uneven split shows as one row with a Duration P90 well above its siblings. A shard whose CPU P75 sits low is the signal that the worker count, rather than the runner size, is the constraint.

The Queue Timings report breaks queue wait down per runner label and stack with Queue Time P75 and P90. Read it before blaming a shard: time spent waiting for a runner is time the Jobs report attributes to queueing rather than execution, and the two together separate a slow suite from a slow start. Both reports support date ranges, filtering, and CSV export when you want the numbers next to your own history. The reasoning behind reading P75 and P90 instead of averages is covered in the guide on GitHub Actions build duration percentiles.

When the cause is still unclear, WarpBuild's CI observability reports OpenTelemetry-based system metrics from the runner agent correlated with GitHub Actions job logs, which separates a CPU-bound worker pool from a shard stuck on a network call. The Action Debugger pauses a workflow and opens an SSH session on the runner, so you can list ~/.cache/ms-playwright on the machine itself and see exactly what the restore produced.

Proof

Public OSS repositories running warp- labels are citable evidence, and two of them run Playwright in the open:

• documenso/documenso runs its Playwright Tests workflow on warp-ubuntu-2204-x64-8x, starting its services and database in the job before npx playwright install --with-deps and the suite itself, then uploading test-results with a 7 day retention (checked on 2026-08-13).
• appsmithorg/appsmith runs its Playwright end-to-end job on warp-ubuntu-latest-x64-4x, caches ~/.cache/ms-playwright, runs the suite with --reporter=blob, reruns only failures with --last-failed, and merges the blobs with npx playwright merge-reports (checked on 2026-08-13).

The second one is worth reading in full if your suite has flaky specs, because the rerun-failures pattern it uses keeps the blob report intact across both attempts.

The same sharding and caching shape applies to the rest of a JavaScript pipeline; the Node.js on GitHub Actions page covers the install and build phases that run before the browsers start.

FAQ

How many shards should a Playwright suite use?

Enough that the slowest shard finishes inside the wall clock you want, and no more. Each extra shard repeats checkout, install, and browser restore, so a suite that already finishes in four minutes per shard gains little from doubling the matrix.

Why does Playwright run one worker on GitHub Actions?

The config generated by npm init playwright sets workers to 1 when process.env.CI is set. That serializes the suite on a machine with 4 or more cores, so set the worker count explicitly per runner size.

What should the Playwright browser cache key hash?

The installed Playwright version, since browser builds are pinned to it. Read it from @playwright/test/package.json into a step output and put that value in the cache key so a Playwright upgrade rolls the entry.

How do I get one report out of a sharded run?

Run each shard with --reporter=blob, upload the per-shard zip as an artifact, then run npx playwright merge-reports --reporter=html over the downloaded blobs in a job that needs the matrix.