Sharding Test Suites Across a Matrix

Sharding a test suite across a GitHub Actions matrix means putting a shard index in strategy.matrix, passing that index and the shard total to the test command so each job runs a disjoint slice, and merging the per shard reports in a dependent job. The split has to be deterministic and balanced by recorded duration, because a matrix fixes the assignment before the run starts and the slowest shard sets the wall clock for the whole gate.

This guide covers the three failures that make a sharded matrix report the wrong answer, the edits that fix each one, a workflow with a shard index matrix and a merge job, and the arithmetic that prices eight small shards against one large runner. For the language neutral definition see the test sharding entry, for matrix semantics see the guide to matrix builds, and for the rest of the run see the hub on speeding up GitHub Actions.

Diagnosis

A sharded matrix fails quietly. The grid is green, the job count is high, and the number that matters has stopped moving. Three failures account for most of it, and each has a distinct symptom.

The split is unbalanced

Splitting by test file count gives eight jobs of eight different lengths. Total billed minutes stay the same, since the same tests run either way, but the run waits for the longest shard while the other seven sit finished. A shard holding 1.5 times the mean work adds that overshoot to every run of the gate.

Read duration P75 and P90 per job in the Jobs report, which reports both percentiles for every repository, workflow, and job name combination with CSV export (reports documentation). Shards from one matrix have distinct job names, so the spread between the fastest and slowest row is the imbalance, in minutes, without any extra instrumentation. On GitHub-hosted runners, pull per job timings from the workflow jobs API and compute the spread yourself.

The assignment is not reproducible

A failure reported by shard 3 is only useful when shard 3 can be rerun on a laptop and produce the same slice. Assignment drifts when the test file list comes from a glob whose order depends on the filesystem, when the runner randomizes test order per process, or when the split is keyed on a durations file that a scheduled job rewrites. The symptom is a shard that fails in the pull request and passes when rerun alone.

The results are never merged

Each shard writes its own report, so something has to combine them into one verdict. Two defaults work against that. A dependent job inherits if: success(), so the merge job is skipped exactly when a shard fails and the report matters most. And actions/upload-artifact@v4 writes immutable artifacts and rejects a second upload under an existing name (upload-artifact, checked on 2026-08-13), so shards that all write test-report fail after the first one wins the race.

Fix

Three properties make a shard split a faithful substitute for running the suite on one machine, and each maps to one edit.

Make the assignment deterministic. Sort the file list before splitting it, pin the shard total in one place so the index and the total can never disagree, and pass an explicit seed when the runner randomizes order. Refresh a durations file on a schedule rather than on every run, so the assignment is stable across the reruns of a single pull request.

Balance by recorded time. Test file count is a poor proxy for duration; recorded per test time is the real one. Most runners either accept a durations file or expose per test timings in their report output, and a split built from last quarter's timings drifts as tests are added. When no timing data exists, sorting the slowest known files first and dealing them round robin across shards gets most of the way there.

Merge, and require the merged job. Collect the per shard artifacts in a dependent job with if: always(), produce one report, and make that job the required status check. Matrix job names carry the shard index, so requiring them means editing branch protection every time the count changes.

Then give the fan-out somewhere to land. A fresh runner is provisioned per job. Generally available Linux and Windows runners do not have plan-level concurrency caps, so raising the shard count is an edit to the matrix rather than a plan upgrade. Which ceiling applies where is covered in the answer on how many jobs a matrix can fan out to.

Configuration

Eight shards on one label, a shard aware test command, per shard artifacts, and a merge job that owns the status check.

name: test
on:
  pull_request:
  push:
    branches: [main]

jobs:
  test:
    name: test ${{ matrix.shard }}/${{ strategy.job-total }}
    runs-on: warp-ubuntu-latest-x64-4x
    strategy:
      fail-fast: false
      matrix:
        shard: [1, 2, 3, 4, 5, 6, 7, 8]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
          cache: npm
      - run: npm ci
      - name: Run shard
        run: npx playwright test --shard=${{ matrix.shard }}/${{ strategy.job-total }} --reporter=blob
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: blob-report-${{ matrix.shard }}
          path: blob-report
          retention-days: 3

  report:
    name: merged test report
    needs: test
    if: always()
    runs-on: warp-ubuntu-latest-x64-2x
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
          cache: npm
      - run: npm ci
      - uses: actions/download-artifact@v4
        with:
          path: all-blob-reports
          pattern: blob-report-*
          merge-multiple: true
      - run: npx playwright merge-reports --reporter=html ./all-blob-reports
      - uses: actions/upload-artifact@v4
        with:
          name: html-report
          path: playwright-report
      - name: Fail the gate when any shard failed
        if: needs.test.result != 'success'
        run: exit 1

What each piece does:

• strategy.job-total is the number of jobs the matrix expanded to, so the shard total is written once in the shard list and read everywhere else (GitHub Actions contexts, checked on 2026-08-13). With one dimension it equals the shard count; adding a second dimension multiplies it, so a two dimensional grid needs an explicit total in a matrix key instead.
• fail-fast: false keeps the other seven shards running after one fails, so a single flaky test costs one rerun rather than a full grid.
• if: always() on the upload step preserves the report from a shard that failed, and the same key on the report job stops the merge from being skipped when the matrix is red.
• Artifact names carry the shard index, which is what keeps eight parallel uploads from colliding under one immutable name. merge-multiple: true on the download unpacks every matching artifact into one directory so the merge command can read them as a set.
• needs.test.result is one string for the whole matrix job and is failure when any cell failed, so the last step turns eight shard outcomes into one required check with a fixed name.
• Where the shard count is computed rather than written, generate the list in a setup job, expose it as an output, and read it with matrix: shard: ${{ fromJSON(needs.setup.outputs.shards) }}.

Any runner with a shard flag drops into the same shape, and the guide to sharding pytest suites covers the flag and the merge step for that framework specifically. Runners without one are sharded from the outside: list the test files, sort them, take every eighth entry starting at the shard index, and pass that slice as arguments.

The labels above resolve to these machines, from the WarpBuild cloud runners documentation, checked on 2026-08-13.

Cost or Time Model

Two formulas cover a shard matrix. Wall clock is s + (T / w) * m, and billed minutes are N * (s + T / w), where T is serial suite execution time, N is the shard count, w is the total worker processes across all shards, s is the per job setup a shard pays before its first test, and m is the share of the work held by the largest shard.

Assumptions: T of 96 minutes, s of 3 minutes for checkout, npm ci, and a warm cache, one worker per vCPU, an even split, no retries, and queue wait excluded. Rates come from the catalog above.

Read the last two rows together, because they are the choice this page exists to price. Eight 4 vCPU shards and one 32 vCPU job hold the same 32 vCPU, finish in the same 6 minutes, and cost the same $0.38, since every Linux x64 size in the catalog charges $0.002 per vCPU minute and the repeated setup on the shard plan is paid on quarter size machines: 8 * 3 * $0.008 is the same $0.19 as 1 * 3 * $0.064. Adding the merge job on warp-ubuntu-latest-x64-2x for 2 minutes puts $0.008 on top of the shard plan.

Three things break that tie, and none of them is price.

• Imbalance is paid by the shard plan alone. A largest shard holding 1.5 times the mean pushes m to 1.5, so wall clock goes from 6 minutes to 7.5 while billed minutes stay at 48. The single machine rebalances at run time and keeps its 6 minutes.
• Rerun granularity favors shards. Rerunning one failed shard costs 6 billed minutes at $0.008, or $0.05. Rerunning the single job costs its 6 minutes at $0.064, or $0.38, because the whole suite goes back on the machine.
• Ceilings favor shards at the top end. A suite needing more than 128 GB, or one approaching the 6 hour single job limit (GitHub Actions usage limits, checked on 2026-08-13), has no single machine option left at all.

Those billed minutes carry a list price. GitHub rates below are from the GitHub Actions billing reference, checked on 2026-08-13.

warp-ubuntu-latest-x64-4x (4 vCPU, 16 GB) costs $0.008 per minute against $0.012 for the 4-core Linux larger runner (4 vCPU, 16 GB): 33 percent lower list price. warp-ubuntu-latest-x64-32x (32 vCPU, 128 GB) costs $0.064 per minute against $0.082 for the 32-core Linux larger runner (32 vCPU, 128 GB): 22 percent lower list price. GitHub list prices checked on 2026-08-13.

Scale the eight shard plan to a working month at 40 runs a day: 1,920 billed minutes a day, which is $15.36 on warp-ubuntu-latest-x64-4x against $23.04 on the GitHub-hosted 4-core larger runner, or $337.92 against $506.88 over 22 working days. The same change returns 21 minutes of wall clock per run against the single 4 vCPU job.

Full rates by runner type are on the pricing page.

FAQ

How do I shard a test suite across a GitHub Actions matrix?

Put a shard index in strategy.matrix, set runs-on to a single label, and pass the index and the shard total to the test command as --shard=${{ matrix.shard }}/${{ strategy.job-total }}. Upload one report artifact per shard under a name that carries the shard index, then merge the artifacts in a dependent job that runs with if: always() so a failed shard still produces a report.

Is it cheaper to run eight small shards or one large runner?

At the Linux x64 catalog rates the two price the same, because every size charges $0.002 per vCPU minute: eight 4 vCPU shards and one 32 vCPU job both hold 32 vCPU and both burn $0.064 per minute of wall clock. The difference is repeated setup on the shard plan against the memory ceiling and the all-or-nothing rerun on the single machine.

How do I keep one required status check when the shard count changes?

Require the merge job rather than the matrix job. Matrix job names carry the shard index, so a name like test 4/8 disappears the moment the count changes and branch protection waits forever on a check that no run will report. The merge job has a fixed name, needs the matrix job, and can fail the gate on needs.test.result.