Parallel .NET Test Runs on Windows Runners

dotnet test runs test assemblies one process at a time by default, and whether the tests inside a single assembly run concurrently is decided by the test framework rather than by the .NET CLI. Parallel .NET test runs on GitHub Actions therefore need both levels turned on, and only after that does a larger Windows label such as warp-windows-latest-x64-16x at $0.064 per minute return anything for the extra cores.

WarpBuild provides Linux x64, Linux ARM64, macOS, and Windows runners. This guide covers the Windows side of a .NET test job: how to tell which parallelism level is capped, what to set at each level, how to tie those settings to the label size, and what one large runner costs against four small ones once both levels are correct.

Diagnosis

Start with peak CPU during the test step rather than with the total job duration. The Jobs section of the WarpBuild Reports page aggregates every run of a given repository, workflow, and job name and prints Duration P75 and P90, Queue Time P75 and P90, and CPU and Memory P75 and P90. Those CPU percentiles turn the question into arithmetic.

One fully busy core on a 16 vCPU runner reports about 6 percent peak CPU. One busy core on a 4 vCPU runner reports about 25 percent. So a warp-windows-latest-x64-16x job whose test step peaks near 6 percent is running a single-threaded test host on a machine renting 16 cores, and no label change fixes that. CPU and memory percentiles come from the runner agent telemetry that WarpBuild's CI observability collects, so enable observability on the runner before reading those columns.

Four causes account for almost every capped .NET test run:

• Assembly-level parallelism is off. The VSTest RunConfiguration.MaxCpuCount setting decides how many test host processes run at once, and it defaults to one. A solution with 12 test projects then executes 12 assemblies back to back.
• The runsettings file is never loaded. A .runsettings file in the repository root does nothing unless dotnet test receives --settings, and a workflow that was copied between repositories often drops that argument.
• In-assembly parallelism is off or unsafe. NUnit and MSTest run tests sequentially inside an assembly until you opt in with an assembly-level attribute, and xUnit parallelizes test collections but keeps everything inside one collection when the suite uses a single shared fixture.
• Thread count exceeds the label. The opposite failure. Sixteen concurrent test host processes, each spawning worker threads sized to the machine, oversubscribe a 16 vCPU runner, and the extra context switching shows up as high CPU with a longer wall clock than a smaller setting produces.

Parallel-unsafe tests read as flakiness rather than as slowness: tests that pass alone and fail together usually share a static field, a fixed TCP port, a file under the working directory, or one database. Reproduce those interactively with the Action Debugger against the same label instead of pushing commits to guess.

Fix

Work down this order. The first two settings cost nothing per minute and usually return more wall clock than a bigger label does.

1. Turn on in-assembly parallelism in the framework. In xUnit, set parallelizeTestCollections and maxParallelThreads in xunit.runner.json. In NUnit, add [assembly: Parallelizable(ParallelScope.Fixtures)] and set LevelOfParallelism, documented on the Parallelizable attribute page. In MSTest, add [assembly: Parallelize(Workers = 0, Scope = ExecutionScope.MethodLevel)], where zero means the processor count, described in the MSTest documentation.

2. Turn on assembly-level parallelism in VSTest. Set MaxCpuCount under RunConfiguration in a .runsettings file, or pass it inline through dotnet test as -- RunConfiguration.MaxCpuCount=N. Zero means every available core. Solutions that have moved to Microsoft Testing Platform configure this level through that platform's own options rather than through VSTest runsettings.

3. Make the suite parallel-safe before you widen it. Give each test class its own temporary directory, bind to port zero and read back the assigned port, and give each worker its own database name or schema. Until that work is done, raising either level converts a slow suite into a flaky one.

4. Match the two levels to the vCPU count of the label. Total concurrent test threads should land near the core count. The product of MaxCpuCount and the framework thread setting is the number to control, since each test host process runs its own in-assembly parallelism.

5. Shard across runners only after both levels are set. Sharding hides a serialized test host behind more machines and bills you for the disguise. Once the settings are right, sharding is a wall-clock decision covered in sharding a test suite across GitHub Actions jobs.

Configuration

The two levels and what controls each

Settings by Windows label size

Windows labels run from 4 to 32 vCPU. These starting points keep the product of the two levels near the core count, leaving headroom for the operating system, Microsoft Defender, and the runner agent.

Rates and shapes come from the WarpBuild cloud runners documentation, checked on 2026-08-13. The full Windows catalog with image families, aliases, and regions lives on WarpBuild Windows runners for GitHub Actions.

Configuration files

// xunit.runner.json, committed next to each test project
{
  "$schema": "https://xunit.net/schema/current/xunit.runner.schema.json",
  "parallelizeAssembly": false,
  "parallelizeTestCollections": true,
  "maxParallelThreads": 4
}

<!-- ci.runsettings, committed at the repository root -->
<RunSettings>
  <RunConfiguration>
    <MaxCpuCount>4</MaxCpuCount>
    <ResultsDirectory>TestResults</ResultsDirectory>
  </RunConfiguration>
</RunSettings>

The workflow, one large runner

MaxCpuCount is passed on the command line here so the value follows the label size through one matrix input rather than living in a file that every label shares.

name: dotnet-tests

on:
  pull_request:
  push:
    branches: [main]

env:
  DOTNET_NOLOGO: "true"
  DOTNET_CLI_TELEMETRY_OPTOUT: "true"
  MSBUILDDISABLENODEREUSE: "1"
  NUGET_PACKAGES: C:\nuget

jobs:
  test:
    runs-on: warp-windows-latest-x64-16x
    timeout-minutes: 30
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 1

      - uses: actions/setup-dotnet@v4
        with:
          dotnet-version: "9.0.x"

      - uses: actions/cache@v4
        with:
          path: C:\nuget
          key: nuget-windows-${{ hashFiles('**/packages.lock.json') }}
          restore-keys: |
            nuget-windows-

      - name: Build
        run: >
          dotnet build Contoso.sln
          --configuration Release
          -maxcpucount

      - name: Test with both parallelism levels on
        env:
          XUNIT_MAX_PARALLEL_THREADS: "4"
        run: >
          dotnet test Contoso.sln
          --configuration Release
          --no-build
          --settings ci.runsettings
          --logger trx
          -- RunConfiguration.MaxCpuCount=4

The workflow, four small runners

The same suite split four ways. Each shard rents 4 vCPU, so the per-host settings drop to match.

jobs:
  test:
    runs-on: warp-windows-latest-x64-4x
    strategy:
      fail-fast: false
      matrix:
        shard: [1, 2, 3, 4]
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 1
      - uses: actions/setup-dotnet@v4
        with:
          dotnet-version: "9.0.x"
      - name: Test shard ${{ matrix.shard }}
        run: >
          dotnet test Contoso.sln
          --configuration Release
          --filter "Shard=${{ matrix.shard }}"
          --settings ci.runsettings
          --logger trx
          -- RunConfiguration.MaxCpuCount=2

Cost or Time Model

WarpBuild Windows rates are linear in vCPU: every Windows vCPU-minute lists at $0.004, so the two layouts rent identical capacity at an identical price.

warp-windows-latest-x64-16x (16 vCPU, 64 GB) costs $0.064 per minute against $0.082 per minute for the 16-core Windows larger runner at the same 16 vCPU and 64 GB shape, a 22 percent lower list price, and warp-windows-latest-x64-4x (4 vCPU, 16 GB) costs $0.016 per minute against $0.022 per minute for the 4-core Windows larger runner at the same 4 vCPU and 16 GB shape, a 27 percent lower list price (GitHub billing reference, checked on 2026-08-13).

The bottom two rows carry a second consequence. GitHub's larger-runner list prices are not linear in vCPU, so the sharded layout there rents the same 16 cores for $0.088 per minute against $0.082, and the layout choice moves the bill. On WarpBuild the layout is neutral on cost, which leaves the decision where it belongs: which arrangement keeps more cores busy.

A worked model

The inputs below are assumptions to replace with numbers from your own run history. Nothing here is a measured result.

• A suite of 12 test assemblies, 4 minutes of sequential work each, 48 minutes in total.
• Fixed per-job overhead of 4 minutes for boot, checkout, SDK setup, cache restore, and build.
• Billing at one-minute granularity, which is how WarpBuild bills GitHub Actions runners.
• 600 test runs per month.

Three readings come out of that table. Fixing the settings on the machine you already rent takes the monthly line from $1,996.80 to $307.20, which is the largest single move available. Sharding an unfixed suite takes it to $614.40, so it buys wall clock while still paying for idle cores. And once both levels are set, the two layouts land on the same cost and the same wall clock, because per-job overhead scales with the cores rented and cancels out at a flat $0.004 per vCPU-minute.

The tie breaks on details the model leaves out. Four shards restore the NuGet cache four times, and on a 4 vCPU machine that restore runs slower than it does on 16 vCPU, so overhead per shard grows. One large runner keeps one warm process tree and one cache restore, and it fails as a single unit when a shard would have isolated the failure. Shards also let a suite whose longest assembly dominates finish in the time of the longest shard rather than the longest assembly.

Current rates for every label are on the pricing page.

The .NET toolchain decisions around this test step, including image family and NuGet handling, are covered in running .NET builds on GitHub Actions, and the short version of the parallelism question is answered in can I run unit tests in parallel on Windows runners.

FAQ

Why does dotnet test still run one assembly at a time on a 16 vCPU Windows runner?

Because the VSTest setting that controls assembly-level parallelism defaults to one process. Pass RunConfiguration.MaxCpuCount=0 on the command line, or set MaxCpuCount in a .runsettings file and point dotnet test at it with --settings. Confirm the file is actually loaded, since a runsettings file sitting in the repository root is ignored unless the workflow passes it.

Should I run one 16 vCPU Windows runner or four 4 vCPU runners?

WarpBuild Windows rates are linear at $0.004 per vCPU-minute, so warp-windows-latest-x64-16x at $0.064 per minute and four warp-windows-latest-x64-4x runners at $0.016 each cost the same per minute for the same 16 vCPU. Choose on whether one test host layout keeps 16 cores busy. If a single machine plateaus below four times what one 4 vCPU shard achieves, shard.

How do I tell whether the runner is sitting idle during the test step?

Read the CPU P75 and P90 columns on the Jobs section of the WarpBuild Reports page. One busy core on a 16 vCPU runner reports around 6 percent peak CPU, and one busy core on a 4 vCPU runner reports around 25 percent. Peak CPU well under the core count during the test step means the parallelism settings are capping the run rather than the machine size.