Sizing Runners for Python Test Suites

The right runner size for a Python test suite is the smallest label whose vCPU count matches the number of pytest workers the suite can actually keep busy, and that number comes from two limits rather than from the core count. The limits are memory per worker and the capacity of any service the workers share, such as one Postgres or Redis container inside the job.

This guide covers how to read both limits from a job, a worker-count table for process-parallel runs against the Linux x64 ladder, a workflow that derives the worker count from the label and pins the service container size next to it, and a cost model that prices four sizes for the same suite. For the fan-out question of one big machine against several small ones see the guide to sharding pytest suites, and for the health-check mechanics around services see the guide to service containers.

Diagnosis

Worker processes are what turn cores into shorter wall clock. pytest -n 8 starts eight separate processes, so the interpreter lock does not cap them, and the machine size only matters through what it lets those eight processes do. Each worker imports the test modules on startup, holds its own fixtures, and opens its own connections to whatever the suite talks to.

That gives two ceilings, and the lower one decides the size.

Memory per worker. Every Linux x64 size carries 4 GB of RAM per vCPU, from 2 vCPU with 8 GB to 32 vCPU with 128 GB, all on 150GB SSD storage (cloud runners documentation, checked on 2026-08-13). A suite that holds 2 GB per worker fills every core at every size. A suite that holds 6 GB per worker, which happens with large fixtures, in-memory dataframes, or a machine-learning model loaded per worker, leaves half the cores idle on any label and gets workers killed if you ask for one per core.

Shared service capacity. When the workers all talk to one Postgres container, one Redis container, or one external staging API, the suite scales with worker count only until that service becomes the queue. The service runs inside the same job on the same machine, so it also takes cores and memory away from the workers.

Measure both before picking a label. Run the suite once at a low worker count and once at a higher one, then read the numbers.

The Observability recommendations view aggregates metrics by repository, workflow, job, and instance type, and reports Maximum CPU Utilization, Maximum Memory Utilization, Maximum Filesystem Utilization, and Maximum Disk I/O per runner instance. It flags an instance as under-provisioned once sustained CPU, memory, or filesystem utilization reaches 80 percent, or once disk I/O reaches 80 percent of supported throughput (observability documentation). Memory per worker is the peak memory minus the idle baseline, divided by the worker count that produced it.

The last row is worth a pass on its own. A job sized for a peak that no longer exists bills the larger rate on every run.

Fix

Work the two limits in order, then pick the label.

Measure memory per worker at a low count. Run the suite at -n 4 on warp-ubuntu-latest-x64-4x and read Memory Utilization from the recommendations view. Peak minus baseline, divided by 4, is the per-worker figure that everything below depends on.

Find the service ceiling. Raise the worker count in steps and record wall clock. The point where wall clock stops falling while CPU and memory stay under 80 percent is the service ceiling, expressed as a worker count. A default Postgres container answers a fixed max_connections, so a suite opening two connections per worker hits that wall earlier than the core count suggests.

Give each worker its own state before adding any. xdist sets PYTEST_XDIST_WORKER to gw0, gw1, and so on. Derive the database name, the temp directory, and any bound port from it in a session fixture. Workers that share one database schema serialize on locks, which looks exactly like a service ceiling and is cheaper to fix.

Reserve capacity for the service, then read the table. Budget the service container first, hold back about 0.5 GB for the runner agent and the Docker daemon, and divide what is left by the per-worker memory. The worker count is the lower of that result and the vCPU count minus the cores the service needs.

Worker counts by label and per-worker memory, with 0.5 GB held back for the runner agent and no service container in the job. Rates and shapes come from the cloud runners documentation, checked on 2026-08-13.

Read a row as the lower of two numbers: the vCPU count, and (RAM - 0.5) / memory per worker rounded down. At 1 GB and 2 GB per worker the core count binds, so the label alone sets the worker count. At 4 GB the two numbers nearly meet, which is what the 4 GB per vCPU ratio means. At 6 GB memory binds everywhere, and a third of the cores on the largest label stay idle.

Subtract the service from the same budget. A Postgres container pinned to 1 core and 4 GB removes one from the vCPU column and 4 from the RAM column before the division, which is how the configuration below arrives at 6 workers on the 8 vCPU label.

Pick the smallest label that reaches your worker ceiling. Above that point the rate rises and the worker count does not, so the run costs more and finishes at the same time. The same arithmetic transfers to the other platforms with their own shapes.

Configuration

This workflow carries the sizing decision in one place. The matrix rows pair a label with the worker count and the service container size that fit it, so a size experiment is a row edit and the production config is the row you keep.

name: pytest
on:
  pull_request:
  workflow_dispatch:

jobs:
  test:
    name: pytest ${{ matrix.label }} n=${{ matrix.workers }}
    runs-on: ${{ matrix.label }}
    strategy:
      fail-fast: false
      matrix:
        include:
          - label: warp-ubuntu-latest-x64-4x
            workers: 2
            db_cpus: "1"
            db_memory: 4g
          - label: warp-ubuntu-latest-x64-8x
            workers: 6
            db_cpus: "1"
            db_memory: 4g
          - label: warp-ubuntu-latest-x64-16x
            workers: 13
            db_cpus: "2"
            db_memory: 8g
    services:
      postgres:
        image: postgres:17
        env:
          POSTGRES_PASSWORD: app
          POSTGRES_DB: app_test
        ports:
          - 5432:5432
        options: >-
          --cpus ${{ matrix.db_cpus }}
          --memory ${{ matrix.db_memory }}
          --health-cmd "pg_isready -h 127.0.0.1 -U postgres -d app_test"
          --health-interval 5s
          --health-timeout 5s
          --health-retries 12
          --health-start-period 20s
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.13"
          cache: pip
      - run: pip install -r requirements.txt -r requirements-dev.txt
      - name: Run suite
        env:
          DATABASE_URL: postgresql://postgres:[email protected]:5432/app_test
          PYTEST_WORKERS: ${{ matrix.workers }}
        run: pytest -n "$PYTEST_WORKERS" --dist loadfile -q

What each piece does:

• matrix.workers is the number from the table above rather than auto, so the worker count stays tied to the label and to the measured per-worker memory. auto reads the CPU count and would ask for 16 workers on the last row, which the 4 GB per worker budget does not hold once Postgres takes 8 GB.
• --cpus and --memory in options reach docker create, so the service gets a fixed slice instead of competing with the workers for the whole machine. Pinning it also makes the worker arithmetic stable: what the container cannot take, the workers can have.
• The health check gates the first step on a TCP connection to the real server. --health-start-period covers the initialization window, which is where a suite otherwise fails with a connection refused on an apparently healthy service.
• --dist loadfile keeps each module on one worker, so module scoped fixtures set up once per worker rather than once per test group. Fewer setups means less memory per worker and a higher worker count at the same label.
• fail-fast: false keeps every row running while you are measuring, so one bad size does not cancel the comparison.

Run the matrix on a workflow_dispatch for a day, read wall clock and Memory Utilization per row, then delete the rows you are not keeping. Runner storage is ephemeral and is deleted when the runner terminates, so nothing carries between rows and each is a clean measurement.

Cost or Time Model

Take a suite with 50 minutes of parallelizable test execution, 3 minutes of setup for checkout, setup-python against a warm pip cache, and the service health wait. Per-worker memory measures at 4 GB. One Postgres container takes 1 core and 4 GB, the runner agent takes about 0.5 GB, and the database saturates at 8 concurrent workers. Each job bills to the next whole minute.

Three readings come out of that table. The 8 vCPU size is the cheapest per run and finishes 16.7 minutes sooner than the 4 vCPU size, because the worker count rises three times while the rate doubles. The 16 vCPU size buys 2 more minutes for 67 percent more per run, since the database ceiling caps the useful workers at 8 whatever the label holds. The 32 vCPU size buys nothing at all and doubles the bill, which is the shape of every size decision made without measuring the shared service first.

Raising the database ceiling changes the answer rather than the label. Give Postgres 2 cores and 8 GB, tune the connection pool, and the 16 vCPU row runs the 13 workers its memory budget holds at 3.8 minutes of test time, so wall clock lands near 6.8 minutes and 7 billed minutes at $0.224. That is the same cost as the 4 vCPU row for a quarter of the wall clock, and it only becomes available once the service stops being the ceiling.

Scale the winning row to a working month at 40 runs a day over 22 days, which is 880 runs. The 8 vCPU row costs $168.96 a month. The 16 vCPU row costs $281.60 for 1,760 fewer minutes of waiting, about 29 hours spread across the team's pull requests. Both numbers are the per-minute rate times billed minutes times run count, so rerun them with your own suite time.

The same shapes on GitHub-hosted larger runners cost more per minute. warp-ubuntu-latest-x64-4x (4 vCPU, 16 GB) costs $0.008 per minute against $0.012 per minute for the 4-core Linux larger runner (4 vCPU, 16 GB): 33 percent lower list price. warp-ubuntu-latest-x64-16x (16 vCPU, 64 GB) costs $0.032 per minute against $0.042 per minute for the 16-core Linux larger runner (16 vCPU, 64 GB): 24 percent lower list price. GitHub list prices checked on 2026-08-13 against the GitHub Actions billing reference.

Every rate above is on the pricing page.

FAQ

What size runner should a Python test suite use?

Take the smallest label whose vCPU count matches the worker count you can actually feed. Two numbers set that count: memory per worker and the concurrency the shared service tolerates. A suite at 2 GB per worker with no database fills every core up to 32 on the Linux x64 ladder, while a suite at 4 GB per worker sharing one Postgres container that saturates at 8 connections stops paying for cores above the 8 vCPU size.

Should I run pytest with -n auto on a larger runner?

Use -n auto only when memory per worker is at or below 4 GB and nothing shared throttles the run, because auto reads the CPU count of the machine and asks for one worker per vCPU. Every Linux x64 size carries 4 GB per vCPU, so a suite needing more than that per worker gets workers killed at the top of the ladder. Set an explicit -n number derived from the sizing table instead.

How do I tell whether memory or the database is capping my suite?

Read the two signals separately. Memory Utilization at or above 80 percent on the Observability recommendations view, or a job exiting 137, means memory per worker is the limit. Wall clock that stops falling as the worker count rises while memory stays low means the shared service is the limit, and the fix is service capacity rather than a bigger runner.