Tracking Build Duration Percentiles on GitHub Actions

GitHub Actions records a duration for each job run and stops there, so a percentile view of a workflow has to be assembled somewhere outside the run log. The WarpBuild Jobs report assembles it for you: every unique repository, workflow, and job name combination carries run count, success rate, duration P75 and P90, and queue time P75 and P90, with CPU and memory percentiles added when Observability is enabled.

This guide covers why the average build time hides the problem you are chasing, what a widening gap between duration P75 and P90 tells you about capacity and flakiness, the exact path to those numbers in the Jobs report, a repeatable way to track them week over week through CSV export, and a time model that converts a P90 reduction into engineer wait time at a stated job volume.

Diagnosis

The average build time is the number most teams put on a dashboard, and it is the number least likely to move when something breaks.

Take a test job with 400 runs a month. Suppose 360 of those runs finish in 8 minutes and 40 of them finish in 24 minutes because they retry a flaky integration step. The mean is 9.6 minutes. Fix nothing, let the flaky step degrade until those 40 runs take 27 minutes, and the mean moves to 9.9 minutes. A 3 minute regression on the runs that hurt the most shows up as an 18 second wiggle on the chart that leadership reads.

Percentiles keep the tail attached to the number. Duration P90 in that same example moves from 24 to 27 minutes, which is the size of the regression, reported at the size it actually is.

The Jobs report exposes two duration percentiles, P75 and P90, and the relationship between them is the diagnostic. P75 describes the run a developer expects. P90 describes the run that makes someone open Slack and ask whether the build is stuck. Read the gap.

A widening P75 to P90 gap has two common causes, and they are separable with one more column each.

Capacity contention. Waiting for a runner is counted separately from running on one. When queue time P90 climbs while duration P90 stays put, the job held its speed and the fleet ran out of room. Queue Timings reports P75 and P90 wait per runner label and stack, so a single label saturating shows up in isolation rather than smeared across the whole estate. On WarpBuild runners this shape has a different cause than a fixed cap: run as many jobs as your workflows need, since generally available Linux and Windows runners do not have plan-level concurrency caps.

Flakiness and retries. Success rate sits on the same row as the duration percentiles for a reason. A job at 94 percent success with duration P90 well above P75 is describing runs that failed partway, retried, and paid for the work twice. Averages absorb that. P90 does not.

One more property matters when you start tracking these numbers seriously. Percentiles do not add up. The duration P90 of a workflow is not the sum of the duration P90 of its jobs, because the slowest 10 percent of one job and the slowest 10 percent of another are usually different runs. Track percentiles per job, and use end to end run time separately when you want the pipeline number.

Fix

The measurement path is short. Everything below lives in the Reports documentation and the Observability documentation.

Step 1: open the Jobs report and set the window. The Jobs section aggregates per unique repository, workflow, and job name combination. Each row is one job across all of its runs in the selected period, and the window is the first thing to fix, because a 7 day window and a 30 day window produce different percentiles from the same data.

Step 2: read the row. These are the columns available per row.

CPU and memory require Observability to be enabled for the runner, and Observability collects metrics only for jobs longer than about one minute. Jobs without telemetry render a dash in those two pairs of columns and still report everything else.

Step 3: sort by duration P90 descending and filter by run count. A job with 4 runs in the window has a P90 that means nothing. Sorting by the tail and reading run count next to it is how you separate a real regression from an artifact of a job that barely ran.

Step 4: put the number on a line. The time-series chart plots one selected metric at one selected percentile over time, with a line per job from the current table page. Set the metric to duration and the percentile to P90, keep the same filters, and a regression is a line that bends. Filters available on the section are repository, workflow, job name, runner label, and stack.

Step 5: cross-check the machine. If duration P90 is climbing and CPU P90 is sitting near the ceiling, the job is core bound and the size is the lever. If duration P90 is climbing while CPU P90 sits in the low tens, the time is going somewhere other than compute, and the slow step walkthrough is the faster path than buying cores.

BYOC runs on AWS, GCP, and Azure, so a job running on a runner inside your own cloud account reports the same percentiles in the same table as a hosted one.

Configuration

Percentile tracking depends on stable aggregation keys. The key is repository, workflow, and job name, so a renamed job starts a new row and loses its history. Name jobs for what they do and leave the names alone.

name: ci
on:
  pull_request:

jobs:
  lint:
    runs-on: warp-ubuntu-latest-x64-2x
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
      - run: npm ci
      - run: npm run lint

  unit-tests:
    runs-on: warp-ubuntu-latest-x64-4x
    strategy:
      fail-fast: false
      matrix:
        shard: [1, 2, 3, 4]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
      - run: npm ci
      - run: npm test -- --shard=${{ matrix.shard }}/4

  integration-tests:
    needs: unit-tests
    runs-on: warp-ubuntu-latest-x64-8x
    steps:
      - uses: actions/checkout@v4
      - run: make integration

Three stable job names, three sizes from the catalog. Filtering the Jobs report by the warp-ubuntu-latest-x64-8x runner label then isolates every job on that size across every repository, which is the view to use when you want to know whether a size is carrying more than it should.

Tracking the numbers week over week

The dashboard answers what happened. A file on disk answers what changed. Every report tab exports CSV, and the export contains all rows matching the current filters and sort order rather than the visible page, which is what makes the export comparable between weeks.

The procedure is four fixed choices and one habit.

1. Fix the window. Use a rolling 7 day window, taken on the same weekday every time.
2. Fix the filters. One repository, or one runner label, whichever unit you plan to defend in a review.
3. Fix the sort. Duration P90 descending, so the file leads with the tail.
4. Export and keep the file, named by the date it was taken.

# each Monday, same window, same filters, same sort
mkdir -p metrics && mv ~/Downloads/jobs.csv metrics/jobs-2026-08-13.csv

# compare duration P90 between two weekly exports, keyed on repo + workflow + job
# set COL to the duration P90 column index in your export
COL=6
awk -F, -v col="$COL" 'NR>1 {print $1"|"$2"|"$3"\t"$col}' metrics/jobs-2026-08-06.csv | sort > /tmp/prev
awk -F, -v col="$COL" 'NR>1 {print $1"|"$2"|"$3"\t"$col}' metrics/jobs-2026-08-13.csv | sort > /tmp/curr
join -t $'\t' /tmp/prev /tmp/curr | awk -F'\t' '{d=$3-$2; if (d>1) printf "%s  +%.1f min\n", $1, d}'

Anything printing a positive delta above a minute is a job whose tail grew in a week. That list is short, it is specific, and it names the repository, workflow, and job to open.

The metrics API guide covers the endpoints, the query parameters, and how to schedule the pull instead of clicking the download button.

For row-level data behind the percentiles, the CI billing tab in Reports is the source: one row per job execution with repository, job name, runner label, stack, execution time, and billed time. Use billed time when you reconcile against an invoice, and execution time when you are studying duration.

Cost or Time Model

A percentile improvement is a wait-time result before it is a billing result, so model the time first.

Assumptions. One integration-tests job, 1,200 runs a month, currently on warp-ubuntu-latest-x64-4x. Duration P75 is 16.0 minutes and duration P90 is 22.0 minutes, a 6.0 minute gap. After moving the job to warp-ubuntu-latest-x64-8x, P75 reads 12.0 minutes and P90 reads 13.0 minutes. Rates are the published per minute rates below. Queue time is excluded, because queued minutes are not billed.

The tail-only bound. P90 describes the slowest 10 percent of runs. At 1,200 runs a month that is 120 runs, each 9.0 minutes shorter: 1,080 minutes, or 18.0 hours a month of engineer wait removed from the runs that were hurting most.

The whole-distribution bound. P75 also moved, by 4.0 minutes. If every run improves by at least that much, 1,200 runs times 4.0 minutes is 4,800 minutes, or 80.0 hours a month.

The real figure sits between 18.0 and 80.0 hours. Settle it by re-exporting after four weeks and reading the two percentiles again rather than by arguing about the model.

Now the bill. Percentiles do not multiply into an invoice, so the cost side needs a mean. Assume the mean duration is 14.0 minutes on the 4x size and 9.0 minutes on the 8x size. Linux x64 rates from the WarpBuild pricing page, billed per minute:

Current spend is 1,200 x 14.0 = 16,800 minutes at $0.008, which is $134.40 a month. After the move, 1,200 x 9.0 = 10,800 minutes at $0.016, which is $172.80 a month. The change costs $38.40 a month and returns somewhere between 18.0 and 80.0 hours of waiting. The break-even mean on the 8x size is 7.0 minutes: below that, the upgrade lowers the bill as well as the clock.

For a list-price baseline on the same shape, GitHub publishes $0.022 per minute for its 8-core Linux larger runner at 8 vCPU and 32 GB in the GitHub billing reference, checked on 2026-08-13. The same 10,800 minutes at that rate is $237.60 a month, against $172.80 on warp-ubuntu-latest-x64-8x at $0.016 per minute for 8 vCPU and 32 GB.

If the number that grew is queue time rather than duration, cut GitHub Actions queue times works that path. For the estate-wide view of what gets reported across repositories, start at GitHub Actions observability with WarpBuild.

FAQ

Which duration percentiles does the WarpBuild Jobs report expose?

Duration P75 and P90, reported per unique repository, workflow, and job name combination, alongside queue time P75 and P90. The time-series chart plots either percentile for the jobs on the current table page, so a regression shows up as a line bending upward.

Why is average build time a bad target to optimize against?

The mean folds the slow runs into the fast ones and moves slowly. A job with 400 runs a month can add three minutes to its slowest 40 runs and shift the mean by 18 seconds. Percentiles keep the tail visible, which is where retries, cold caches, and queue contention live.

What does a widening gap between duration P75 and P90 mean?

A tail is forming while the typical run holds steady. Check the queue time P75 and P90 in the same row to separate waiting from running, then check success rate, because retried and flaky runs land in the tail before they land in the mean.

Why do the CPU and memory columns show a dash for some jobs?

Those columns need Observability enabled for the runner, and Observability collects metrics only for jobs longer than about one minute. Rows without telemetry still report run count, success rate, duration percentiles, and queue time percentiles.