Alerting on GitHub Actions Build Time Regressions

GitHub Actions has no alert for a build that got slower, because the run log carries one duration per run and never compares that run against the ones before it. The alert has to be assembled from aggregated percentiles: baseline duration P90 per job over a week in the WarpBuild Jobs report, store one threshold per job, then run a scheduled workflow that reads the same report and fails when a job crosses its line.

This guide covers what counts as a regression worth waking someone for, the four resource threshold alerts that ship with the observability feature and what they actually cover, a threshold-setting method built on a baseline week of percentile data, the runbook that says which report to open first for each alert, and what the watcher costs to run.

Diagnosis

A regression is a shift in the distribution of a job's runs. A single 24 minute run of a job that normally finishes in 9 minutes proves that one machine had a bad afternoon. The same job holding a duration P90 of 12.5 minutes across a week is the thing that costs the team hours, and it is invisible to any alert that watches individual runs.

That is why per-run alerting produces noise on both ends. It pages on the outlier and stays silent through the slow slide, which is the failure mode of every threshold set against a single run.

GitHub gives you the raw material and no aggregation. The workflow runs REST API returns timestamps and conclusions per run, so building percentiles from it means paginating history, joining runs to jobs, and storing the result somewhere. The WarpBuild Jobs report aggregates per unique repository, workflow, and job name combination and hands back run count, success rate, duration P75 and P90, queue time P75 and P90, and CPU and memory percentiles when observability is enabled for the runner.

Four resource threshold alerts ship separately from the duration numbers, in the Recommendations view of the observability feature.

Read what those four cover before you rely on them. They are a filter over the fleet that highlights under-provisioned and over-provisioned instances, applied to the organization's last 7 days, and each flagged job carries a recommendation type of upgrade or downgrade, the current runner label, a recommended label, and the subset of resources that triggered it. They answer the question of whether the machine is the wrong size. They say nothing about a job that got slower while sitting at 40 percent CPU the whole time, which is the common shape of a build time regression caused by more tests, a bigger dependency graph, or a cache that stopped hitting. Duration alerting is your job to wire, and the report data for it already exists.

Two collection limits shape what you will see. Observability collects metrics and logs only for jobs longer than about one minute, so short jobs show a dash in the CPU and memory columns, and collection can be paused, which stops system logs and GitHub Actions logs along with the metrics.

Fix

Step 1: take a baseline week. Pull the Jobs report for the last seven days and record duration P75, duration P90, and run count per job. The threshold rule that survives contact with a real repository is the baseline P90 plus the larger of 2.0 minutes or 20 percent of that baseline. The floor stops short jobs from paging on a 30 second wobble, and the proportional part keeps long jobs from paging on normal variance.

The table below is illustrative rather than measured, and it shows the four cases you will meet in a real baseline.

api-tests takes 9.4 plus 2.0, because 20 percent of 9.4 is smaller than the floor. e2e takes 33.5 plus 6.7, because at that length the proportional term wins. release runs nine times in the week, so its P90 is effectively one run and any threshold set on it will fire on the next flaky afternoon. Jobs under roughly 30 runs a week belong in a weekly review.

Step 2: keep P75 in the alert payload. The pair is the diagnosis. P75 and P90 climbing together means the job got heavier for every run. P75 flat with P90 climbing means a tail is forming on a minority of runs, which points at retries, cold caches, or contention rather than at the code.

Step 3: run the runbook. Each alert type has one report that answers it fastest.

Step 4: re-baseline after every accepted change. A deliberate change that adds two minutes to a build has to move the threshold, otherwise the alert becomes a thing people mute. Re-read the week after the change lands and write the new number back.

Configuration

The watcher is one scheduled workflow. It reads the Jobs report through the API with a key carrying the ci scope, compares each job against a thresholds file checked into the repository, and fails when a job is over its line.

name: build-time-regression-watch
on:
  schedule:
    - cron: "0 14 * * *"
  workflow_dispatch:

jobs:
  check-duration:
    runs-on: warp-ubuntu-latest-x64-2x
    steps:
      - uses: actions/checkout@v4

      - name: Pull the last 7 days of job percentiles
        env:
          WARPBUILD_API_KEY: ${{ secrets.WARPBUILD_API_KEY }}
        run: |
          curl -sS -G 'https://api.warpbuild.com/api/v1/reports/jobs' \
            -H "Authorization: Bearer $WARPBUILD_API_KEY" \
            -H 'Accept: application/json' \
            --data-urlencode "start_date=$(date -u -d '7 days ago' +%Y-%m-%dT%H:%M:%SZ)" \
            --data-urlencode "end_date=$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
            --data-urlencode 'chart_metric=duration' \
            --data-urlencode 'chart_percentile=p90' \
            --data-urlencode 'sort_by=duration_p90' \
            --data-urlencode 'sort_order=desc' \
            --data-urlencode 'per_page=100' \
            > jobs.json

      - name: Compare against the baseline thresholds
        run: |
          jq -r --slurpfile t .github/ci-thresholds.json '
            .table.items[]
            | . as $row
            | ($t[0][$row.job_name] // empty) as $limit
            | select($row.duration_p90 > $limit)
            | "\($row.repo) \($row.workflow_name) \($row.job_name): p90 \($row.duration_p90) over \($limit) (p75 \($row.duration_p75), runs \($row.run_count))"
          ' jobs.json | tee regressions.txt
          test ! -s regressions.txt

.github/ci-thresholds.json holds one number per job name and is reviewed like any other config. Keep the thresholds in the same unit the report returns for duration_p90 so the comparison stays a plain numeric test.

{
  "api-tests": 11.4,
  "web-build": 8.0,
  "e2e": 40.2
}

Queue time needs no second call, because queue_time_p75 and queue_time_p90 sit on the same row. Add a second select on $row.queue_time_p90 with its own limits and the one workflow covers execution drift and wait drift together. Replace the final test with a Slack step when you want a message instead of a red check.

The resource threshold alerts come from a different endpoint, /api/v1/org_metrics/job_runner_recommendations, which returns one entry per job that should move plus the resources that triggered it. Running that weekly next to the daily duration check gives you both halves: whether the machine is wrong, and whether the work got slower. Adding --data-urlencode 'format=csv' to the reports call returns the same rows as CSV, matching every row under the current filters instead of the visible page.

Two configuration notes. CPU and memory columns need observability enabled for the runner, and on BYOC instances the agent needs egress on port 33931, otherwise those columns stay empty while duration and queue time keep reporting normally.

Cost or Time Model

The watcher runs on the smallest Linux size in the catalog. Rates below come from the WarpBuild pricing page, billed per minute, checked on 2026-08-13.

A daily run that finishes inside one billed minute is 30 minutes a month at $0.004, or $0.12. Adding the weekly recommendations check at one minute each brings the total to about $0.14 a month. The watcher costs less than a single run of the job it watches.

The number that matters is on the other side. Take web-build from the baseline table above at 500 runs a month, sitting on warp-ubuntu-latest-x64-4x at $0.008 per minute, with duration P90 moving from 9.0 to 12.5 minutes after a dependency change. Left alone for three weeks, that is roughly 345 runs carrying 3.5 extra minutes each: 1,207 minutes of added wall clock, which is 20 hours of people waiting on pull requests, and $9.66 of added spend. Caught on the next day's check, the same regression costs about 16 runs, 56 minutes, and $0.45 before someone looks at it. The spend difference is small and the wait difference is the reason to wire the alert.

A second model for the queue side. If queue_time_p90 for one runner label moves from 20 seconds to 4 minutes across 2,000 jobs a week, that is 7,600 minutes of added wait per week, absent from the duration percentiles and absent from the bill, because queue time is billed to nobody. The Queue Timings report is where that shows up, split per runner label and stack.

Structural notes for anyone budgeting this. Pricing is purely usage based. There is no base subscription fee, no platform fee, and no seat fee, so the observability data behind these alerts adds no line item of its own beyond the minutes the watcher consumes. Signup includes $10 free credits, which is enough to baseline a week of real jobs before you commit to thresholds. The same reports cover every one of them, including BYOC instances in your own cloud account. The alerting described here reads from those observability reports.

For the full report and API surface, start at GitHub Actions observability with WarpBuild. If the thresholds themselves are the open question, build time budgets covers turning them into a target the team agrees on. When the alert that fires is a memory one, can I alert when a runner runs out of memory covers that signal, and fix GitHub Actions jobs that run out of memory covers the repair.

FAQ

How do I alert when a GitHub Actions build gets slower?

Baseline duration P90 per job over a week in the Jobs report, store one threshold per job, then run a scheduled workflow that reads the same report through the API and fails when a job crosses its threshold. GitHub Actions reports one duration per run and never compares it against earlier runs, so the comparison has to happen outside the run log.

What threshold should I set for a build time alert?

Set it from a baseline week rather than from a guess. Take the job's duration P90 over seven days and add the larger of 2.0 minutes or 20 percent of that baseline. Jobs with fewer than about 30 runs in the week get a weekly review instead, because their P90 is one or two runs and moves on noise.

Which report do I open first when a build time alert fires?

The Jobs report, filtered to the job that fired, with duration P75 read next to duration P90. If queue time P90 moved instead, the Queue Timings report breaks the wait down per runner label and stack. Resource threshold alerts point at the Recommendations view instead, where the flagged instance carries its current label and a recommended label.