Yocto and Embedded Builds on GitHub Actions

A Yocto build on GitHub Actions is bound by three directories that a fresh runner throws away every time: the sstate-cache, the downloads directory, and the local build tree. Run bitbake on warp-ubuntu-latest-x64-32x with a snapshot runner and all three are already on disk when the job starts, so the second build restores most tasks from shared state instead of rebuilding a cross toolchain from source.

The switch is two changes. Point runs-on at a warp- label carrying a snapshot key, and move DL_DIR, SSTATE_DIR, and TMPDIR to a path outside /tmp so the snapshot captures them. The sections below give the working workflow, the sizing arithmetic against the 150GB disk, and the monthly cost model.

Overview

A bitbake run for an ordinary embedded image walks several thousand tasks across several hundred recipes. It fetches upstream source, configures and compiles a cross toolchain, builds a kernel and a userspace, then packages and assembles a root filesystem image. A cold run does all of that work. A warm run does very little of it, because bitbake hashes the inputs of every task and restores the output from the shared state cache when the hash already exists there.

That restore path is the whole game. The sstate-cache is what turns a multi-hour image build into a short one, and it lives on disk in SSTATE_DIR. Alongside it sits DL_DIR, the download cache holding upstream tarballs and git mirrors that bitbake would otherwise refetch from dozens of independent upstream hosts.

GitHub-hosted runners work against both. The standard hosted Linux runner gives a private repository 2 vCPU and 8GB of memory, which is well under what a parallel bitbake run can use, and every job starts on a machine with an empty disk.

Yocto work belongs on the Linux x64 line, where the runner catalog lists sizes from 2 to 32 vCPUs, each with a 150GB SSD, on Ubuntu 22.04, 24.04, and 26.04 images that carry the same tooling as GitHub-hosted runners. The host packages bitbake wants, gawk through zstd, install from apt exactly as they do on a developer workstation.

Runners are ephemeral VMs, allocated per job and destroyed afterward. Snapshot runners are the answer to that for Yocto: capture the VM disk mid-workflow and boot later jobs from that image. Snapshot runners are one part of a product surface that also includes remote Docker builders, CI observability, an MCP server, and the Action Debugger.

One boundary to know before you design around it. BYOC runs on AWS, GCP, and Azure, and snapshots are unsupported on all three. Snapshot labels apply to WarpBuild Cloud Ubuntu runners only.

Configuration

Why a snapshot beats a cache archive here

A warm sstate-cache for a single machine target holds hundreds of thousands of small files. A cache archive has to tar that tree, compress it, upload it, then download and untar it on the next run, and it repeats that round trip for the downloads directory as well. Small-file tar throughput, and not network bandwidth, is what sets the floor on that step.

A snapshot skips the round trip. The runner boots from the saved disk image in 45 to 60 seconds and the directories are simply there, along with the apt packages you installed, the poky checkout, and whatever survived in build/tmp. The label form is the runner label with the snapshot key appended after a semicolon:

warp-ubuntu-latest-x64-32x;snapshot.key=<alias>

snapshot.key=<alias> boots from the existing snapshot for that alias when one exists and falls back to the base image when it does not. snapshot.enabled=true turns the feature on and always boots from the base image, which is the label you want on the job that produces a clean snapshot. Both forms are documented on the snapshot runners page.

The workflow

name: yocto

on:
  push:
    branches: [main]
  pull_request:

env:
  BUILD_ROOT: /home/runner/yocto
  MACHINE: qemuarm64
  SNAPSHOT_ALIAS: yocto-qemuarm64

jobs:
  image:
    runs-on: >-
      ${{ github.ref == 'refs/heads/main'
        && 'warp-ubuntu-latest-x64-32x;snapshot.enabled=true'
        || 'warp-ubuntu-latest-x64-32x;snapshot.key=yocto-qemuarm64' }}
    timeout-minutes: 240
    steps:
      - uses: actions/checkout@v5
        with:
          path: layers/meta-acme

      - name: Install bitbake host dependencies
        run: |
          sudo apt-get update
          sudo apt-get install -y --no-install-recommends \
            gawk wget git diffstat unzip texinfo gcc build-essential \
            chrpath socat cpio python3 python3-pip python3-pexpect \
            xz-utils debianutils iputils-ping python3-git python3-jinja2 \
            python3-subunit zstd liblz4-tool file locales libacl1

      - name: Prepare persistent directories
        run: |
          mkdir -p "$BUILD_ROOT"/{downloads,sstate-cache,build}
          df -h /

      - name: Sync poky
        run: |
          if [ ! -d "$BUILD_ROOT/poky/.git" ]; then
            git clone -b scarthgap https://git.yoctoproject.org/poky "$BUILD_ROOT/poky"
          fi
          git -C "$BUILD_ROOT/poky" fetch origin scarthgap
          git -C "$BUILD_ROOT/poky" checkout FETCH_HEAD

      - name: Write build configuration
        run: |
          cd "$BUILD_ROOT/poky"
          source oe-init-build-env "$BUILD_ROOT/build" > /dev/null
          cat > conf/auto.conf <<EOF
          MACHINE = "${MACHINE}"
          DL_DIR = "${BUILD_ROOT}/downloads"
          SSTATE_DIR = "${BUILD_ROOT}/sstate-cache"
          TMPDIR = "${BUILD_ROOT}/build/tmp"
          BB_NUMBER_THREADS = "32"
          PARALLEL_MAKE = "-j 32"
          BB_GENERATE_MIRROR_TARBALLS = "1"
          INHERIT += "rm_work"
          EOF
          bitbake-layers show-layers | grep -q meta-acme \
            || bitbake-layers add-layer "$GITHUB_WORKSPACE/layers/meta-acme"

      - name: Build image
        run: |
          cd "$BUILD_ROOT/poky"
          source oe-init-build-env "$BUILD_ROOT/build" > /dev/null
          bitbake core-image-minimal

      - name: Prune before snapshot
        if: github.ref == 'refs/heads/main'
        run: |
          rm -rf "$BUILD_ROOT/build/tmp/work" "$BUILD_ROOT/build/tmp/work-shared"
          rm -rf "$HOME/.ssh" "$HOME/.aws"
          df -h /

      - name: Save snapshot
        if: github.ref == 'refs/heads/main'
        uses: WarpBuilds/snapshot-save@v1
        with:
          alias: "yocto-qemuarm64"
          fail-on-error: true
          wait-timeout-minutes: 60

Five details carry the design.

BUILD_ROOT lives outside /tmp. The /tmp directory is cleaned on reboot and a snapshot boot is a reboot, so anything bitbake wrote under /tmp is gone on the next run. Yocto's default TMPDIR sits inside the build directory rather than in /tmp, and the auto.conf line above pins it there explicitly so no local override moves it.

Configuration goes in conf/auto.conf, not local.conf. bitbake reads auto.conf automatically, and writing the file fresh each run keeps a snapshot-booted job from appending duplicate assignments to a local.conf that already survived from the previous build.

The main branch produces the snapshot, pull requests consume it. The runs-on expression sends main to snapshot.enabled=true, so the authoritative snapshot is always built from the base image, and sends every pull request to snapshot.key=yocto-qemuarm64.

snapshot-save takes three inputs. alias is required. fail-on-error defaults to true, and leaving it there means a snapshot that fails to capture fails the job instead of quietly leaving the next run cold. wait-timeout-minutes defaults to 30, and a Yocto disk of this size is worth raising to 60.

Snapshots expire after 15 days. A repository with steady main-branch traffic refreshes the alias long before that. A quiet fork will find the alias gone, in which case the job still runs and boots from the base image, paying the cold build back on that run.

Sizing

The Linux x64 catalog against GitHub's published list prices, with rates from the WarpBuild pricing page:

GitHub rates are from the GitHub Actions billing reference, checked on 2026-08-13. The ubuntu-latest row is the private repository shape; GitHub gives public repositories a 4 vCPU shape at no charge.

Take the 32 vCPU size for image builds. bitbake scales on two dials at once: BB_NUMBER_THREADS sets how many recipe tasks run concurrently, and PARALLEL_MAKE sets the make jobs inside each compile task. Setting both to 32 keeps the dependency graph wide enough to fill the cores through the long middle of a build, when hundreds of independent recipes are eligible at the same time.

Memory is what makes that safe. Every Linux size carries 4 GB per vCPU, so the 32 vCPU runner has 128 GB. Toolchain, kernel, and browser-engine recipes are the ones that will find the ceiling. If a do_compile step dies with a killed compiler, lower PARALLEL_MAKE to -j 16 and leave BB_NUMBER_THREADS at 32, which keeps bitbake's scheduler busy while cutting peak memory per task.

The 150GB SSD is the real constraint. Yocto grows in three directions at once, and a build that runs out of disk fails late and confusingly. Budget the space before the first run:

Four prunes hold that line before the snapshot is taken:

1. INHERIT += "rm_work" deletes each recipe's work directory as the recipe completes, which is the single largest saving available.
2. rm -rf build/tmp/work build/tmp/work-shared before the save step. Both are reproducible from sstate, so dropping them costs nothing on the next boot.
3. Trim duplicate sstate entries with the sstate cache management script in poky's scripts directory, which removes superseded task archives that accumulate as recipes change.
4. Delete stale images under build/tmp/deploy/images beyond the build you just published.

Run df -h / immediately before snapshot-save. When a snapshot capture is going to be tight, the job log then says so plainly.

Worked cost model

Take an embedded team running 200 image builds a month, averaging 45 minutes each on 32 vCPU machines, holding two snapshot aliases alive for two machine targets:

The WarpBuild total is $620.00 against $738.00 for the same minutes on GitHub-hosted larger runners, a difference of $118.00 per month. As a standalone rate comparison: warp-ubuntu-latest-x64-32x (32 vCPU, 128 GB) costs $0.064 per minute against $0.082 per minute for the 32-core Linux larger runner (32 vCPU, 128 GB): 22 percent lower list price. GitHub list price checked on 2026-08-13.

The model holds minutes equal on both sides.

Full rates for every size and platform are on the pricing page.

Bottlenecks

Three failure modes account for most of the wall clock in a Yocto pipeline on GitHub Actions.

Cold sstate. With an empty SSTATE_DIR, bitbake executes every task instead of restoring it, which means compiling binutils, gcc, glibc, and a kernel before your own recipes get a turn. This is the bottleneck the snapshot removes. Two things quietly reset a warm cache back to cold: a poky branch bump, since the task input hashes change with the metadata, and a distro or machine configuration change that shifts hashes across the whole graph. Both are worth landing on main deliberately so the snapshot refresh happens on one build rather than on every pull request in flight.

Source downloads. A cold DL_DIR fetches from many independent upstream hosts, and a single slow or unreachable mirror stalls the fetch phase while the cores idle. The durable fixes are ordinary Yocto practice: keep DL_DIR inside the snapshot, set BB_GENERATE_MIRROR_TARBALLS = "1" so git checkouts become reusable tarballs, and point PREMIRRORS and SSTATE_MIRRORS at your own object storage so a snapshot miss falls back to a fast internal source rather than the public internet.

Single-threaded configure phases. do_configure for an autotools recipe is a shell script that runs hundreds of compiler probes in sequence, and PARALLEL_MAKE does nothing for it. The build fills the machine only when enough recipes are in flight to overlap their serial phases, which is why BB_NUMBER_THREADS matters more than raw core count on the early and late stretches of a build. Watch for the shape where one core is pinned and 31 are idle: that is a narrow point in the dependency graph, and the fix is usually splitting an over-broad recipe dependency rather than buying a larger machine.

Telling these apart is a measurement problem. WarpBuild's CI observability streams system metrics from the runner and correlates them with GitHub Actions job logs, so a stalled fetch reads as idle cores with network activity, a cold sstate reads as sustained full-core compile, and a configure choke point reads as one busy core. For a build that hangs or fails only in GitHub Actions, the Action Debugger pauses the workflow and opens an SSH session on the live runner, which is the fastest way to read bitbake-diffsigs output in place.

Proof

Public repositories running warp- labels are the citable evidence. The bitcoin/bitcoin GitHub Actions workflow routes its Linux jobs across warp- labels sized per job, from warp-ubuntu-latest-x64-2x for lint up to warp-ubuntu-latest-x64-16x for the fuzz and MSan matrices, with compiler caches restored and saved between runs (checked on 2026-08-13). It is a long native compile matrix rather than a Yocto build, and the sizing and cache-persistence pattern is the same one this page applies to bitbake.

Related reading: the snapshot runner reference covers the label forms and lifecycle in full, the incremental builds guide generalizes the warm-state pattern beyond Yocto, the C++ builds page covers compiler-cache tuning for the native toolchain work underneath a BSP, and the large repository guide deals with checkout and layer trees that outgrow a default runner.

FAQ

Which runner size should a Yocto build on GitHub Actions use?

Start on warp-ubuntu-latest-x64-32x at $0.064 per minute, with BB_NUMBER_THREADS set to 32 and PARALLEL_MAKE set to -j 32. The 128GB of memory is what keeps 32 concurrent bitbake tasks alive through kernel and toolchain compiles. Drop PARALLEL_MAKE to -j 16 if a large recipe gets OOM killed.

Is a snapshot runner better than a cache archive for the sstate-cache?

For Yocto, yes. A sstate-cache is hundreds of thousands of small files, so a cache archive pays tar, upload, download, and untar on every run. A snapshot boots the whole runner disk in 45 to 60 seconds with sstate-cache, downloads, build/tmp, and the installed host packages already in place.

Do snapshot labels work on BYOC, Windows, or macOS runners?

No. Snapshot runners are supported only on WarpBuild Cloud Ubuntu runners. A snapshot.key or snapshot.enabled label on a BYOC, Windows, or macOS runner is silently ignored and the job runs at full setup cost with no error.

Will a Yocto build fit inside the 150GB runner disk?

A core image fits with room to spare when you budget the space: roughly 40GB for downloads, 45GB for sstate-cache, 45GB for build/tmp, and 20GB for host packages and headroom. Inherit rm_work and delete build/tmp/work before snapshotting to hold that line.