Supercharge your CI with Snapshot Runners

Why Local Builds and Tests Are Faster than CI

Developers often experience faster builds locally compared to their CI systems, largely because their local machine has cached dependencies, libraries, and other artifacts. Every CI runner starts fresh, losing this advantage. This disparity in build times can cause frustration and delay continuous integration and deployment workflows.

Caching in CI Runners

CI builds can be sped up by caching dependencies, Docker layers, and other artifacts. However, traditional CI caching mechanisms can't fully replicate the performance of local builds due to the limitations of what can be cached.

Enter Snapshot Runners

Snapshot Runners offer a more powerful caching approach. Instead of relying on external caches, Snapshot Runners capture a complete snapshot of the VM runner just before it exits. This snapshot includes all dependencies, build caches, and system-level optimizations, allowing subsequent runners to start with a fully primed environment. This drastically reduces initialization time and improves overall build performance.

Seamlessly Integrate Snapshot Runners into Your CI Pipelines

Integrating Snapshot Runners into your existing workflows is easy. Simply change the warp runner in runs-on to have a snapshot.key parameter. This key is used to identify the snapshot and load it into the runner.

- runs-on: "warp-ubuntu-latest-x64-4x"
+ runs-on: "warp-ubuntu-latest-x64-4x;snapshot.key=pocketbase-snp-warp"

Also, add the WarpBuilds/snapshot-save@v1 action at the end of your workflow or at the point where you want to create the snapshot.

It is recommended to clean up all credentials and sensitive information before creating a snapshot.

Here's an example workflow which we modified to use snapshot runners. This is the basebuild action for Pocketbase, a popular open-source project.

name: basebuild

on:
  pull_request:
  push:

jobs:
  goreleaser:
    runs-on: "warp-ubuntu-latest-x64-4x;snapshot.key=pocketbase-snp-warp"

    steps:
      - name: Checkout
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Log GitHub context
        uses: actions/github-script@v7
        with:
          script: |
            console.log('GitHub context:', context);
            core.debug('Full GitHub context object:');
            core.debug(JSON.stringify(context, null, 2));

      - name: Set up Node.js
        uses: WarpBuilds/setup-node@v4
        with:
          node-version: 20.11.0

      - name: Ensure GCC is installed
        run: |
          if ! command -v gcc &> /dev/null
          then
            echo "GCC could not be found, installing..."
            sudo apt-get update
            sudo apt-get install -y gcc
          else
            echo "GCC is already installed"
          fi

      - name: Set up Go
        uses: WarpBuilds/setup-go@v5
        with:
          go-version: ">=1.22.5"
          cache: false

      - name: Build Admin dashboard UI
        run: npm --prefix=./ui ci && npm --prefix=./ui run build

      - name: Run tests
        run: go test ./...

      - name: Run GoReleaser
        uses: goreleaser/goreleaser-action@v3
        with:
          distribution: goreleaser
          version: latest
          args: release --clean
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

      - name: Create snapshot
        if: github.event_name == 'push'
        uses: WarpBuilds/snapshot-save@v1
        with:
          wait-timeout-minutes: 30
          fail-on-error: "false"
          alias: "pocketbase-snp-warp"

Benchmarks

We modified some popular CI workflows to use Snapshot Runners. Here are the results:

Common Pitfalls and Best Practices

Network attached disks

Currently, snapshot runners use a network attached disk which might perform worse than Warpbuild's cache action in case of large number of files. For large directories like node_modules, it may be better to use cache action instead of relying on files in snapshot. Combine both of these approaches based on your use case.

Refreshing Snapshots

While Snapshot Runners can store everything from previous builds, it is a good idea to refresh the base snapshot periodically to prevent outdated or unnecessary artifacts from accumulating. The snapshot can be refreshed by using the snapshot-save action with the same key.

Runner Compatibility

Snapshot runners are a WarpBuild exclusive feature and works only with WarpBuild Runners. Get Started with WarpBuild.