Using GitHub Actions Cache with popular languages

16 May 2024•21 minute read
Surya OrugantiFounder and CEO

GitHub Actions Cache

GitHub Actions provides a powerful CI/CD platform enabling developers to automate workflows and streamline development pipelines. One valuable feature to speed up these pipelines is caching. By caching dependencies and other build artifacts, you can drastically reduce build times, particularly for frameworks that rely on extensive dependency fetching and compilation.

GitHub provides a cache action that allows workflows to cache files between workflow runs.

In this post, we'll dive into how to use this action for popular programming languages and frameworks, including Node.js, Python, Rust, Go, PHP, and Java. We'll also highlight common pitfalls, considerations, and shortcomings of the cache action to provide a comprehensive understanding.

Benefits

Caching dependencies offers several benefits:

  1. Faster Builds: By caching dependencies, you can avoid the time-consuming process of downloading and installing them on every workflow run. This leads to faster build times and quicker feedback loops.

  2. Reduced Network Bandwidth: Caching minimizes the need to download dependencies repeatedly, saving network bandwidth and reducing the load on package registries.

  3. Improved Reliability: Caching ensures that your builds are less susceptible to network issues or outages of package registries, as the cached dependencies can be restored locally.

Using the Cache Action

Note

GitHub provides official environment setup actions for a few popular languages and frameworks. These actions support caching dependencies out of the box. For the rest, you can use the actions/cache action directly to cache the relevant directories.

Node.js

Caching dependencies in Node.js involves storing the package manager cache.

The official setup-node action supports caching by using actions/cache under the hood, abstracting out the setup required to cache the required package manager cache directories. It supports caching for npm, yarn, and pnpm with the cache input. Caching is disabled be default.

Note

Caching the node_modules is not recommended by GitHub as it fails across Node versions and doesn't work well with npm ci.

1name: Node CI
2on: [push, pull_request]
3
4jobs:
5  build:
6    runs-on: ubuntu-latest
7
8    steps:
9      - uses: actions/checkout@v4
10      - name: Setup Node.js
11        uses: actions/setup-node@v4
12        with:
13          node-version: 20
14          cache: "npm" # or 'yarn' or 'pnpm'
15
16      - name: Install dependencies
17        run: npm ci # or 'yarn install --frozen-lockfile' or 'pnpm install --frozen-lockfile'

The above action uses the relevant lockfile used by the package manager to create the cache key. For more control over caching, such as using your own cache keys or caching the node_modules directory, you can use the actions/cache action directly. Here's an example of caching the package directory for an npm project:

1name: Node CI
2on: [push, pull_request]
3
4jobs:
5  build:
6    runs-on: ubuntu-latest
7
8    steps:
9      - uses: actions/checkout@v4
10      - name: Setup Node.js
11        uses: actions/setup-node@v4
12        with:
13          node-version: 20
14
15      - name: Cache .npm directory
16        uses: actions/cache@v4
17        with:
18          path: ~/.npm
19          key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
20          restore-keys: |
21            ${{ runner.os }}-node-
22
23      - name: Install dependencies
24        run: npm ci

Note

Make sure to use the correct cache directory for your package manager (npm, yarn, or pnpm). You can get the cache directory by running npm config get cache or yarn cache dir. Learn more here.

Python

Caching in Python projects also involves storing the package manager cache directory. Similar to Node.js, the official setup-python action also supports caching via the cache input.

1name: Python CI
2on: [push, pull_request]
3
4jobs:
5  build:
6    runs-on: ubuntu-latest
7
8    steps:
9      - uses: actions/checkout@v4
10      - name: Setup Python
11        uses: actions/setup-python@v5
12        with:
13          python-version: "3.12"
14          cache: "pip" # or 'poetry' or 'pipenv'
15
16      - name: Install dependencies
17        run: pip install -r requirements.txt

Similarly, you can also use the actions/cache action directly for more control over caching:

1name: Python CI
2on: [push, pull_request]
3
4jobs:
5  build:
6    runs-on: ubuntu-latest
7
8    steps:
9      - uses: actions/checkout@v4
10      - name: Setup Python
11        uses: actions/setup-python@v5
12        with:
13          python-version: "3.12"
14
15      - name: Cache pip packages
16        uses: actions/cache@v4
17        with:
18          path: ~/.cache/pip
19          key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
20          restore-keys: |
21            ${{ runner.os }}-pip-
22
23      - name: Install dependencies
24        run: pip install -r requirements.txt

Note

You can find the correct cache directory for pip by running pip cache dir and for poetry by running poetry config cache-dir. Learn more here

Golang

For Go projects, caching the Go modules directory speeds up the build process. The directory is generally located at ~/go/pkg/mod and can be found by running go env GOMODCACHE.

The setup-go action provides caching support for Go projects with the cache input. Unlike Node.js and Python, this input is a boolean value that enables or disables caching. Caching is enabled by default.

1name: Go CI
2on: [push, pull_request]
3
4jobs:
5  build:
6    runs-on: ubuntu-latest
7    steps:
8      - uses: actions/checkout@v4
9      - name: Setup Go
10        uses: actions/setup-go@v5
11        with:
12          go-version: "1.22"
13          cache: true # Note: this is not required as caching is enabled by default
14
15      - name: Build
16        run: go build ./...

You can also disable the caching by setting cache: false in the actions/setup-go action and use the actions/cache action directly for more control.

1name: Go CI
2on: [push, pull_request]
3
4jobs:
5  build:
6    runs-on: ubuntu-latest
7    steps:
8      - uses: actions/checkout@v4
9      - name: Setup Go
10        uses: actions/setup-go@v5
11        with:
12          go-version: "1.22"
13          cache: false
14
15      - name: Cache Go modules
16        uses: actions/cache@v4
17        with:
18          path: ~/go/pkg/mod
19          key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
20          restore-keys: |
21            ${{ runner.os }}-go-
22
23      - name: Build
24        run: go build ./...

Note

If you use vendor directories, the modules get loaded from your project's vendor directory instead of downloading from the network or restoring from cache. In such cases, caching the Go modules directory may not be necessary.

Rust

Rust's package manager, Cargo caches the modules and binaries in the ~/.cargo directory. The compiled dependencies are stored in the target directory of the project.

Rust has no official GitHub Action for setup, so you can use the actions/cache action directly to cache the relevant directories.

1name: Rust CI
2on: [push, pull_request]
3
4jobs:
5  build:
6    runs-on: ubuntu-latest
7    steps:
8      - uses: actions/checkout@v4
9      - name: Setup Rust
10        uses: dtolnay/rust-toolchain@stable
11
12      - name: Cache cargo registry and build
13        uses: actions/cache@v4
14        with:
15          path: |
16            ~/.cargo/bin/
17            ~/.cargo/registry/index/
18            ~/.cargo/registry/cache/
19            ~/.cargo/git/db/
20            target
21          key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
22          restore-keys: |
23            ${{ runner.os }}-cargo-
24
25      - name: Build and test
26        run: cargo test --all

Java

Java projects commonly use the Gradle or Maven build tools which have their corresponding cache directories.

Java does have an official setup-java action that supports caching via the cache input. This input takes the name of the build tool (maven, gradle or sbt) to cache their relevant directories. Caching is disabled by default.

1name: Java CI
2on: [push, pull_request]
3
4jobs:
5  build:
6    runs-on: ubuntu-latest
7    steps:
8      - uses: actions/checkout@v4
9      - name: Setup Java
10        uses: actions/setup-java@v4
11        with:
12          distribution: "temurin"
13          java-version: "17"
14          cache: "maven" # or 'gradle' or 'sbt'
15
16      - name: Build with Maven
17        run: mvn -B clean verify

You can also use the actions/cache action directly for more control over caching.

Maven Example

1name: Java CI
2on: [push, pull_request]
3
4jobs:
5  build:
6    runs-on: ubuntu-latest
7    steps:
8      - uses: actions/checkout@v4
9      - name: Setup Java
10        uses: actions/setup-java@v4
11        with:
12          distribution: "temurin"
13          java-version: "17"
14      - name: Cache Maven repository
15        uses: actions/cache@v4
16        with:
17          path: ~/.m2/repository
18          key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
19          restore-keys: |
20            ${{ runner.os }}-maven-
21      - name: Build with Maven
22        run: mvn -B clean verify

Gradle Example

1name: Java Gradle CI
2on: [push, pull_request]
3
4jobs:
5  build:
6    runs-on: ubuntu-latest
7    steps:
8      - uses: actions/checkout@v4
9      - name: Setup Java
10        uses: actions/setup-java@v4
11        with:
12          distribution: "temurin"
13          java-version: "17"
14
15      - name: Cache Gradle wrapper
16        uses: actions/cache@v4
17        with:
18          path: |
19            ~/.gradle/caches
20            ~/.gradle/wrapper
21          key: ${{ runner.os }}-gradle-${{ hashFiles('**/gradle/wrapper/gradle-wrapper.properties') }}
22          restore-keys: |
23            ${{ runner.os }}-gradle-
24
25      - name: Build with Gradle
26        run: chmod +x gradlew && ./gradlew build

PHP

PHP projects with Composer can cache their dependencies by caching the Composer cache directory. PHP has no official GitHub Action for setup, so you can use the actions/cache action directly.

1name: PHP Cache Workflow
2on: [push, pull_request]
3
4jobs:
5  build:
6    runs-on: ubuntu-latest
7
8    steps:
9      - name: Checkout code
10        uses: actions/checkout@v4
11
12      - uses: shivammathur/setup-php@v2
13          with:
14            php-version: '8.3'
15
16      # The cache directory is usually located at ~/.composer/cache
17      # This step can be skipped and the cache directory can be hardcoded
18      # in the `path` field of the `actions/cache` step.
19      - name: Get Composer Cache Directory
20        id: composer-cache
21        run: |
22          echo "dir=$(composer config cache-files-dir)" >> $GITHUB_OUTPUT
23
24      - name: Cache Composer dependencies
25        uses: actions/cache@v4
26        with:
27          path: ${{ steps.composer-cache.outputs.dir }}
28          key: ${{ runner.os }}-composer-${{ hashFiles('**/composer.lock') }}
29          restore-keys: |
30            ${{ runner.os }}-composer-
31
32      - name: Install dependencies
33        run: composer install --prefer-dist

Ruby

The official ruby/setup-ruby action provides caching support for Ruby projects with the bundler-cache input. This input takes a boolean value to enable or disable caching. Caching is disabled by default.

1name: Ruby CI
2on: [push, pull_request]
3
4jobs:
5  build:
6    runs-on: ubuntu-latest
7    steps:
8      - uses: actions/checkout@v4
9      - name: Setup Ruby
10        uses: ruby/setup-ruby@v1
11        with:
12          ruby-version: "3.3"
13          bundler-cache: true
14
15      # Installing dependencies via `bundle install` or `gem install bundler` 
16      # is not required as the action automatically installs dependencies.
17
18      - name: Run tests
19        run: bundle exec rake test

We can also directly cache the bundle directory using the actions/cache action.

Note

Manually caching this directory is not recommended, and the suggested approach is to use the ruby/setup-ruby action as shown above.

1name: Ruby CI
2on: [push, pull_request]
3
4jobs:
5  build:
6    runs-on: ubuntu-latest
7    steps:
8      - uses: actions/checkout@v4
9      - name: Setup Ruby
10        uses: ruby/setup-ruby@v1
11        with:
12          ruby-version: "3.3"
13
14      - name: Cache gems
15        uses: actions/cache@v4
16        with:
17          path: vendor/bundle
18          key: ${{ runner.os }}-gems-${{ hashFiles('**/Gemfile.lock') }}
19          restore-keys: |
20            ${{ runner.os }}-gems-
21
22      - name: Install dependencies
23        run: |
24          gem install bundler
25          bundle install --path vendor/bundle
26
27      - name: Run tests
28        run: bundle exec rake test

.NET (NuGet)

The official setup-dotnet action provides caching support for .NET projects with the cache input. This input takes a boolean value to enable or disable caching. Caching is disabled by default.

Note

Passing an explicit NUGET_PACKAGES is also recommended for caching the NuGet packages directory instead of the global cache directory since there might be some huge packages pre-installed.

1name: .NET CI
2on: [push, pull_request]
3
4jobs:
5  build:
6    runs-on: windows-latest
7    env:
8      NUGET_PACKAGES: ${{ github.workspace }}/.nuget/packages
9    steps:
10      - uses: actions/checkout@v4
11      - name: Setup .NET
12        uses: actions/setup-dotnet@v4
13        with:
14          dotnet-version: "6.x"
15          cache: true
16
17      - name: Restore dependencies
18        run: dotnet restore --locked-mode
19
20      - name: Build
21        run: dotnet build my-project

For caching manually, use the actions/cache action directly.

1name: .NET CI
2on: [push, pull_request]
3
4jobs:
5  build:
6    runs-on: windows-latest
7    env:
8      NUGET_PACKAGES: ${{ github.workspace }}/.nuget/packages
9    steps:
10      - uses: actions/checkout@v4
11      - name: Setup .NET
12        uses: actions/setup-dotnet@v4
13        with:
14          dotnet-version: "6.x"
15
16      - name: Cache NuGet packages
17        uses: actions/cache@v4
18        with:
19          path: ${{ github.workspace }}/.nuget/packages
20          key: ${{ runner.os }}-nuget-${{ hashFiles('**/packages.lock.json') }} # Or **/*.csproj
21          restore-keys: |
22            ${{ runner.os }}-nuget-
23
24      - name: Restore dependencies
25        run: dotnet restore --locked-mode
26
27      - name: Build 
28        run: dotnet build my-project

Considerations

While the cache action speeds up workflows, be mindful of these considerations:

  1. Cache Keys and Restoring: Using unique cache keys ensures a cache is correctly restored or created. However, overly specific keys may result in missed cache hits.
  2. Sequencing: Sequence of cache commits and restores could lead to unpredictable behavior in workflows, especially if the cache keys are insufficiently defined.
  3. Cache Size: Large caches can take longer to restore, reducing the performance gain.
  4. Invalidation: Changes in dependencies (like updates in package-lock.json or go.sum) will cause a new cache to be created.
  5. Security Risks: Ensure sensitive files are not accidentally cached.

Common Pitfalls

  1. Concurrency Issues: Parallel jobs can overwrite the cache leading to incomplete or corrupted data.
  2. Storage Limits: Exceeding storage limits (10 GB per repository) will cause cache eviction and is a very low limit for many use cases.
  3. Compatibility: Some caches may not be compatible across different operating systems or configurations.

Conclusion

Leveraging the GitHub Actions cache action can significantly accelerate your CI/CD workflows, especially with common languages like Node.js, Python, Rust, Go, PHP, and Java. However, it's essential to manage cache keys, avoid overly large caches, and be cautious of security issues. For optimal results, test different strategies to see which works best for your specific project requirements.

Unlimited, fast cache with WarpBuilds/cache action

While GitHub Actions provides great flexibility and functionality, workflow speeds can always benefit from improvements. This is where WarpBuild comes in. Offering GitHub Actions runners with unlimited, superfast caching capabilities, WarpBuild accelerates your builds with blazing speed. The WarpBuilds/cache action is a drop-in replacement for the actions/cache, so you can get started instantly. Here are the cache docs.

  • Unlimited Caching: Never worry about hitting cache size limits or losing important build data.
  • Fast Caching: Save substantial time by utilizing highly optimized caching mechanisms.

By seamlessly integrating WarpBuild into your workflows, you can significantly speed up your CI/CD pipelines without compromising flexibility or reliability. Try it out.

References

Previous post

A Complete Guide to Self-hosting GitHub Actions Runners

29 April 2024
GitHub ActionsGitHubGuideEngineering
Next post

Rate Limit Cheatsheet for Self-Hosting Github Runners

12 June 2024
Rate LimitsGitHub Actions