Using key-based caching in your builds
Key-based caching requires defining a key that identifies a cache archive. A key can be static or dynamically generated, and you can use templates in your keys. There are dedicated key-based caching Steps for dependency managers that require no configuration.
Key-based caching requires two Steps that must be used together:
-
Save Cache: saves the build files into a cache archive, identified by a key.
-
Restore Cache: restores build files into a cache archive, identified by a key.
Dedicated key-based caching Steps
You can use dedicated key-based caching Steps, such as the Save NPM Cache, to cache npm or yarn dependencies.
For a full list of dedicated key-based caching Steps, check out Dedicated caching Steps for dependency managers.
Both Steps access cache archives via key strings. These keys need to be specified as the values of the Cache keys Step input; each key identifies a separate cache archive.
Cache retention and eviction policy
Your cache entries are retained for seven days since the last use. If they're not used in your workflows for a continuous period of seven days, they will be automatically removed.
If you run out of your cache storage allowance, the Least Recently Used (LRU) cache entries on your app will be replaced with any new ones produced. LRU is a cache replacement algorithm that removes the least recently used data in order to make room for new data. This might mean that your cache hit rate could reduce if the replaced cache entries are needed by any Workflows. We recommend upgrading to a higher plan to get more storage so that you don't lose any cache entries that might be needed by your Workflows.
Creating a new cache archive
-
Add the Save Cache Step at the end of your Workflow.
-
In the Cache key input, define a cache key. This key will be used to identify the cache archive.
Key limitations
The maximum length of a cache key is 512 characters (longer keys get truncated). Commas (,) are not allowed in keys.
You can use templates and functions to create dynamic keys that change depending on the build environment or other factors.
You can specify multiple keys; the Step will evaluate them in order and match the first one. Read more: Key matching for cache archives.
Conditional caching with dynamic keys
You can configure the Step in a way that allows it to automatically skip archiving and uploading the cache if its content has not changed during the build: Using conditional caching with dynamic keys.
Example 1. Creating a new cache archive with a key referring to the OSIn this example, we're creating a cache archive with a key that refers to the type of the operating system of the build machine.
- save-cache@1: inputs: - key: |- npm-cache-{{ .OS }}
-
In the Paths to cache input, define the files and folders that you want to cache.
The input allows for wildcards:
*
and**
. The input value is evaluated at runtime.Archive size limit
The size of a single cache archive cannot exceed 15 GB.
Example 2. Caching all files and folders recursively in the node_modules folder- save-cache@1: inputs: - key: |- npm-cache-{{ .OS }} - paths: node_modules/
Once a cache archive has been created, you can access it on the App Settings page: Accessing key-based cache archives on the website.
Restoring an existing cache archive
Cache archives cannot be shared between builds running on Linux and Mac stacks
A cache archive saved on a Linux-based stack can only be restored in builds also running on Linux machines. Similarly, Mac builds can only restore caches previously saved in a Mac build. The Restore Cache Step fails with a "The specified key does not exist." error if the build tries to restore a cache archive saved on a different OS.
Cache archives can be shared among stacks with different versions of the same OS (for example, between different Xcode stacks).
-
Add the Restore Cache Step at the start of your Workflow.
-
In the Cache key input, type the key of the cache that you want to restore.
You can use templates and functions to create dynamic keys that change depending on the build environment or other factors.
You can specify multiple keys; the Step will evaluate them in order and select the first matching one. Read more: Key matching for cache archives.
Example 3. Restoring a cache archive from one of two keysIn this example, we're restoring one of two keys:
-
First, we'll look for a cache archive with a key that includes the name of the current Workflow. For example, if the current Workflow's name is
primary
, the Step will look for an archive with the keynpm-cache-primary
. -
Second, we'll look for an archive with a key that includes the name of the current branch. For example, if the current branch's name is
main
, the Step will look for an archive with the keynpm-cache-main
.
- restore-cache@1: inputs: - key: |- npm-cache-{{ .Workflow }} npm-cache-{{ .Branch }}
-
Key-based caching templates and functions
Both key-based caching Steps support using template elements in their Step inputs. The Steps evaluate the key template at runtime and the final cache key to be used can change depending on the build environment or on certain files in the repository.
Available caching templates
Template expression |
Definition |
Possible values |
---|---|---|
|
Current git branch the build runs on. |
The exact name of any existing branch of the app. |
|
SHA-256 hash of the git commit the build runs on. |
Any existing commit hash. |
|
Current Bitrise workflow name (for example, |
The exact name of any existing Workflow of the app. |
|
Current operating system of the build stack ( |
|
Using functions in caching templates
The key-based caching templates support the use of two different functions:
-
checksum
: This function computes the SHA-256 checksum of the contents of one or more files. This is useful for creating unique cache keys based on files that describe content to cache. Example 1, “Using the checksum function”. -
getenv
: This function returns the value of an Environment Variable (Env Var) or an empty string if the variable is not defined. Example 2, “Using the getenv function”
Use the checksum
function to create a key that computes the checksum of the package-lock.json
file:
- save-cache@1: inputs: - key: npm-cache-{{ checksum "package-lock.json" }} - paths: node_modules
Use the checksum
function to create a key that computes a checksum for any .gradle
file and the gradle.properties
file:
- save-cache@1: inputs: - key: gradle-cache-{{ checksum "**/*.gradle*" "gradle.properties" }} - paths: AndroidApp
Use the getenv
function to create a key that contains the value of the BITRISE_BUILD_NUMBER
Env Var.
- save-cache@1: inputs: - key: npm-cache-{{ getenv "BITRISE_BUILD_NUMBER" }} - paths: node_modules
Key matching for cache archives
It's possible to define more than one key in the Cache keys input of the key-based caching Steps. You can specify additional keys by listing one key per line. The list is in priority order, so the Step will first try to find a match for the first key you provided, and if there is no cache stored for the key, it will move on to find a match for the second key (and so on).
inputs: key: | key-1 key-2 key-3
In addition to listing multiple keys, each key can be a prefix of a saved cache key and still get a matching cache archive. For example, the key my-cache-
can match an existing archive saved with the key my-cache-a6a102ff
. We recommend configuring the keys in a way that the
first key is an exact match to a checksum key, and to use a more generic prefix key as a fallback:
inputs: key: | npm-cache-{{ checksum "package-lock.json" }} npm-cache-
Using conditional caching with dynamic keys
You might not want to update a cache archive every time you run a build. By skipping caching, you can:
-
Avoid saving potentially incorrect build data in the cache.
-
Make the PR validation workflow faster.
-
Limit the amount of data stored and transferred.
If the cached content did not change during your build at all, the Save cache Step can automatically skip compressing and uploading the cache. To make this work, you need to make sure your cache archive has a unique cache key that changes whenever the contents of the cache change: in practice, this requires a dynamic key with a checksum and a file that describes the cached content. The checksum will be calculated based on the contents of the file; if the file remains unchanged, so does the checksum.
We strongly recommend configuring the Step in a way to take advantage of this feature. Follow the steps below, or skip ahead to our YAML example.
Skipping the Save cache Step entirely
If you don't wish to update the cache archive even if the content that would be cached has changed, you can either remove the Save cache Step from your Workflow or you can set conditions for the Step: Enabling or disabling a Step conditionally.
-
Log in to Bitrise and select Bitrise CI on the left, then select your project.
-
Click the
button on the main page. -
Open your Workflow and select the Save cache Step.
-
Set the Unique cache key input to
true
.If the input is set to
false
, the Step can still skip uploading the cache but it will need to create the cache archive first to calculate its checksum. This takes time. -
In the Cache key input, create a key with a checksum for a file that describes the cached content.
For example,
npm-cache-{{ checksum "package-lock.json" }}
is a dynamic key that remains unchanged as long as thepackage-lock.json
file does not change. -
Add the same dynamic key to the Cache keys input of the Restore cache Step.
If everything is configured correctly, if the keys in the two caching Steps match, the cache archive will not be updated.
In this example, the cache key calculates the sum for any .gradle
file and the gradle.properties
file. Both the Restore cache and the Save cache Step looks for the same cache
key:
workflows: caching: steps: - restore-cache@1: inputs: - key: gradle-cache-{{ checksum "**/*.gradle*" "gradle.properties" }} - save-cache@1: inputs: - key: gradle-cache-{{ checksum "**/*.gradle*" "gradle.properties" }} - is_key_unique: 'true'