Modular YAML configuration

Abstract

Modular YAML Configuration enables you to break down large, complex YAML files into smaller, modular components. It allows easier reuse across multiple repositories.

Enterprise only

Modular YAML configuration is only available for Workspaces with Enterprise plans. You also need to store your configuration files in your own repository, not on bitrise.io.

If you are not on an Enterprise plan but interested in modular YAML, contact us!

Modular YAML Configuration enables you to break down large, complex YAML files into smaller, modular components. It allows easier reuse across multiple repositories. By modularizing YAML files, you can quickly locate and update configurations, reducing the risk of errors and merge conflicts.

A modular YAML configuration includes:

A bitrise.yml file in the root of your repository.
Other YAML files in the same or a different repository. To include a file from a different repository, the repository must belong to the same Git account or organization as the primary repository.
One or more include keywords in the bitrise.yml file. These point to other YAML files and bring their configuration into the main project configuration.

Once an additional YAML file has been included in the bitrise.yml file, you can refer to any of its Workflows or Pipelines as to any other Workflow or Pipeline in your configuration.

Including configuration from multiple YAML files

To create a modular configuration, you need to:

Store your bitrise.yml file in your repository.
Create more YAML files in addition to bitrise.yml and commit them to your repository.

File limit

You can have a maximum of 10 includes per file, and a total of 20 configuration files, including the root level bitries.yml file.

In bitrise.yml, you can include these additional YAML files in your configuration with the include keyword. It requires one parameter: path, which is the location of the YAML file to be included.

The provided path must be relative to the repository's root:

include:
  - path: file/path/common.yml

You can include a file from a different repository by specifying the repository that contains the file. In addition to the repository property, you need to also set at least one of the following:

branch: The branch containing the YAML file.
commit: The specific commit hash from which to include the YAML file.
tag: The Git tag that points to the YAML file.

The repository must belong to the same Git account or organization that your project's primary repository belongs to. You can check your Bitrise project's repository URL on bitrise.io. For example, if your repository URL is [email protected]:MyOrg/main_repo.git, you can only refer to repositories belonging to MyOrg on GitHub.

Accessing other repositories

Bitrise needs read access to all repositories where configuration YAML files are hosted. There are two ways to make sure Bitrise can access your repo:

If you use the GitHub App integration, you can link additional repositories to your project.
For other authentication methods, check out this guide: Projects with submodules or private repo dependencies

You only need to refer to the repository's name. For example, you can refer to [email protected]:MyOrg/another_repo.git with the value another_repo.

include:
  - path: shared/common.yml
    branch: test_branch
    repository: another_repo

The `include` format

Table 1. The include format

Parameter	Required?	Description	Default
`path`	Required	The location of the YAML file you want to include. The path is relative to either: The root of the repository in a CI environment. The current directory in a local environment.	N/A
`branch`	Optional	The branch from which to include the YAML file.	Same as the including YAML file.
`tag`	Optional	The tag from which to include the YAML file. If `branch` and `tag` are both specified, tag takes priority.	Same as the including YAML file.
`commit`	Optional	The specific commit hash from which to include the YAML file. If `branch`, `tag`, and `commit` are all specified, the commit takes priority.	Same as the including YAML file.
`repository`	Optional	The repository from which to pull the YAML file. It must be specified if you want to use a file from a different repository than your default.	Bitrise CI environment: The repository URL of your Bitrise project. Local environment: The checked out local Git repository URL.

Defining configuration modules

A configuration module is a full YAML configuration file. That means that all configuration elements of a valid bitrise.yml file are available: you can define the format_version, app level Environment Variables, default stacks and machine types, and of course Pipelines, Stages, and Workflows in a configuration module.

For example, you can create a bare minimum bitrise.yml file that simply points to another module:

include:
  - path: path/to/config_module.yml

And in this case, config_module.yml contains the entire configuration for the build:

format_version: 13
default_step_lib_source: https://github.com/bitrise-io/bitrise-steplib.git
project_type: android
app:
  envs:
  - MY_NAME: My Name
workflows:
  test:
    steps:
    - script:
        inputs:
        - content: echo "Hello ${MY_NAME}!"

A configuration module can contain any configuration entity defined on the root level. For example, a configuration module defining only a single Workflow or just app-level Environment Variables is perfectly valid.

The bitrise.yml file:

format_version: 13
default_step_lib_source: https://github.com/bitrise-io/bitrise-steplib.git
include:
  - path: path/to/config_module.yml

The config_module.yml file:

app:
  envs:
  - USER_NAME: UserName
workflows:
  test:
    steps:
    - script:
        inputs:
        - content: echo "Hello ${USERNAME}!"

However, non-root level entities cannot stand alone in a separate module file either: you can't, for example, include Step inputs like this.

Nesting included modules

You can use the include property in included configuration files:

Root level bitrise.yml file:

format_version: 13
default_step_lib_source: https://github.com/bitrise-io/bitrise-steplib.git
include:
  - path: path/to/config_module.yml

The config_module.yml file:

include:
  - path: path/to/another_module.yml

The another_module.yml file:

workflows:
  ui_test:
    steps:
    - pull-intermediate-files@1:
        inputs:
        - artifact_sources: build_tests.build_for_ui_testing

With this configuration, you'll be able to run the ui_test Workflow.

Nesting has the following limitations:

A depth of 5, including the root level file.
Maximum 10 includes per file.
Maximum 20 configuration files in total, including the root level file.

Merge rules for included modules

When Bitrise runs a build, we merge your configuration modules into a single bitrise.yml file:

Included modules are read and merged into the configuration in the order defined in the root bitrise.yml file.
If an included module also uses include, the nested module is merged first recursively.
After all configuration files added with include are merged together, the resulting configuration is merged with the bitrise.yml file on the root level. As such, you can override configuration values on the root level.

Switching back to bitrise.io

If you have a modular configuration with multiple YAML files and you switch back to storing your configuration on bitrise.io, we use the same merge rules to merge all your files into a single bitrise.yml file.

When merging the configurations, it is possible to encounter overlaps and override included configuration values. When merging YAML modules, the most recently read file takes priority over the existing merged configuration. Here are the rules:

Items of simple types (such as integers and booleans): The value from the most recently read file is used.
Items of object type (for example, Pipelines, Stages, Workflows):
- If a property is only present in the existing merged configuration, that value is retained.
- If a property is present in both, and their values are hash maps, the values are merged.
- In any other case, the value from the most recently read file is used.
Items of array type (for example, lists of Environment Variables or trigger map items): if the collection is present in both the most recently read file and the existing merged configuration, the merged value is an ordered array of values, with all values from the merged configuration followed by the values from the most recently read file.

You can see the merged configuration file in the Workflow Editor: select bitrise.yml from the left navigation menu to view it.

Example 1. Merging rules

We have the root bitrise.yml file which includes a config_module.yml file.

The bitrise.yml:

format_version: 13
default_step_lib_source: https://github.com/bitrise-io/bitrise-steplib.git
project_type: android

include: 
  - path: path/to/config_module.yml

app:
  envs:
  - USER_ID: UserId
  - PASSWORD: SecurePassphrase

The config_module.yml:

format_version: 10

app:
  envs:
  - USERNAME: UserName

workflows:
  test:
    steps:
    - script:
        inputs:
        - content: echo "Hello ${USERNAME}!"

During the merge, the config_module.yml and is merged into the configuration. Then the root level bitrise.yml is read and then merged. The final configuration looks like this:

format_version: 13
default_step_lib_source: https://github.com/bitrise-io/bitrise-steplib.git
project_type: android

app:
  envs:
  - USER_ID: UserId
  - PASSWORD: SecurePassphrase
  - USERNAME: UserName

workflows:
  test:
    steps:
    - script:
        inputs:
        - content: echo "Hello ${USERNAME}!"

The root level format_version property is present in both files. The root level bitrise.yml file is merged into the overall configuration last, therefore its value of format_version is used.
The apps property is present in both files so all key-value pairs are added to the overall configuration.
The test Workflow is only present in config_module.yml so it's added to the overall configuration and bitrise.yml does not override it.

In this section: