Skip to main content

Pipelines with stages

Pipelines with stages are the older way of configuring Pipelines. We recommend switching to the new way where you can define Workflow dependencies to create more flexible configurations, and you no longer have to waste time waiting for a stage to finish.

However, you can keep using Pipelines with stages. Such Pipelines have three main building blocks:

  • Steps: Blocks of script execution, each defining a single task in a CI/CD process.

  • Workflows: Collections of Steps. When a build of an app is running, the Steps will be executed in the order that is defined in the Workflow.

  • Stages: Collections of Workflows. A Stage can contain multiple Workflows which all run in parallel in the same Stage. If all Workflows are successful in a Stage, the Pipeline moves on to the next Stage. If any of the Workflows fail, the Pipeline ends without running the other Stages unless you configure a given Stage to always run.

Configuring a Pipeline with stages

Configuring a Pipeline that contains stages is only possible by directly editing the bitrise.yml file. You can create and modify Workflows in the graphical Workflow Editor but you need to define Pipelines and Stages in YAML format.

Defining Pipelines with stages

The bitrise.yml file contains the full configuration of your Pipelines. As with all bitrise.yml files, first you need to define the format version and the project type.

---
format_version: '8'
default_step_lib_source: 
project_type: android

This is a bare minimum bitrise.yml configuration. To define your Pipelines, you will need to use the pipelines attribute.

pipelines:  
  pipeline-successful:
    stages:
    - stage-successful-1: {}
    - stage-successful-2: {}
    - stage-successful-3: {}

In this example, we have a Pipeline called pipeline-successful, with three Stages that will run consecutively. This means that if stage-successful-1 finishes successfully, stage-successful-2 starts. If any of the Stages fail, the subsequent Stage will not start: instead, the Pipeline will be aborted and marked as failed.

Each Stage has to be defined separately under the stages attribute. Defining a Stage means specifying the Workflows that are part of the Stage.

stages:
  stage-successful-1:
    workflows:
    - test-1: {}
  stage-successful-2:
    workflows:
    - build-1: {}
    - build-2: {}
  stage-successful-3:
    workflows:
    - deploy-1: {}
    - deploy-2: {}

In this example, the Stages run the test-1, build-1, build-2, deploy-1, and deploy-2 Workflows.

Configuring a Stage to always run

By default, if a Stage fails - because one of its Workflows failed -, any other subsequent Stages of the Pipeline will not run. However, you can configure your Pipeline to run certain Stages unless the Pipeline is aborted.

To do so, you just need to set the should_always_run attribute of the Stage to true:

stages:
  stage-always-run-successful-1:
    should_always_run: true
    workflows:
    - deploy-1: {}
    - deploy-2: {}

In the example above, the Stage called stage-always-run-successful-1 will always run, regardless of the status of previous Stages. The only way these Stages will not run is if the Pipeline build is aborted by the user.

Aborting the Workflows of a failed Stage

By default, if a Workflow in a particular Stage fails, the other Workflows in the same Stage aren’t automatically aborted: these Workflows will run but the next Stage won’t start. However, you can change this behavior to immediately and automatically abort all other Workflows in the same Stage.

To do so, you need to set the abort_on_fail attribute to true:

stages:
  stage-abort-on-fail-1:
    abort_on_fail: true
    workflows:
    - deploy-1: {}
    - deploy-2: {}

Sharing Env Vars between Pipeline Stages

You can reuse any environment variable from a Workflow and reuse it in subsequent Stages using the Share Pipeline variablesStep.

Optional Workflows using run_if conditions

You can easily combine the Share Pipeline variables Step with run_if expressions to create Pipelines with optional Workflows. For more information, check out Setting up run_if conditions for optional Workflows using Pipelines.Setting up run_if conditions for optional Workflows using Pipelines

To do so:

  1. Add the Share Pipeline variables Step to the Workflow.

  2. Optionally, you can define additional run conditions in the Additional run conditions input. The Step will only run if the conditions you specify here are true.

  3. Add the Env Var(s) you would like to use in subsequent Stages in the Variables to share between Pipeline Stages input.

    Using environment variable keys

    You can define Env Vars using a {key}={value} syntax. For example, MY_ENV_KEY=value, or INSTALL_PAGE_URL=$BITRISE_PUBLIC_PAGE_URL.

    If you want to use the default environment variable keys, you can use a shorthand syntax. For example, EXISTING_ENV_KEY.

    Sharing Env Vars using this Step does not override existing Env Vars defined in the app.

That's it! You can now use the Env Var in any subsequent Stage!

Configuring a Pipeline with stages

Configuring a Pipeline that contains stages is only possible by directly editing the bitrise.yml file. You can create and modify Workflows in the graphical Workflow Editor but you need to define Pipelines and Stages in YAML format.

Defining Pipelines with stages

The bitrise.yml file contains the full configuration of your Pipelines. As with all bitrise.yml files, first you need to define the format version and the project type.

---
format_version: '8'
default_step_lib_source: 
project_type: android

This is a bare minimum bitrise.yml configuration. To define your Pipelines, you will need to use the pipelines attribute.

pipelines:  
  pipeline-successful:
    stages:
    - stage-successful-1: {}
    - stage-successful-2: {}
    - stage-successful-3: {}

In this example, we have a Pipeline called pipeline-successful, with three Stages that will run consecutively. This means that if stage-successful-1 finishes successfully, stage-successful-2 starts. If any of the Stages fail, the subsequent Stage will not start: instead, the Pipeline will be aborted and marked as failed.

Each Stage has to be defined separately under the stages attribute. Defining a Stage means specifying the Workflows that are part of the Stage.

stages:
  stage-successful-1:
    workflows:
    - test-1: {}
  stage-successful-2:
    workflows:
    - build-1: {}
    - build-2: {}
  stage-successful-3:
    workflows:
    - deploy-1: {}
    - deploy-2: {}

In this example, the Stages run the test-1, build-1, build-2, deploy-1, and deploy-2 Workflows.

Configuring a Stage to always run

By default, if a Stage fails - because one of its Workflows failed -, any other subsequent Stages of the Pipeline will not run. However, you can configure your Pipeline to run certain Stages unless the Pipeline is aborted.

To do so, you just need to set the should_always_run attribute of the Stage to true:

stages:
  stage-always-run-successful-1:
    should_always_run: true
    workflows:
    - deploy-1: {}
    - deploy-2: {}

In the example above, the Stage called stage-always-run-successful-1 will always run, regardless of the status of previous Stages. The only way these Stages will not run is if the Pipeline build is aborted by the user.

Aborting the Workflows of a failed Stage

By default, if a Workflow in a particular Stage fails, the other Workflows in the same Stage aren’t automatically aborted: these Workflows will run but the next Stage won’t start. However, you can change this behavior to immediately and automatically abort all other Workflows in the same Stage.

To do so, you need to set the abort_on_fail attribute to true:

stages:
  stage-abort-on-fail-1:
    abort_on_fail: true
    workflows:
    - deploy-1: {}
    - deploy-2: {}

Sharing Env Vars between Pipeline Stages

You can reuse any environment variable from a Workflow and reuse it in subsequent Stages using the Share Pipeline variablesStep.

Optional Workflows using run_if conditions

You can easily combine the Share Pipeline variables Step with run_if expressions to create Pipelines with optional Workflows. For more information, check out Setting up run_if conditions for optional Workflows using Pipelines.Setting up run_if conditions for optional Workflows using Pipelines

To do so:

  1. Add the Share Pipeline variables Step to the Workflow.

  2. Optionally, you can define additional run conditions in the Additional run conditions input. The Step will only run if the conditions you specify here are true.

  3. Add the Env Var(s) you would like to use in subsequent Stages in the Variables to share between Pipeline Stages input.

    Using environment variable keys

    You can define Env Vars using a {key}={value} syntax. For example, MY_ENV_KEY=value, or INSTALL_PAGE_URL=$BITRISE_PUBLIC_PAGE_URL.

    If you want to use the default environment variable keys, you can use a shorthand syntax. For example, EXISTING_ENV_KEY.

    Sharing Env Vars using this Step does not override existing Env Vars defined in the app.

That's it! You can now use the Env Var in any subsequent Stage!

Currently supported use cases for the iOS platform

We have prepared some Pipeline recipes based on common iOS use cases. These contain some of the most frequent tasks our users need to run.

(iOS) Run tests in parallel on multiple simulators

Description

This example uses the sample-swift-project-with-parallel-ui-test iOS Open Source sample app, which has some example Unit and UI tests and uses Test Plans to group the tests.

The example Pipeline config showcases how to run all the test cases of the project on different iOS simulators.

run_tests_on_simulators Pipeline runs two Stages sequentially:

  1. build_tests stage that runs the build_tests Workflow. This Workflow git clones the sample project and runs the xcode-build-for-test Step to build the target and associated tests. The built test bundle is transferred to the next Stage (run_tests_on_simulators) via the deploy-to-bitrise-io Step.

    The build test bundle is compressed

    xcode-build-for-test Step compresses the built test bundle and moves the generated zip to the $BITRISE_DEPLOY_DIR. That directory’s content is deployed to the Workflow artifacts by default via the deploy-to-bitrise-io Step.

    Artifact file size limitation

    There is no limitation on the number of files deployed to Artifacts per build. There is a limitation, however, on the file size which is 2GB per file.

  2. run_tests_on_simulators Stage runs three Workflows in parallel: run_tests_iPad, run_tests_iPhone, and run_tests_iPod. Both of these Workflows use the new xcode-test-without-building Step, which executes the tests based on the previous stage built test bundle. The pre-built test bundle is pulled by the _pull_test_bundle utility Workflow.

    iOS_example_modified.png

Instructions

To test the configuration in a new Bitrise example project, do the following:

  1. Visit the Create New App page to create a new App.

  2. When prompted to select a git repository, choose Other/Manual and paste the sample project repository URL (https://github.com/bitrise-io/sample-swift-project-with-parallel-ui-test) in the Git repository (clone) URL field.

  3. Confirm that this is a public repository in the resulting pop-up.

  4. Select the master branch to scan.

  5. Wait for the project scanner to complete.

  6. Select any of the offered Distribution methods (for example development, it does not really matter as now we are focusing on testing).

  7. Confirm the offered stack, skip choosing the app icon and the webhook registration and kick off the first build.

  8. Open the new Bitrise project’s Workflow Editor.

  9. Go to the bitrise.yml tab and replace the existing bitrise.yml with the contents of the example bitrise.yml file.

  10. Click the Start/Schedule a Build button, and select the run_tests_on_simulators option in the “Workflow, Pipeline” dropdown menu at the bottom of the popup.

bitrise.yml

GitHub link: https://github.com/bitrise-io/workflow-recipes/blob/main/recipes/ios-run-tests-in-parallel-on-multiple-simulators.md#bitriseyml

(iOS) Run test groups in parallel

Description

This example uses the sample-swift-project-with-parallel-ui-test iOS Open Source sample app, which has some example Unit and UI tests and uses Test Plans to group the tests.

XCode test plans

Xcode Test Plans provide a way to run a collection of tests with different test configurations. raywenderlich.com has a great tutorial on how to get started with Xcode Test Plans.

The example Pipeline config showcases how to run different test groups in parallel.

run_tests_groups Pipeline runs two Stages sequentially:

  1. build_tests Stage that runs the build_tests Workflow. This Workflow git clones the sample project and runs the xcode-build-for-test Step to build the target and associated tests. The built test bundle is transferred to the next Stage (run_tests_groups) via the deploy-to-bitrise-io Step.

  2. run_tests_groups Stage runs two Workflows in parallel: run_ui_tests and run_unit_tests. Both of these Workflows use the new xcode-test-without-building Step, which executes the tests based on the previous Stage built test bundle. The pre-built test bundle is pulled by the _pull_test_bundle utility Workflow.

    iOS_example_2.png

Instructions

  1. Visit the Create New App page to create a new App.

  2. When prompted to select a git repository, choose Other/Manual and paste the sample project repository URL (https://github.com/bitrise-io/sample-swift-project-with-parallel-ui-test) in the Git repository (clone) URL field.

  3. Confirm that this is a public repository in the resulting pop-up.

  4. Select the master branch to scan.

  5. Wait for the project scanner to complete.

  6. Select any of the offered Distribution methods (for example development, it does not really matter as now we are focusing on testing).

  7. Confirm the offered stack, skip choosing the app icon and the webhook registration and kick off the first build.

  8. Open the new Bitrise project’s Workflow Editor.

  9. Go to the bitrise.yml tab and replace the existing bitrise.yml with the contents of the example bitrise.yml file.

  10. Click the Start/Schedule a Build button, and select the run_tests_groups option in the “Workflow, Pipeline” dropdown menu at the bottom of the popup.

bitrise.yml

GitHub link: https://github.com/bitrise-io/workflow-recipes/blob/main/recipes/ios-run-test-groups-in-parallel.md#bitriseyml

(iOS) Merging test results and deploying to the Test Reports add-on

Description

Test Reports add-on is tied to Bitrise builds. To make all the test reports generated in different builds appear on a single page in the add-on, the reports must be merged and deployed in an additional build.

This example uses the sample-swift-project-with-parallel-ui-test iOS Open Source sample app and extends the ‘Run iOS test groups in parallel’ example Pipeline config with merging and deploying test results.

run_ui_tests and run_unit_tests Workflows are extended with a deploy-to-bitrise-io Step to make the generated test results available for the next Stage.

run_tests_groups Pipeline is extended with a new Stage: deploy_test_results. This Stage runs the deploy_test_results Workflow:

  1. artifact-pull Step downloads all the zipped test results previously generated by Stage: run_tests_groups.

  2. script Step unzips each test result into a new test run directory within the Test Report add-on deploy dir and creates the related test-info.json file.

  3. deploy-to-bitrise-io Step deploys the merged test results.

    iOS_example_3.png

Instructions

  1. Visit the Create New App page to create a new App.

  2. When prompted to select a git repository, choose Other/Manual and paste the sample project repository URL (https://github.com/bitrise-io/sample-swift-project-with-parallel-ui-test) in the Git repository (clone) URL field.

  3. Confirm that this is a public repository in the resulting pop-up.

  4. Select the master branch to scan.

  5. Wait for the project scanner to complete.

  6. Select any of the offered Distribution methods (for example development, it does not really matter as now we are focusing on testing).

  7. Confirm the offered stack, skip choosing the app icon and the webhook registration and kick off the first build.

  8. Open the new Bitrise project’s Workflow Editor.

  9. Go to the bitrise.yml tab and replace the existing bitrise.yml with the contents of the extended example bitrise.yml file.

  10. Click the Start/Schedule a Build button, and select the run_tests_groups option in the “Workflow, Pipeline” dropdown menu at the bottom of the popup.

  11. Open the Pipeline’s build page.

  12. Select the deploy_test_results build.

  13. Click on Details & Add-ons on the build details page and select the Test Reports add-on to view the merged test reports.

bitrise.yml

GitHub link: https://github.com/bitrise-io/workflow-recipes/blob/main/recipes/ios-merging-test-results-and-deploying-to-the-test-reports-add-on.md#bitriseyml

Currently supported use cases for the Android platform

We have prepared some Pipeline recipes based on common Android use cases. These contain some of the most frequent tasks our users need to run. The examples include entire Workflows that can be copied and pasted for the most part.

(Android) Run UI tests in parallel on multiple devices or shards

Description

Running the UI (instrumentation) tests of a single module in parallel Workflows utilizing pipelines. You can run the tests in parallel by shards or by devices.

The Pipeline contains two Stages that are run serially:

  1. build_for_ui_testing: This Stage executes a Workflow — also named build_for_ui_testing — that runs the android-build-for-ui-testing Step to build APKs for use in testing, and runs the deploy-to-bitrise-io Step to save those APKs for use in the later Stages. Performing this Stage separately from the actual testing allows for each test Stage to use these pre-built APKs rather than having to rebuild them for each test Stage.

  2. run_ui_tests_on_devices: This Stage executes three UI test Workflows in parallel — ui_test_on_phone, ui_test_on_tablet, ui_test_on_foldable — which use the android-instrumented-test Step to run the UI tests on the APKs built in the previous Workflow on each specific device type.

    android_example.png

Instructions

To test this configuration in a new Bitrise example project, do the following:

  1. Visit the Create New App page to create a new App.

  2. When prompted to select a git repository, choose Other/Manual and paste the sample project repository URL (https://github.com/bitrise-io/Bitrise-Android-Modules-Sample.git) in the Git repository (clone) URL field.

  3. Confirm that this is a public repository in the resulting pop-up.

  4. Select the main branch to scan.

  5. Wait for the project scanner to complete.

  6. Enter app as the specified module.

  7. Enter debug as the specified variant.

  8. Continue through the prompts as normal — no changes are needed.

  9. Open the new Bitrise project’s Workflow Editor.

  10. Go to the bitrise.yml tab, and replace the existing yaml contents with the contents of the example bitrise.yml.

  11. Click the Start/Schedule a Build button, and select the ui_test_on_multiple_devices option in the Workflow, Pipeline dropdown menu at the bottom of the popup.

bitrise.yml

GitHub link: https://github.com/bitrise-io/workflow-recipes/blob/main/recipes/android-parallel-ui-tests-on-multiple-devices.md#bitriseyml

(Android) Running unit and UI tests in parallel

Description

Run unit tests and UI tests in parallel utilizing Pipelines.

This Pipeline contains one Stage — stage_unit_and_ui_test — that executes two Workflows in parallel:

  1. unit_tests: This Workflow simply runs the unit tests of the given module and variant using the android-unit-test Step.

  2. ui_tests: This Workflow builds the given module and variant using the android-build-for-ui-testing Step, spins up an emulator using the avd-manager Step, waits for the emulator to boot using the wait-for-android-emulator Step, and runs the UI tests using the android-instrumented-test Step.

    android_example2.png

Instructions

  1. Visit the Create New App page to create a new App.

  2. When prompted to select a git repository, choose Other/Manual and paste the sample project repository URL (https://github.com/bitrise-io/Bitrise-Android-Modules-Sample.git) in the Git repository (clone) URL field.

  3. Confirm that this is a public repository in the resulting pop-up.

  4. Select the main branch to scan.

  5. Wait for the project scanner to complete.

  6. Enter app as the specified module.

  7. Enter debug as the specified variant.

  8. Continue through the prompts as normal — no changes are needed.

  9. Open the new Bitrise project’s Workflow Editor.

  10. Go to the bitrise.yml tab, and replace the existing yaml contents with the contents of the example bitrise.yml.

  11. Click the Start/Schedule a Build button, and select the pipeline_unit_and_ui_test option in the Workflow, Pipeline dropdown menu at the bottom of the popup.

bitrise.yml

GitHub link: https://github.com/bitrise-io/workflow-recipes/blob/main/recipes/android-parallel-unit-and-ui-tests.md#bitriseyml

(Android) Unit test sharding by module

Description

Run the unit tests of a modularized app in parallel Workflows utilizing Pipelines.

This Pipeline contains one Stage — stage_unit_test — that executes two Workflows in parallel:

  1. unit_test_app: This Workflow runs the unit tests of the app module using the android-unit-test Step.

  2. unit_test_library: This Workflow runs the unit tests of the lib-example module using the android-unit-test Step.

    android_example_3.png

Instructions

  1. Visit the Create New App page to create a new App.

  2. When prompted to select a git repository, choose Other/Manual and paste the sample project repository URL (https://github.com/bitrise-io/Bitrise-Android-Modules-Sample.git) in the Git repository (clone) URL field.

  3. Confirm that this is a public repository in the resulting pop-up.

  4. Select the main branch to scan.

  5. Wait for the project scanner to complete.

  6. Enter app as the specified module.

  7. Enter debug as the specified variant.

  8. Continue through the prompts as normal — no changes are needed.

  9. Open the new Bitrise project’s Workflow Editor.

  10. Go to the bitrise.yml tab, and replace the existing yaml contents with the contents of the example bitrise.yml.

  11. Click the Start/Schedule a Build button, and select the pipeline_unit_test option in the Workflow, Pipeline dropdown menu at the bottom of the popup.

bitrise.yml

GitHub link: https://github.com/bitrise-io/workflow-recipes/blob/main/recipes/android-parallel-testing-unit-test-shards.md#bitriseyml

In this section:

A Step is a block of script execution that encapsulates a build task on Bitrise: the code to perform that task, the inputs and parameters you can define for the task, and the outputs the task generates.

A Workflow is a collection of Steps, Environment Variables, and other configurations. When Bitrise starts a build, it runs one or more Workflows according to the configuration defined in the bitrise.yml file.

A Stage is a collection of Workflows. Stages are the top-level building blocks of Pipelines. A Stage can contain multiple Workflows, which all run in parallel in the same Stage. If all Workflows are successful in a Stage, the Pipeline moves on to the next Stage. If any of the Workflows fail, the Pipeline ends without running the other Stages unless you configure a given Stage to always run.

A Bitrise Pipeline is the top level of the Bitrise CI/CD configuration. Pipelines can be used to organize the entire CI/CD process and to set up advanced configurations with multiple different tasks running parallel and/or sequentially.

The bitrise.yml file stores your entire build configuration for an app. It specifies your stack and the build triggers, and defines the Workflows of the app. When you make changes on the graphical UI of our Workflow Editor, you actually modify the bitrise.yml file.

appapp