Skip to main content

Pipelines with stages

Abstract

Pipelines with stages are the older way of configuring Pipelines. We recommend switching to the new way where you can define Workflow dependencies to create more flexible configurations, and you no longer have to waste time waiting for a stage to finish.

Pipelines with stages are the older way of configuring Pipelines. We recommend switching to the new way where you can define Workflow dependencies to create more flexible configurations, and you no longer have to waste time waiting for a stage to finish.

You can convert a Pipeline with stages into the new format: Converting a Pipeline with stages into a graph Pipeline.

However, if you need to, you can keep using Pipelines with stages. Such Pipelines have three main building blocks:

  • Steps: Blocks of script execution, each defining a single task in a CI/CD process.

  • Workflows: Collections of Steps. When a build of an app is running, the Steps will be executed in the order that is defined in the Workflow.

  • Stages: Collections of Workflows. A Stage can contain multiple Workflows which all run in parallel in the same Stage. If all Workflows are successful in a Stage, the Pipeline moves on to the next Stage. If any of the Workflows fail, the Pipeline ends without running the other Stages unless you configure a given Stage to always run.

Configuring a Pipeline with stages

Configuring a Pipeline that contains stages is only possible by directly editing the bitrise.yml file. You can create and modify Workflows in the graphical Workflow Editor but you need to define Pipelines and Stages in YAML format.

Defining Pipelines with stages

The bitrise.yml file contains the full configuration of your Pipelines. As with all bitrise.yml files, first you need to define the format version and the project type.

---
format_version: '8'
default_step_lib_source: 
project_type: android

This is a bare minimum bitrise.yml configuration. To define your Pipelines, you will need to use the pipelines attribute.

pipelines:  
  pipeline-successful:
    stages:
    - stage-successful-1: {}
    - stage-successful-2: {}
    - stage-successful-3: {}

In this example, we have a Pipeline called pipeline-successful, with three Stages that will run consecutively. This means that if stage-successful-1 finishes successfully, stage-successful-2 starts. If any of the Stages fail, the subsequent Stage will not start: instead, the Pipeline will be aborted and marked as failed.

Each Stage has to be defined separately under the stages attribute. Defining a Stage means specifying the Workflows that are part of the Stage.

stages:
  stage-successful-1:
    workflows:
    - test-1: {}
  stage-successful-2:
    workflows:
    - build-1: {}
    - build-2: {}
  stage-successful-3:
    workflows:
    - deploy-1: {}
    - deploy-2: {}

In this example, the Stages run the test-1, build-1, build-2, deploy-1, and deploy-2 Workflows.

Configuring a Stage to always run

By default, if a Stage fails - because one of its Workflows failed -, any other subsequent Stages of the Pipeline will not run. However, you can configure your Pipeline to run certain Stages unless the Pipeline is aborted.

To do so, you just need to set the should_always_run attribute of the Stage to true:

stages:
  stage-always-run-successful-1:
    should_always_run: true
    workflows:
    - deploy-1: {}
    - deploy-2: {}

In the example above, the Stage called stage-always-run-successful-1 will always run, regardless of the status of previous Stages. The only way these Stages will not run is if the Pipeline build is aborted by the user.

Aborting the Workflows of a failed Stage

By default, if a Workflow in a particular Stage fails, the other Workflows in the same Stage aren’t automatically aborted: these Workflows will run but the next Stage won’t start. However, you can change this behavior to immediately and automatically abort all other Workflows in the same Stage.

To do so, you need to set the abort_on_fail attribute to true:

stages:
  stage-abort-on-fail-1:
    abort_on_fail: true
    workflows:
    - deploy-1: {}
    - deploy-2: {}

Sharing Env Vars between Pipeline Stages

You can reuse any environment variable from a Workflow and reuse it in subsequent Workflows using the Share Pipeline variables Step.

Optional Workflows using run_if conditions

You can easily combine the Share Pipeline variables Step with run_if expressions to create Pipelines with optional Workflows. For more information, check out Setting up run_if conditions for optional Workflows using Pipelines.Setting up run_if conditions for optional Workflows using Pipelines

To do so:

  1. Add the Share Pipeline variables Step to the Workflow.

  2. Optionally, you can define additional run conditions in the Additional run conditions input. The Step will only run if the conditions you specify here are true.

  3. Add the Env Var(s) you would like to use in subsequent Workflows in the Variables to share between Pipeline Workflows input.

    Using environment variable keys

    You can define Env Vars using a {key}={value} syntax. For example, MY_ENV_KEY=value, or INSTALL_PAGE_URL=$BITRISE_PUBLIC_PAGE_URL.

    If you want to use the default environment variable keys, you can use a shorthand syntax. For example, EXISTING_ENV_KEY.

    Sharing Env Vars using this Step does not override existing Env Vars defined in the app.

That's it! You can now use the Env Var in any subsequent Stage!

Configuring a Pipeline with stages

Configuring a Pipeline that contains stages is only possible by directly editing the bitrise.yml file. You can create and modify Workflows in the graphical Workflow Editor but you need to define Pipelines and Stages in YAML format.

Defining Pipelines with stages

The bitrise.yml file contains the full configuration of your Pipelines. As with all bitrise.yml files, first you need to define the format version and the project type.

---
format_version: '8'
default_step_lib_source: 
project_type: android

This is a bare minimum bitrise.yml configuration. To define your Pipelines, you will need to use the pipelines attribute.

pipelines:  
  pipeline-successful:
    stages:
    - stage-successful-1: {}
    - stage-successful-2: {}
    - stage-successful-3: {}

In this example, we have a Pipeline called pipeline-successful, with three Stages that will run consecutively. This means that if stage-successful-1 finishes successfully, stage-successful-2 starts. If any of the Stages fail, the subsequent Stage will not start: instead, the Pipeline will be aborted and marked as failed.

Each Stage has to be defined separately under the stages attribute. Defining a Stage means specifying the Workflows that are part of the Stage.

stages:
  stage-successful-1:
    workflows:
    - test-1: {}
  stage-successful-2:
    workflows:
    - build-1: {}
    - build-2: {}
  stage-successful-3:
    workflows:
    - deploy-1: {}
    - deploy-2: {}

In this example, the Stages run the test-1, build-1, build-2, deploy-1, and deploy-2 Workflows.

Configuring a Stage to always run

By default, if a Stage fails - because one of its Workflows failed -, any other subsequent Stages of the Pipeline will not run. However, you can configure your Pipeline to run certain Stages unless the Pipeline is aborted.

To do so, you just need to set the should_always_run attribute of the Stage to true:

stages:
  stage-always-run-successful-1:
    should_always_run: true
    workflows:
    - deploy-1: {}
    - deploy-2: {}

In the example above, the Stage called stage-always-run-successful-1 will always run, regardless of the status of previous Stages. The only way these Stages will not run is if the Pipeline build is aborted by the user.

Aborting the Workflows of a failed Stage

By default, if a Workflow in a particular Stage fails, the other Workflows in the same Stage aren’t automatically aborted: these Workflows will run but the next Stage won’t start. However, you can change this behavior to immediately and automatically abort all other Workflows in the same Stage.

To do so, you need to set the abort_on_fail attribute to true:

stages:
  stage-abort-on-fail-1:
    abort_on_fail: true
    workflows:
    - deploy-1: {}
    - deploy-2: {}

Sharing Env Vars between Pipeline Stages

You can reuse any environment variable from a Workflow and reuse it in subsequent Workflows using the Share Pipeline variables Step.

Optional Workflows using run_if conditions

You can easily combine the Share Pipeline variables Step with run_if expressions to create Pipelines with optional Workflows. For more information, check out Setting up run_if conditions for optional Workflows using Pipelines.Setting up run_if conditions for optional Workflows using Pipelines

To do so:

  1. Add the Share Pipeline variables Step to the Workflow.

  2. Optionally, you can define additional run conditions in the Additional run conditions input. The Step will only run if the conditions you specify here are true.

  3. Add the Env Var(s) you would like to use in subsequent Workflows in the Variables to share between Pipeline Workflows input.

    Using environment variable keys

    You can define Env Vars using a {key}={value} syntax. For example, MY_ENV_KEY=value, or INSTALL_PAGE_URL=$BITRISE_PUBLIC_PAGE_URL.

    If you want to use the default environment variable keys, you can use a shorthand syntax. For example, EXISTING_ENV_KEY.

    Sharing Env Vars using this Step does not override existing Env Vars defined in the app.

That's it! You can now use the Env Var in any subsequent Stage!

In this section:

A Step is a block of script execution that encapsulates a build task on Bitrise: the code to perform that task, the inputs and parameters you can define for the task, and the outputs the task generates.

A Workflow is a collection of Steps, Environment Variables, and other configurations. When Bitrise starts a build, it runs one or more Workflows according to the configuration defined in the bitrise.yml file.

A Stage is a collection of Workflows. Stages are the top-level building blocks of Pipelines. A Stage can contain multiple Workflows, which all run in parallel in the same Stage. If all Workflows are successful in a Stage, the Pipeline moves on to the next Stage. If any of the Workflows fail, the Pipeline ends without running the other Stages unless you configure a given Stage to always run.

A Bitrise Pipeline is the top level of the Bitrise CI/CD configuration. Pipelines can be used to organize the entire CI/CD process and to set up advanced configurations with multiple different tasks running parallel and/or sequentially.

The bitrise.yml file stores your entire build configuration for an app. It specifies your stack and the build triggers, and defines the Workflows of the app. When you make changes on the graphical UI of our Workflow Editor, you actually modify the bitrise.yml file.

appapp