Connecting your AWS instance to your Bitrise Workspace

Abstract

To successfully launch your instance with the Bitrise AMI on AWS, you need to connect the Bitrise runner pool you added on bitrise.io to the instance, using the token you received during the process. This allows you to run builds on Bitrise using your AWS EC2 instance as your build stack.

You have two ways of connecting to the instance:

Using the AWS Secret Manager and setting up the required scripts during launching the instance.
Connecting to the instance directly, using SSH. We recommend using this method for troubleshooting purposes only.

Using the AWS Secret Manager

Get the token from the process of adding the runner pool on Bitrise.
Create an AWS Manager Secret and store the token in the secret.
Create an IAM role with permission to read the secret and attach it to your EC2 instance.

Modify the User data of the instance: add the command to launch the Bitrise runner, using the Secret you created in the AWS Secret Manager:

Mac instance

Linux instance

For bare metal:

TOKEN=$(aws --region ${Region} secretsmanager get-secret-value --secret-id MY_SECRET | jq -r '.SecretString')

sudo sed -i '' “s/BITRISE_AGENT_TOKEN/$TOKEN/” ~/Library/LaunchDaemons/io.bitrise.self-hosted-agent.plist

For virtualized offering, you also need to provide the number of concurrencies and the stack you wish to run:

Available stacks

You can get the list of available stacks with the following command:

/opt/virtualization-cli/bin/virtualization-cli list

From the output, you need the name attribute of the stack.

Available concurrencies

The allowed values for your concurrency preference are 1 or 2.

With 1 CC, the runner will schedule 1 VM with 8 vCPU and 12 GB RAM.
With 2 CC, the runner will schedule 2 VMs with 4 vCPU and 6 GB RAM each.

TOKEN=$(aws secretsmanager get-secret-value --secret-id MY_SECRET | jq  -r '.SecretString')
sudo sed -i '' 's/BITRISE_AGENT_CC/<YOUR_CONCURRENCY_PREFERENCE>/' /Users/ec2-user/Library/LaunchDaemons/io.bitrise.self-hosted-agent.plist
sudo sed -i '' 's/BITRISE_AGENT_STACK/<YOUR_DESIRED_STACK>/' /Users/ec2-user/Library/LaunchDaemons/io.bitrise.self-hosted-agent.plist
sudo sed -i '' “s/BITRISE_AGENT_TOKEN/$TOKEN/” ~/Library/LaunchDaemons/io.bitrise.self-hosted-agent.plist

TOKEN=$(aws secretsmanager get-secret-value --secret-id MY_SECRET | jq  -r '.SecretString')

sudo sed -i “s/BITRISE_AGENT_TOKEN/$TOKEN/” /etc/systemd/system/bitrise-den-agent.service
sudo systemctl start bitrise-den-agent.service

Troubleshooting the instance connection

If you encounter any errors with your EC2 instance, there's a few things to check first:

On your EC2 dashboard, check if the instance has appeared under the EC2 instances. If not, the instance was not launched correctly and it should be launched again.
If the instance has appeared and seems to be stuck in Starting state, wait for a few minutes for the instance to finish the launch.
If the instance has appeared but its state is neither Starting nor Running, it needs to be terminated a new instance started.
If the instance state is Running but the status is Initializing, wait for a few minutes for the instance to finish initialization.
If the instance state is Running and the status is Impaired, the instance needs to be restarted.
If the instance state is Running, the status is Ready but the instance is not visible on your Bitrise Workspace's page, we recommend waiting a few minutes and then, if it's still not visible, proceed to checking the instance's connection to the Workspace.

To check the instance connection to the Workspace, you can connect to your EC2 instance directly, using SSH.

Bare metal Mac instance

Virtualized Mac instance

Linux instance

Make sure that your instance can access the following endpoints:
- https://den.services.bitrise.io
- https://build-log.services.bitrise.io
Without accessing these endpoints, you won't be able to run builds even after connecting the instance.
Get the token from the process of adding the runner pool on Bitrise.

Connect to your instance on AWS and run the following commands on it:

sudo sed -i '' 's/BITRISE_AGENT_TOKEN/<YOUR_AGENT_POOL_TOKEN>/' ~/Library/LaunchDaemons/io.bitrise.self-hosted-agent.plist
sudo launchctl load -w /Users/ec2-user/Library/LaunchDaemons/io.bitrise.self-hosted-agent.plist

Validate that the runner pool is running on the instance:
```
ps aux | grep bitrise-den-agent
```

Make sure that your instance can access the following endpoints:
- https://den.services.bitrise.io
- https://build-log.services.bitrise.io
Without accessing these endpoints, you won't be able to run builds even after connecting the instance.
Get the token from the process of adding the runner pool on Bitrise.

Connect to your instance on AWS and run the following commands on it:

sudo sed -i '' 's/BITRISE_AGENT_TOKEN/<YOUR_AGENT_POOL_TOKEN>/' ~/Library/LaunchDaemons/io.bitrise.self-hosted-agent.plist
sudo sed -i '' 's/BITRISE_AGENT_CC/<YOUR_CONCURRENCY_PREFERENCE>/' /Users/ec2-user/Library/LaunchDaemons/io.bitrise.self-hosted-agent.plist
sudo sed -i '' 's/BITRISE_AGENT_STACK/<YOUR_DESIRED_STACK>/' /Users/ec2-user/Library/LaunchDaemons/io.bitrise.self-hosted-agent.plist
sudo launchctl load -w /Users/ec2-user/Library/LaunchDaemons/io.bitrise.self-hosted-agent.plist

Validate that the runner pool is running on the instance:
```
ps aux | grep bitrise-den-agent
```

Make sure that your instance can access the following endpoints:
- https://den.services.bitrise.io
- https://build-log.services.bitrise.io
Without accessing these endpoints, you won't be able to run builds even after connecting the instance.

Connect to your instance on AWS and run the following commands on it:

sudo sed -i 's/BITRISE_AGENT_TOKEN/<YOUR_AGENT_POOL_TOKEN>/' /etc/systemd/system/bitrise-den-agent.service
sudo systemctl start bitrise-den-agent.service

Validate that the runner pool is running on the instance:

ps aux | grep bitrise-den-agent

or

sudo systemctl status bitrise-den-agent.service

Cleaning up a persistent build environment

Abstract

You can configure the Bitrise CLI to run in agent mode: this allows cleaning up persistent build environments when running builds on self-hosted agents.

On Bitrise, a new virtual machine is created every time a build starts and it is destroyed when the build is finished. You can, however run Bitrise builds in a persistent build environment: for example, you can use our on-premise runner or run builds on an AWS EC2 instance. In such an environment, the products of one build might affect subsequent builds.

On self-hosted infrastructure, one Bitrise runner executes multiple builds. This allows sharing data between builds on the local filesystem, but it also requires care in order to avoid one build affecting another.

To avoid the problem, you can configure the Bitrise CLI to run in agent mode. Agent mode requires placing an agent-config.yml in the host machine's ~/.bitrise/ directory. In the file, you can specify which directories to clean up when starting a new build or at the end of a build. It also allows you to run your own custom scripts if you have a more advanced use case than a simple cleanup.

Configuring agent mode

Add the agent-config.yml file to your ~/.bitrise/ directory.
If you wish to configure the exact folders in which you want to do cleanup operations, define a bitrise_dirs property. The property overrides the default settings: for example, you can set a different path for the BITRISE_DEPLOY_DIR environment variable than the default.

Under bitrise_dirs, define the directories that you wish to perform some action on at the start or at the end of builds in a KEY: path format. For example, you can define separate directories for all source code checkout, deployable artifacts, and deployable test result artifacts.

# Customize the common Bitrise directories
bitrise_dirs:
  # Root directory for all Bitrise data produced at runtime
  BITRISE_DATA_HOME_DIR: /opt/bitrise
  
  # Directory for source code checkout.
  BITRISE_SOURCE_DIR: /opt/bitrise/workspace/$BITRISE_APP_SLUG
  
  # Directory for deployable artifacts.
  BITRISE_DEPLOY_DIR: /opt/bitrise/$BITRISE_APP_SLUG/$BITRISE_BUILD_SLUG/artifacts
  
  # Directory for deployable test result artifacts.
  BITRISE_TEST_DEPLOY_DIR: /opt/bitrise/$BITRISE_APP_SLUG/$BITRISE_BUILD_SLUG/test_results
  
  # Directory for the html reports
  BITRISE_HTML_REPORT_DIR: /opt/bitrise/$BITRISE_APP_SLUG/$BITRISE_BUILD_SLUG/html_reports

Multiple projects of the same Workspace

Don’t forget that multiple projects of the same Workspace could run on the same agent. Make sure to always include $BITRISE_APP_SLUG in the directory hierarchy to separate projects and their source code checkouts.

# wrong:
BITRISE_SOURCE_DIR: /opt/bitrise/workspace
# correct:
BITRISE_SOURCE_DIR: /opt/bitrise/$BITRISE_APP_SLUG/workspace

Add a hooks property to the agent-config.yml file. This property will define the actions to perform at the start or end of builds. It has four different parameters:
- cleanup_on_build_start: Defines a directory that is cleaned up when a new build is started.
- cleanup_on_build_end: Defines a directory that is cleaned up whenever a build is finished. Not guaranteed to run in the case of build failure.
- do_on_build_start: Defines a custom script that runs when a new build is started.
- do_on_build_end: Defines a custom script that runs when a build is finished. Not guaranteed to run in the case of build failure.
Nested Workflows

A Script Step in a Workflow can execute bitrise run nested_workflow and trigger a nested workflow. This nested Workflow inherits all the envs and parameters of the parent Workflow, and the parent Workflow waits for the completion of the nested Workflow. When the nested Workflow is launched this way, hooks and directory cleanups are not executed in this process to avoid unexpected behavior.
To test the agent mode, start a build. The log should show the following message at the top:
```
Running in agent mode
Config file: .bitrise/agent-config.yml
```

Common configuration examples

Example 1. Fully isolated builds

To minimize state and maximize reliability, each build is assigned unique directories. Source code is checked out from scratch in each build.

bitrise_dirs:
  BITRISE_DATA_HOME_DIR: /opt/bitrise
  BITRISE_SOURCE_DIR: /opt/bitrise/$BITRISE_APP_SLUG/$BITRISE_BUILD_SLUG/workspace
  BITRISE_DEPLOY_DIR: /opt/bitrise/$BITRISE_APP_SLUG/$BITRISE_BUILD_SLUG/artifacts
  BITRISE_TEST_DEPLOY_DIR: /opt/bitrise/$BITRISE_APP_SLUG/$BITRISE_BUILD_SLUG/test_results

hooks:
  # Since dirs are unique to each build, there is nothing to clean up here:
  cleanup_on_build_start: []
  
  # Clean up everything after the build ends:
  cleanup_on_build_end:
    - $BITRISE_SOURCE_DIR
    - $BITRISE_DEPLOY_DIR
    - $BITRISE_TEST_DEPLOY_DIR

Example 2. Shared source code directory for warm checkouts

For faster source code checkouts, it’s possible to reuse a previous build’s source code directory.

bitrise_dirs:
  BITRISE_DATA_HOME_DIR: /opt/bitrise
  # Use a warm clone of the repo for all builds:
  BITRISE_SOURCE_DIR: /opt/bitrise/$BITRISE_APP_SLUG/workspace
  # For artifacts and test results, it's still a good idea to place them into unique dirs
  BITRISE_DEPLOY_DIR: /opt/bitrise/$BITRISE_APP_SLUG/$BITRISE_BUILD_SLUG/artifacts
  BITRISE_TEST_DEPLOY_DIR: /opt/bitrise/$BITRISE_APP_SLUG/$BITRISE_BUILD_SLUG/test_results

hooks:
  # Since dirs are unique to each build, there is nothing to clean up here:
  cleanup_on_build_start: []
  
  # Optional: clean up artifacts and test results.
  # Note that $BITRISE_SOURCE_DIR is NOT clean up here!
  cleanup_on_build_end:
    - $BITRISE_DEPLOY_DIR
    - $BITRISE_TEST_DEPLOY_DIR

In this section:

Connecting your AWS instance to your Bitrise Workspace

Using the AWS Secret Manager

Available stacks

Available concurrencies

Troubleshooting the instance connection

Cleaning up a persistent build environment

Configuring agent mode

Multiple projects of the same Workspace

Nested Workflows

Common configuration examples

Search results