To run Xcode tests on Bitrise, you need to have test targets defined in your projects. You need two Steps to run Xcode tests and view their results:
- Xcode Test for iOS.
- Deploy to Bitrise.io.
The Xcode Test for iOS step runs the pre-defined Xcode tests. It has a default configuration that will work if the tests are written correctly. You can find the same configuration options in Xcode, too.
You can also generate code coverage files, and export the test results as a compressed zip file.
Configuring the Xcode tests on Bitrise ⚓
The default input values of the Xcode Test for iOS Step could work if your tests are written correctly. However, we recommend making sure that all the inputs have the value you want so the Step can do what you need!
Check the following required inputs of the Xcode Test for iOS Step before running a build. These inputs determine the sort of tests the Step will run.
- Scheme name: the scheme you use must be marked as Shared in Xcode!
- Device: set it to the value that is shown in Xcode’s device selection dropdown menu.
- OS version: set it to the value that is shown in Xcode’s device selection dropdown menu.
- Platform: set it to the value that is shown in Xcode’s device selection dropdown menu.
Rerunning failed tests (Not available in Xcode 13+) ⚓
The Should retry test on failure? (Not available in Xcode 13+) input in the Debug section of the Xcode Test for iOS Step allows you to automatically rerun ALL your tests, not just the failed ones. If you set this input to
yes, the Step will run
xcodebuild one more time in the case of test failure.
From Xcode 13 and above the Should retry test on failure? (Not available in Xcode 13+) feature is not available anymore. We recommend you use Test Repetitions Mode (Available in Xcode 13+) input with the
retry_on_failure option selected. This allows you to rerun only the failed test/s instead of running all your tests. You can find this test repetition feature with our Xcode Test for iOS Step from version 3.0.0 and above.
Test Repetitions ⚓
Xcode’s test repetition modes are available with the Xcode Test for iOS Step on stacks running Xcode 13 and above on Bitrise. With test repetitions, you can run any type of tests multiple times in various ways such as retry on failure, until failure and until max repetitions.
The Xcode Test for iOS Step’s Test Repetitions Mode (Available in Xcode 13+) input offers the following options:
none: The tests won’t repeat.
until_failure: Repeats a test until the test fails or until the maximum repetition. The default number of test runs is three.
retry_on_failure: Failed tests run until they succeed or until the repetition number you specify. The default number of test repetitions is three.
up_until_maximum_repetitions: Reruns all tests until maximum test repetition is reached regardless of the test outcomes.
Please note that these options are only available from Xcode Test for iOS version 3.0.0 and above.
With the Maximum Test Repetitions (Available in Xcode 13+) input you can specify the maximum number of test repetitions. Please note that you have to add a greater number than one if the Test Repetition Mode (Available in Xcode 13+) input is set to other than
Enable the Relaunch Tests for Each Repetition (Available in Xcode 13+) input to launch tests in a completely new process for each repetition.
Deploy an app to Bitrise to be able to download the .ipa file and install it on devices specified in the app's Development Provisioning Profile. This way, your internal testers...
You can build and deploy your iOS application to a simulator, to show it off in a browser, for example. On Bitrise, we have a dedicated Step to build for...
With Bitrise's iOS device testing solution, you can run iOS tests on physical devices without having to set up and register your own devices: you just need to use our...