Test Reports
You can run both UI tests and unit tests on Bitrise. You can view them in Test Reports, which you can access from the Add-ons section on the Build Details page.
Test Reports allows you to view all your test results in a convenient way.
Use our testing Steps, and all your test results will be deployed to the same directory. From there, Test Reports can process and display them in a way that’s easy to read and analyze.
Test reports overview
You can run both UI tests and unit tests on Bitrise and with Test Reports, you can view the results of the different test types on the same page.
To be able to use Test Reports, you need to make sure your test results end up in the correct directory and in the correct format. From that directory, the Deploy to Bitrise.io Step exports the results to the add-on.
There are three ways to make sure your test results end up in the correct location and in the appropriate format:
-
Using our dedicated testing Steps that automatically export their results. With these Steps, all you need to do is put both the testing Step and the Deploy to Bitrise.io Step to the Workflow and run the build.
-
Using the Export test results to the Test reports add-on Step.
-
Using a custom Script Step.
Exporting the results from any testing Step
Both the second and the third method allows you to export test results from any Step. Read the details in our Exporting to Test Reports from any Step guide.
Test Reports can read and display test results that are in either of two formats:
You can use Test Reports with five of our testing Steps by default. These Steps will run the tests defined in your repository and then store the output in an Environment Variable. The Deploy to Bitrise.io Step will do the rest: it can read the results and send it to Test Reports. The Steps are:
-
Xcode Test for iOS.
-
Android Unit Test.
-
iOS Device Testing.
-
Virtual Device Testing for Android.
-
Flutter Test.
With these Steps, you don't have to worry about the format or the location of the test results. All you need to make sure is that you include the Deploy to Bitrise.io Step at the end of your Workflow.
Testing guides
Check out our testing guides to read more about how to use these Steps on Bitrise:
Configuring Test Reports
Test Reports doesn’t require any sort of complicated configuration. All you need is to have tests in your repository, and two Steps in your Workflow.
Deploy to Bitrise.io version
The Deploy to Bitrise.io Step must be of version 1.4.1 or newer - older versions of the Step do NOT support Test Reports. We recommend using version 1.5.0 or higher for the best experience.
-
Add tests to your app’s repository.
-
Add the appropriate testing Steps to your Workflow. For example, if you have an Android project, you can use the Android Unit Test Step to run Android unit tests.
-
Make sure you have the Deploy to Bitrise.io Step in your Workflow.
-
Run your build.
And that’s it! Once the build is finished, you will be able to view your test reports.
Checking test results
Test Reports display the most important information about all the tests you ran on the Test Summary tab. You will see all the tests you ran, their duration, and their results. You can also click on the tabs of the individual tests to see their details.
To access the result of a particular test:
-
Open the Bitrise CI page and select your project from the project list.
-
Select the build you want to check out.
-
Go to the Tests tab.
-
You can see failed tests on the Tests tab itself. If you had no failed tests, click the button.
-
On the Test Summary tab, find the test you want to check. Alternatively, on the top bar, click the tab of the test set you want to check out.To filter tests based on their results, open the menu on the top right, which is set to All by default.
-
Click Test cases to see the details.
UI tests are grouped according to device while unit tests are grouped according to test cases.
You will find:
-
The total number of tests you ran, as well as the ratio of successful and failed tests.
-
The duration of each individual test (both unit tests and UI tests).
-
In the case of UI tests, the orientation and the locale of the device.
You can check, depending on the test type:
-
Individual test cases
-
Performance data
-
Videos
-
Screenshots
-
Test artifacts
-
Logs
As noted above, this depends on test type, too: for unit tests, you won’t see screenshots or videos, of course.
Downloading files
Every file can be downloaded from Test Reports: that includes screenshots, videos, or APK and IPA files.
-
Open a test set.
-
On the left menu, click a tab that contains files.
For example, Test Artifacts.
-
Click Download next to the file name.
If you want to download all files available on the tab, click Download All Files.