Exporting to Test Reports from any Step

Test Reports allows you to view all your test results in a convenient way. By default, five Steps support the Test Reports add-on:

  • Xcode Test for iOS

  • Android Unit Test

  • iOS Device Testing

  • Virtual Device Testing for Android

  • Flutter Test

To export the test results generated by any other Step to the Deploy to Bitrise.io Step, you have two options:

Using the Export test results to the Test reports add-on Step

You can use the Export test results to the Test reports add-on Step to make sure your test results appear in the Test Reports add-on, even if you use Steps that don’t automatically export their results.

With the correct configuration, the Step finds the test results in your app’s repository, and puts them in the export directory.

Screenshots and other images

The Step can only export test results, in either .xcresult or JUnit XML format. If your test results are in an .xcresult file, screenshots generated as part of the test are included and exported to the Test Reports add-on.

If you use the JUnit XML format, you need to export screenshots and other images separately: Exporting screenshots and other images to Test Reports.

  1. Go to your app’s page on bitrise.io.

  2. Go to Workflows to open the Workflow Editor.

  3. From the WORKFLOW menu, select the Workflow you use to run your tests.

  4. Add the Export test results to the Test reports add-on Step AFTER the testing Step.

  5. In the The name of the test input, set the name of the test run. The test results will be under this name in the Test Reports add-on.

  6. In the Path where custom test results reside input, set the path where your test results can be found.

    Managing an app's bitrise.yml configuration

    We recommend setting a folder here, though you can also set a specific filepath. The default value is the source directory of your app. Example patterns:

    • If your app’s root directory is app: app/build/test-results/testDemoDebugUnitTest/

    • If your test results are within an app folder but app is not the root directory: ./app/build/test-results/testDemoDebugUnitTest/

  7. In the Test result search pattern input, set a pattern that matches your test result file.

    This search pattern is used to search every file and folder of the path that was set in the Path where custom test results reside input.

    If there is more than one match, the Step will export the first match with a warning in the logs. If you set a specific filepath in the previous input, just set * here.

    Example patterns:

    • Matching all files within the base path: *

    • Matching all files within a given directory of the base path: */build/test-results/testDemoDebugUnitTest/*

  8. In the Step’s test result directory input, make sure the path is correct.

    Do NOT modify this input’s value: this is the folder where the Deploy to Bitrise.io Step will look for the test results to export them. It should be set to the $BITRISE_TEST_RESULT_DIR Env Var.

  9. Make sure you have a Deploy to Bitrise.io Step in your Workflow.

Using custom scripts to export test results

You can use your own scripts in a Script Step to export your test results. You just need to make sure that:

  • Your test results are in a JUnit XML format.

  • You place the results in the $BITRISE_TEST_RESULT_DIR, which is an Environment Variable pointing to the correct path.

  • The results of each test run should be in its own sub-directory. You place a test-info.json file in each sub-directory to correspond to the test run.

The test-info.json file

The test-info.json file must have the name of the test run defined in it. Each test run should have its own test-info.json file. The value of test-name will appear as the name of the test run on the Test Reports page.

  // Test Name ... { "test-name":"My first test" }

Here’s an example script to export your test results:

#!/bin/env bash
set -ex

# Creating a sub-directory for the test run within the BITRISE_TEST_RESULT_DIR:

test_run_dir="$BITRISE_TEST_RESULT_DIR/result_dir_1"
mkdir "$test_run_dir"

# Exporting the JUnit XML test report:

cp "MY/TEST/REPORT/XML/FILE/PATH.xml" "$test_run_dir/UnitTest.xml"

# Creating the test-info.json file with the name of the test run defined:

echo '{"test-name":"MY TEST RUN NAME"}' >> "$test_run_dir/test-info.json"

If all goes well, you should be able to see your test results on the Test Reports page.

Exporting screenshots and other images to Test Reports

The Export test results to Test Reports add-on Step can only export test result files to the Deploy to Bitrise.io Step which supports two formats for results:

If your tests produce an .xcresult file, screenshots generated during the test run are part of the test result. As such, they will be exported to the Test Reports add-on.

In other cases, screenshots and other images aren't automatically exported. However, you can make sure they also end up in the Test Reports add-on. All you need to do is create a script that places the images in the $BITRISE_TEST_RESULT_DIR folder.

Example 1. Script to export an image
#!/bin/env bash
set -ex

# Creating a sub-directory for the test run within the BITRISE_TEST_RESULT_DIR:

test_run_dir="$BITRISE_TEST_RESULT_DIR/result_dir_1"
mkdir "$test_run_dir"

# copying the image:

cp "MY/TEST/SCREENSHOTS/screenshot.png" "$test_run_dir/screenshot.png"

Associating images with test runs

You can associate an image with a specific test run by placing the image in the directory of that test run. In that case, the image will appear under all test suits of the given test run.