Trainer fails when Scan fails

[fastlane-community]

Trainer fails when Scan fails

[KrauseFx]

Are you sure this wasn't always the case? The only change that was recently applied was 002f5d6

[bdrelling]

Actually no, I'm not confident that this is recent. Looking through my Fastfile history I had broken trainer out into another lane previously that was actually being left before reaching it, so it's entirely possible the issue has been around for longer.

This is a bug in trianer though, right?

[bdrelling]

@KrauseFx I just wanted to bring this back up since it seems like a bug. I have the same problem with it failing and have to catch the exception in order for everything to work. I run the scan followed directly by trainer, then it fails and shows the little 💥 mark in the fastlane summary next to trainer having run for 0ms. Bottom of script says:

[22:03:24]: fastlane finished with errors

[!] Unit tests failed
script returned exit code 1

My workaround in the meantime is to just do:

begin
      trainer
rescue => exception
end

And everything actually works fine.

[vytis]

We are also experiencing the same issue. It seems that even when fail_build: true is set (as seen from output), the whole build is flagged as failure and fastlane test exits with status 1:

[15:15:38]: ------------------
[15:15:38]: --- Step: scan ---
[15:15:38]: ------------------

< Xcode output omitted>

** TEST EXECUTE FAILED **
[15:16:20]: Exit status: 65
+--------------------+----+
|      Test Results       |
+--------------------+----+
| Number of tests    | 11 |
| Number of failures | 1  |
+--------------------+----+

[15:16:20]: ---------------------
[15:16:20]: --- Step: trainer ---
[15:16:20]: ---------------------

+------------------+---------------------------------------------------------------------------+
|                                  Summary for trainer 0.8.0                                   |
+------------------+---------------------------------------------------------------------------+
| output_directory | test                                                                      |
| fail_build       | false                                                                     |
| path             | DerivedData/Logs/Test/Test-Wire-iOS-2018.10.04_15-15-56-+0200.xcresult/T  |
|                  | estSummaries.plist                                                        |
| extension        | .xml                                                                      |
+------------------+---------------------------------------------------------------------------+

[15:16:20]: `Helper.gem_path` is deprecated. Use the `ROOT` constant from the appropriate tool module instead.
Successfully generated 'test/TestSummaries.xml'
+----------------------------+---------------------------------------------------------------------------+
|                                              Lane Context                                              |
+----------------------------+---------------------------------------------------------------------------+
| DEFAULT_PLATFORM           | ios                                                                       |
| PLATFORM_NAME              | ios                                                                       |
| LANE_NAME                  | ios test                                                                  |
| SCAN_DERIVED_DATA_PATH     | DerivedData                                                               |
| SCAN_GENERATED_PLIST_FILES | ["DerivedData/Logs/Test/Test-Wire-iOS-2018.10.04_15-15-56-+0200.xcresult  |
|                            | /TestSummaries.plist"]                                                    |
| SCAN_GENERATED_PLIST_FILE  | DerivedData/Logs/Test/Test-Wire-iOS-2018.10.04_15-15-56-+0200.xcresult/T  |
|                            | estSummaries.plist                                                        |
+----------------------------+---------------------------------------------------------------------------+
[15:16:20]: Unit tests failed

+------+------------------+-------------+
|           fastlane summary            |
+------+------------------+-------------+
| Step | Action           | Time (in s) |
+------+------------------+-------------+
| 1    | opt_out_usage    | 0           |
| 2    | default_platform | 0           |
| 3    | scan             | 41          |
| 💥   | trainer          | 0           |
+------+------------------+-------------+

[15:16:20]: fastlane finished with errors

[!] Unit tests failed

$> echo $status
1

We are running fastlane 2.105.2 and trainer 0.8.0 with Xcode 10.

[rist]

We have the same issue with 2.108.0 and fail_build set to false

+-------------------------+---------+---------+
|                Used plugins                 |
+-------------------------+---------+---------+
| Plugin                  | Version | Action  |
+-------------------------+---------+---------+
| fastlane-plugin-trainer | 0.4.1   | trainer |
+-------------------------+---------+---------+

.....

+----------------------------+-----------------------------------------------------------------------------------------+
|                                               Summary for scan 2.108.0                                               |
+----------------------------+-----------------------------------------------------------------------------------------+
| devices                    | ["iPhone XS"]                                                                           |
| skip_testing               | ["GeorgeGoAPI-Unit-Tests/AustrianLoginAPITests"]                                        |
| scheme                     | George Go Debug                                                                         |
| fail_build                 | false                                                                                   |
| workspace                  | ./George Go.xcworkspace                                                                 |
| derived_data_path          | /Users/ristd/Library/Developer/Xcode/DerivedData/George_Go-cejywaflmbkysxasspomufxpouav |
| skip_detect_devices        | false                                                                                   |
| clean                      | false                                                                                   |
| skip_build                 | false                                                                                   |
| output_directory           | ./fastlane/test_output                                                                  |
| buildlog_path              | ~/Library/Logs/scan                                                                     |
| include_simulator_logs     | false                                                                                   |
| disable_concurrent_testing | false                                                                                   |
| should_zip_build_products  | false                                                                                   |
| result_bundle              | false                                                                                   |
| open_report                | false                                                                                   |
| skip_slack                 | false                                                                                   |
| slack_only_on_failure      | false                                                                                   |
| use_clang_report_name      | false                                                                                   |
| reinstall_app              | false                                                                                   |
| xcode_path                 | /Applications/Xcode.app                                                                 |
+----------------------------+-----------------------------------------------------------------------------------------+

[15:29:43]: Unit tests failed

+------+--------------------+-------------+
|            fastlane summary             |
+------+--------------------+-------------+
| Step | Action             | Time (in s) |
+------+--------------------+-------------+
| 1    | xcversion          | 0           |
| 2    | clear_derived_data | 15          |
| 3    | scan               | 483         |
| 💥   | trainer            | 0           |
+------+--------------------+-------------+

[15:29:43]: fastlane finished with errors

[!] Unit tests failed