Support for Xcode 11

[fastlane-community]

Support for Xcode 11

[joshdholtz]

@juliangrosshauser Thanks so much for feeding me files needed to make this change 😊 I will merge the PR and get a new release out today at some point 💪🚀

[joshdholtz]

@thibault-ml Looking into! Will hopefully have something for you soon 😊

[juliangrosshauser]

@joshdholtz Here's a .xcresult directory of one of our PSPDFKit test suites. Should contain over 1000 test results:

PSPDFKit.xcresult.zip

Your PR fails with:

bundler: failed to load command: trainer (/Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/bin/trainer)
NoMethodError: [!] undefined method `[]' for nil:NilClass
  /Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/trainer-e6907dbe9186/lib/trainer/xcresult.rb:308:in `initialize'
  /Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/trainer-e6907dbe9186/lib/trainer/xcresult.rb:337:in `new'
  /Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/trainer-e6907dbe9186/lib/trainer/xcresult.rb:337:in `block in initialize'
  /Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/trainer-e6907dbe9186/lib/trainer/xcresult.rb:336:in `map'
  /Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/trainer-e6907dbe9186/lib/trainer/xcresult.rb:336:in `initialize'
  /Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/trainer-e6907dbe9186/lib/trainer/xcresult.rb:238:in `new'
  /Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/trainer-e6907dbe9186/lib/trainer/xcresult.rb:238:in `initialize'
  /Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/trainer-e6907dbe9186/lib/trainer/xcresult.rb:209:in `new'
  /Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/trainer-e6907dbe9186/lib/trainer/xcresult.rb:209:in `initialize'
  /Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/trainer-e6907dbe9186/lib/trainer/xcresult.rb:181:in `new'
  /Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/trainer-e6907dbe9186/lib/trainer/xcresult.rb:181:in `block in initialize'
  /Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/trainer-e6907dbe9186/lib/trainer/xcresult.rb:180:in `map'
  /Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/trainer-e6907dbe9186/lib/trainer/xcresult.rb:180:in `initialize'
  /Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/trainer-e6907dbe9186/lib/trainer/test_parser.rb:141:in `new'
  /Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/trainer-e6907dbe9186/lib/trainer/test_parser.rb:141:in `parse_xcresult'
  /Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/trainer-e6907dbe9186/lib/trainer/test_parser.rb:61:in `initialize'
  /Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/trainer-e6907dbe9186/lib/trainer/test_parser.rb:47:in `new'
  /Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/trainer-e6907dbe9186/lib/trainer/test_parser.rb:47:in `block in auto_convert'
  /Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/trainer-e6907dbe9186/lib/trainer/test_parser.rb:33:in `each'
  /Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/trainer-e6907dbe9186/lib/trainer/test_parser.rb:33:in `auto_convert'
  /Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/trainer-e6907dbe9186/lib/trainer/commands_generator.rb:34:in `block (2 levels) in run'
  /Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/gems/commander-fastlane-4.4.6/lib/commander/command.rb:178:in `call'
  /Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/gems/commander-fastlane-4.4.6/lib/commander/command.rb:153:in `run'
  /Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/gems/commander-fastlane-4.4.6/lib/commander/runner.rb:476:in `run_active_command'
  /Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/gems/fastlane-2.130.0/fastlane_core/lib/fastlane_core/ui/fastlane_runner.rb:76:in `run!'
  /Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/gems/commander-fastlane-4.4.6/lib/commander/delegates.rb:15:in `run!'
  /Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/trainer-e6907dbe9186/lib/trainer/commands_generator.rb:40:in `run'
  /Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/trainer-e6907dbe9186/lib/trainer/commands_generator.rb:10:in `start'
  /Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/bundler/gems/trainer-e6907dbe9186/bin/trainer:6:in `<top (required)>'
  /Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/bin/trainer:23:in `load'
  /Volumes/CI/ci/.asdf/installs/ruby/2.4.1/lib/ruby/gems/2.4.0/bin/trainer:23:in `<top (required)>'

[joshdholtz]

@juliangrosshauser Thanks for sending that over! I had a feeling my PR would need some tweaking for something a little more complex 🙃 I will make sure it works with you sent over ❤️

[juliangrosshauser]

@joshdholtz I tested your newest changes with Objective-C and Swift failures. Looks like everything works now! 🎉 Thank you again!

[joshdholtz]

@vrutberg I think that should work! But you should also be able to do...

scan(
  result_bundle: true,
  output_directory: "./" # optional 
)

[snatchev]

hi @thibault-ml thanks for bringing this up. I will take a look at this once I get some of these JSON files.

[thibault-ml]

Thanks!

Google already parses them:
https://chromium.googlesource.com/chromium/src/+/refs/heads/master/ios/build/bots/scripts/
https://github.com/chromium/chromium/tree/master/ios/build/bots/scripts

Bitrise too: https://github.com/bitrise-steplib/steps-deploy-to-bitrise-io/tree/79d467e68e077eddf8f67277a212e42d0b4cd52a/test/converters/xcresult3

And XCResultKit too: https://github.com/davidahouse/XCResultKit

[joshdholtz]

@thibault-ml Hey! Do you happen to have a xcresult file you could send me? 😊 It would be good to have a legit file that I can test on 🙏

[thibault-ml]

@joshdholtz I'll have a look if we can, not sure 100% I'm at liberty to just yet. Will have a look :-)

[joshdholtz]

@thibault-ml If you don't want to send but are able to run my PR that would also be 💯 😊 The PR works for me on my simple test use cases (I don't have any huge test suites). I got a few fixes to do on it and then will merge.

[joshdholtz]

@juliangrosshauser Got it solved! Thanks sending this 🙌

[joshdholtz]

@juliangrosshauser Alright, PR should be good to go again if you'd like to test when you get some time. I was able to get a successful output from the xcresults that you sent me 👌 I made sure all the data is getting pulled out safely now so that shouldn't happen anymore 😇

[juliangrosshauser]

@joshdholtz Great work! Thank you so much, really appreciate it!

I tested your PR and it works flawlessly now. The only problematic .xcresult directory I could find it this one: PSPDFKit-1-test-failure.zip

It contains 1008 test results. 1007 tests succeed, but the testStyledStringToPDFAsFileURL test fails. The problem is that the resulting XML file contains 1008 successful tests, but no failures.

[joshdholtz]

Oh boy 😱 I found the reason. The testCaseName name in the xcresulttool JSON output is something not as straight forward to match on 🤔 Will work on a solution though

"producingTarget" : {
    "_type" : {
        "_name" : "String"
    },
    "_value" : "PSPDFKitTests"
},
"testCaseName" : {
    "_type" : {
        "_name" : "String"
    },
    "_value" : "-[PSPDFProcessorTests testStyledStringToPDFAsFileURL]"
}

[joshdholtz]

@juliangrosshauser Thanks for testing again! Here is a commit for fixing that 😊 3f88286

[vrutberg]

Thank you guys for working on this!

Is the correct way to use this to specify the -resultBundlePath to xcodebuild, like this, when invoking scan?

xcargs: "-resultBundlePath ./TestSummaries.xcresult"