Comments (12)
Improving the performance of the conversion step is a blocker for us, not an "enhancement" (as currently tagged). It takes over 10 minutes to process our xcresult bundle on a CircleCI runner. Adding codecov would thus increase our test jobs runtime by 50%. We were considering starting a paid plan and give us your money, but as is we can't.
Happy to provide you with a sample xcresult (through a private channel).
from uploader.
Hopefully this isn't just viewed as piling on, but given that it's been six months since there's any update, I'm hoping there's some concrete on the roadmap for this. The canny.io link from @drazisil-codecov above doesn't appear to work - I'm presented with a sign-in prompt even when signed up.
We're evaluating Codecov and have it set up to upload xcresult
bundles with the --xcode --xcodeArchivePath [XCRESULT]
options, which appears to work, but it takes a long time to complete the Running xcode coversion...
[sic] part of things.
As potential customers, what can we expect here? I'm a little disappointed and surprised by what I've encountered so far:
- It's non-obvious that
--xcode
is required when--xcodeArchivePath
is specified - maybe I'm missing something, but the uploader doesn't do anything expected when you just try passing--xcodeArchivePath
without the--xcode
flag. Instead the uploader finds an unrelated fastlane report and then complains about it being invalid. Shouldn't the presence of--xcodeArchivePath
alone be enough to make the uploader try to use that? - A typo in the
Running xcode coversion
log statement - it makes me wonder if this feature is well tested and supported. I get it, typos happen, but that particular line of code has been in the uploader for nearly a year and not fixed - not confidence inspiring. - It takes as long to convert and upload the coverage as it does to build and run the entire test suite in CI.
I'm really not trying to just complain here, but as is, Codecov doesn't feel useable for us and it's honestly pretty surprising since I have successfully used Codecov in the past. The conversion and upload time is a major blocker. We could somewhat modularize the jobs and have the coverage conversion and upload kicked off later in the pipeline so developers can get their test results faster, then see coverage sometime after that, but this doesn't feel like something we should have to do - ideally the Codecov upload would be fairly quick, as quick as I've seen it in the past.
Given the lack of traction (unless I'm missing something) I'm inclined to say we will probably have to rule out Codecov, but I'd love to be proven wrong here. Do you have customers using the new uploader on non-trivial iOS apps?
from uploader.
We've since moved away from Codecov and CircleCI. Our project increased in complexity such that the Codecov conversion step was taking almost double the time it took to build and run our test suite, so half the cost we were paying for CircleCI was for Codecov's slow uploader alone. Completely unacceptable. We're now using Xcode Cloud and Xcode's native test coverage reports, as Codecov seems entirely uninteresting in solving this problem.
from uploader.
@thomasrockhu-codecov that's a big improvement. Results from time ./codecov -t [REDACTED] --xs fastlane/test_output/[REDACTED].xcresult
from uploader.
Copied to Canny as a feature request: https://codecov.canny.io/feedback/p/native-support-for-xcode-test-coverage-xcresult-format
@codecov/product
from uploader.
@james-rantmedia @jeffremer @bobergj, I'm not an xcode developer, but I'm interested in taking a crack at trying to work on this. Do you have an open-source project that is running into this issue?
from uploader.
@james-rantmedia @jeffremer @bobergj, I'm not an xcode developer, but I'm interested in taking a crack at trying to work on this. Do you have an open-source project that is running into this issue?
Thanks for the reply, but no @thomasrockhu-codecov - our project is proprietary.
from uploader.
@jeffremer, thanks for getting back so quickly. Would you be able to share how you are collecting coverage?
from uploader.
We have code coverage enabled in a XCTest plan for all the pertinent targets and the coverage information ends up being stored in the resulting xcresult
bundle. This is viewable directly in Xcode, but I've also used third party tools such as xcov
to generate HTML reports. xcov
only takes a few seconds to generate a coverage report from an xcresult
bundle.
from uploader.
@jeffremer, I made some changes to the uploader (v0.4.0
). Can you try adding --xs
to the codecov
command (e.g. ./codecov --xs
from uploader.
@bobergj this might be helpful for you too. Any numbers about how long it takes would be helpful too
from uploader.
@jeffremer woo hoo! glad that it helped!
from uploader.
Related Issues (20)
- Release 0.6.1
- Do not upload test reports to Codecov
- Codecov uploader fails on Linux ARM HOT 2
- Feature Request
- Sent usd
- your_title
- Feature Request
- `CircleCI` orb stuck at "Searching for coverage files..." HOT 20
- This is a third-party issue.
- The uploader should retry on `ETIMEDOUT` but it doesn't
- CI Support: SemaphoreCI
- Codecov uploader hangs at the end of uploading. HOT 8
- [Feature Request] support organization's global token HOT 4
- Ignore CI option in the yml does not seem to work HOT 4
- your_title HOT 1
- Carryforward not working HOT 2
- Bitbucket pipeline upload fail when git it not available HOT 2
- Troubles with global upload token HOT 2
- `spawn Unknown system error -86` with uploader 0.8.0 (via codecov-action) HOT 4
- macOS binary is not x86_64 but arm64 HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from uploader.