Comments (39)
Works fine for me. I was using an older version.
from karma-parallel.
I have not attempted to run with code-coverage enabled. It is possible that it is already working, although I highly doubt it.
The https://github.com/rschuft/karma-sharding project was the primary inspiration behind this project. He added code coverage support. You could look at his project and see if it would be possible to implement code coverage in the same way.
You could also try running karma-parallel with code coverage on and post back here the environment / config you used and any errors / inconsistencies in reports generated.
from karma-parallel.
I'm actually looking at it right now and trying to figure out how to fuse the coverage reporter into this project now.
Right now I'm using this with the angular CLI
Everything seems to work surprisingly well except it seems like, the code coverage is under-reporting. It's not completely broken it still reporting back something, just likely not the correct numbers. I thought maybe the values were quartered since I'm using 4 executors, but it doesn't seem to be the case? Branch coverage appears to be 0 in a lot of cases. I'm not really sure what that is about.
In karma.conf.js
I have
frameworks: ['jasmine', '@angular/cli'],
plugins: [
require('karma-parallel'),
require('karma-jasmine'),
require('karma-chrome-launcher'),
require('karma-jasmine-html-reporter'),
require('karma-coverage-istanbul-reporter'),
require('karma-istanbul-threshold'),
require('@angular/cli/plugins/karma'),
require('karma-spec-reporter')
],
parallelOptions:{
executors: 4,
shardStrategy: 'round-robin'
},
reporters: config.angularCli && config.angularCli.codeCoverage
? ['progress', 'coverage-istanbul', 'istanbul-threshold', 'spec']
: ['progress', 'kjhtml'],
I forked the repo and started making changes here https://github.com/BustyLoli-Chan/karma-parallel
I tried to mimick the code coverage set up from karma sharding in here
https://github.com/BustyLoli-Chan/karma-parallel/blob/master/lib/framework.js
but it seems like the way the browsers are stored/tracked differs between the projects?
I'm also currently using istanbul-threshold to report coverage. I don't know if the coverage isn't working or if I have istanbul-threshold set up in a way that is incompatible.
Thank you for your help!
from karma-parallel.
So when I look at the karma debug output I do see that with karma-parallel
24 01 2018 17:18:15.995:DEBUG [reporter.coverage-istanbul]: Writing coverage reports: [ 'html', 'lcovonly', 'json' ]
24 01 2018 17:18:23.920:DEBUG [reporter.coverage-istanbul]: Writing coverage reports: [ 'html', 'lcovonly', 'json' ]
24 01 2018 17:18:37.717:DEBUG [reporter.coverage-istanbul]: Writing coverage reports: [ 'html', 'lcovonly', 'json' ]
24 01 2018 17:18:50.867:DEBUG [reporter.coverage-istanbul]: Writing coverage reports: [ 'html', 'lcovonly', 'json' ]
Happens four times whereas without it only happens once without running karma-parallel
24 01 2018 17:25:03.320:DEBUG [reporter.coverage-istanbul]: Writing coverage reports: [ 'html', 'lcovonly', 'json' ]
I probably need a way to get them to only write the combined data once.
from karma-parallel.
Hmm, interesting. I merged your changes into a test branch and added the karma-coverage
reporter and it seemed to work. This notably has a lot less moving parts since its not using angular / webpack. Do you want to try it out?
https://github.com/joeljeske/karma-parallel/commits/example/coverage-impl
It looks like the copied reporter from karma-sharding is hard-coded to use karma-coverage and not anything else like karma-istanbul-threshold or what-have-you. (https://github.com/BustyLoli-Chan/karma-parallel/blob/master/lib/reporter.js#L31) We might need to make this configurable somehow. Or perhaps even better, we can "merge" the coverage results before they ever get to the standard karma coverage interface so we can be compatible with any karma coverage reporter
from karma-parallel.
Sure, I'd love to pull down the test branch and look at it!
I kind of figured something like that just from the way it's included in index.js, but I wasn't 100% sure
https://github.com/BustyLoli-Chan/karma-parallel/blob/master/lib/index.js#L5
Yeah, the last suggestion definitely sounds ideal, although to be honest I'm not entirely sure how one would do that. I don't know a lot about karma or it's moving parts (maybe this would take place in fullConfig somewhere?)
from karma-parallel.
Alright so I gave it a shot but it seems like it's still failing.
From everything I looked into it seems like I guess it looks like coverage-istanbul
is the default coverage reporter for @angular/cli
projects
There's actually even an option
coverageIstanbulReporter: {
combineBrowserReports: true,
But it doesn't seem to have any effect as I'm still seeing the files get written four times and now I'm seeing four separate blocks of low coverage errors for each of them.
I'll probably try messing around and seeing if it's as easy as targeting coverage-istanbul
instead of just coverage
from karma-parallel.
We have already integrated karma-parallel into our build process and lost coverage support as well, but we check the coverage locally by running tests without karma-parallel. I hope it will be added soon and it is nice to see active discussion here.
karma-sharding coverage doesn't work with CLI as was discussed here rschuft/karma-sharding#1. All the hopes for karma-parallel :-).
from karma-parallel.
Okay I've worked on this a lot recently
I've made some changes to how code coverage is exported so that it basically replaces not just the typical karma coverage
but will also replace the coverage-istanbul
reporter or can be referenced with coverage-parallel
. It would be nice if there was a way to check if we are referencing coverage-parallel
, because then we could just use that and not overwrite coverage
or coverage-istanbul
if for some reason users want (potentially messed up?) coverage results not from the parallel framework. I tried to do this, but I guess we don't have access to rootConfig
from inside index.js
. If you know a way around this I'd like to know it.
I kind of simplified the output coverage dir (it no longer includes the browser name) and I changed it so that I can specify an output dis as follows (I didn't really understand what it was doing before, so maybe this was a change for the worst).
coverageReporter: {
reporters: [
{ type: 'html' },
{ type: 'json' }
],
dir: './coverage'
},
If you understand what it was doing before, or if it's needed for compatibility with certain karma configurations it would be good to revert it.
Like this I have it writing reports and almost properly working (it seems four of my files no longer meet code coverage that they seem to have met with istanbul-coverage before?) but more than anything I'm just happy it's writing html and json reports that require('karma-istanbul-threshold'),
can read.
I also have an issue on output where it says TypeError: Cannot read property 'text' of undefined
which traces back to istanbul at node_modules\istanbul\lib\report\html.js:242:52
if (type === 'no') {
if (endLine !== startLine) {
endLine = startLine;
endCol = structuredText[startLine].text.originalLength();
Where I've noticed that structuredText[startLine]
is undefined because structuredText
is an array with only 10 elements. So there might be some mis-match with how our reporter is writing and how instanbul is interpreting this...
Some thoughts:
I don't know the differences between how karma-coverage
and coverage-istanbul
write reports... I wonder if there's a way to pull the source reporter from istanbul, and modify it like the karma-coverage one we have and then choose either or depending on the situation. Maybe that will make everything play nicely 100%.
I made the changes in a branch here. Sorry for the novel. Please let me know your thoughts.
https://github.com/BustyLoli-Chan/karma-parallel/tree/example/coverage-impl
from karma-parallel.
Thanks for all your help on this! I've update my branch to reflect a couple of changes in how the coverages are aggregated. Currently, I am aggregating them at the karma-reporter api level. This means I am not changing the coverage reporter code at all, or relying on them to implement the aggregation. Instead, I am wrapping their reporter instance in a fake reporter. This aggregate fake reporter takes the events from karma regarding each browser instance, and reports events to the real reporter under a fake browser id. After very limited testing, it appears to work. I would like to test it out more heavily in different configurations and with different coverage reporters, but I am pretty confident in this approach and think it should work.
If you have time, please go through and give me your thoughts on the changes, and test it out in various configurations and let me know how it works.
from karma-parallel.
I looked at it and gave it a shot... but I'm currently just getting the error
30 01 2018 11:45:35.692:ERROR [reporter]: Can not load "parallel-coverage"!
Error: No provider for "baseReporterDecorator"! (Resolving: webServer -> reporter -> reporter:coverage-istanbul -> baseReporterDecorator)
I don't know if this a side effect of require('karma-coverage-istanbul-reporter'),
even if I exclude it from there and exclude coverage-istanbul
from the reporters list in the CLI the angular CLI still requires it and gets upset
from karma-parallel.
Ah yes, i've been doing the bulk of my testing with karma-coverage instead of karma-coverage-istanbul-reporter. I wasn't fully complying to the karma-reporter api. I fixed that and it appears to work on a vanilla ng-cli project now. Hopefully it works for you too
from karma-parallel.
I've tried again and now I'm getting the error
31 01 2018 10:07:54.709:ERROR [reporter]: Can not load "parallel-coverage"!
RangeError: Maximum call stack size exceeded
at Injector.get (Angular-App\node_modules\di\lib\injector.js:25:21)
at Injector.get (Angular-App\node_modules\di\lib\injector.js:54:19)
at Injector.get (Angular-App\node_modules\di\lib\injector.js:54:19)
at Injector.get (Angular-App\node_modules\di\lib\injector.js:54:19)
at Injector.get (Angular-App\node_modules\di\lib\injector.js:54:19)
at Injector.get (Angular-App\node_modules\di\lib\injector.js:54:19)
at Injector.get (Angular-App\node_modules\di\lib\injector.js:54:19)
at Injector.get (Angular-App\node_modules\di\lib\injector.js:54:19)
at Injector.get (Angular-App\node_modules\di\lib\injector.js:54:19)
at Injector.get (Angular-App\node_modules\di\lib\injector.js:54:19)
at Injector.get (Angular-App\node_modules\di\lib\injector.js:54:19)
at Injector.get (Angular-App\node_modules\di\lib\injector.js:54:19)
at Injector.get (Angular-App\node_modules\di\lib\injector.js:54:19)
at Injector.get (Angular-App\node_modules\di\lib\injector.js:54:19)
at Injector.get (Angular-App\node_modules\di\lib\injector.js:54:19)
at Injector.get (Angular-App\node_modules\di\lib\injector.js:54:19)
that's if I include 'parallel-coverage'
as a reporter. If I don't include it I don't really get any errors, but it is clear that code coverage still gets written 5 times and looking through the HTML shows that a lot of files are missing coverage (it seems the same as if the reporters were never combined)
from karma-parallel.
Ahh yes, I probably have a bug in the reporter filter thats causing the max call stack. But the reporter is automatically added to the list and should not need to be explicitly specified.
As for the writing out 5 times, I'm not sure. It would really help if I could have access to your project or a sample project that experiences this issue. Do you have it pubic somewhere?
from karma-parallel.
Sadly I don't :(
I'm working for a company releasing commercial software so I do my best to obfuscate even all posted code samples. I can try to create a dummy project with a few tests to see if it works the same.
looking at the HTML I can still see that a lot of files look like they are missing coverage...
This is from the latest code on the example/coverage-impl
branch, should I be using master?
31 01 2018 11:16:49.185:DEBUG [reporter.coverage-istanbul]: Writing coverage reports: [ 'html', 'json' ]
31 01 2018 11:16:54.375:DEBUG [reporter.coverage-istanbul]: Writing coverage reports: [ 'html', 'json' ]
31 01 2018 11:16:58.117:DEBUG [reporter.coverage-istanbul]: Writing coverage reports: [ 'html', 'json' ]
31 01 2018 11:17:01.807:DEBUG [reporter.coverage-istanbul]: Writing coverage reports: [ 'html', 'json' ]
31 01 2018 11:17:04.649:DEBUG [karma]: Run complete, exiting.
31 01 2018 11:17:04.650:DEBUG [launcher]: Disconnecting all browsers
31 01 2018 11:17:04.756:WARN [launcher]: ChromeHeadless was not killed in 15000 ms, sending SIGKILL.
Waiting for remapped coverage source timed out…
31 01 2018 11:17:04.760:DEBUG [launcher]: Process ChromeHeadless exited with code 0
31 01 2018 11:17:04.761:DEBUG [temp-dir]: Cleaning temp dir Angular-App\AppData\Local\Temp\karma-25698132
31 01 2018 11:17:04.799:DEBUG [launcher]: Finished all browsers
it seems like it should also be throwing errors... based on line coverage and my settings
istanbulThresholdReporter: {
src: './coverage/coverage-final.json',
reporters: ['text'],
thresholds: {
global: {
statements: 0,
branches: 0,
lines: 75,
functions: 0
},
each: {
statements: 0,
branches: 0,
lines: 75,
functions: 0
}
}
},
although if the coverage source is timing out, I guess it explains why this isn't exploding.
from karma-parallel.
Can you post a gist of the karma config perhaps?
from karma-parallel.
// Karma configuration file, see link for more information
// https://karma-runner.github.io/0.13/config/configuration-file.html
module.exports = function (config) {
config.set({
basePath: '',
frameworks: ['parallel', 'jasmine', '@angular/cli'],
plugins: [
require('karma-parallel'),
require('karma-jasmine'),
require('karma-chrome-launcher'),
require('karma-jasmine-html-reporter'),
require('karma-istanbul-threshold'),
require('karma-coverage-istanbul-reporter'),
require('@angular/cli/plugins/karma'),
require('karma-spec-reporter')
],
parallelOptions:{
executors: 4,
shardStrategy: 'round-robin'
},
client:{
clearContext: false // leave Jasmine Spec Runner output visible in browser
},
files: [
{ pattern: './src/test.ts', watched: false },
'./node_modules/jquery/dist/jquery.min.js'
],
preprocessors: {
'./src/test.ts': ['@angular/cli']
},
mime: {
'text/x-typescript': ['ts','tsx']
},
istanbulThresholdReporter: {
src: './coverage/coverage-final.json',
reporters: ['text'],
excludes: [],
thresholds: {
global: {
statements: 0,
branches: 0,
lines: 75,
functions: 0
},
each: {
statements: 0,
branches: 0,
lines: 75,
functions: 0
}
}
},
coverageIstanbulReporter: {
reports: [ 'html', 'json' ],
fixWebpackSourcePaths: true
},
angularCli: {
environment: 'dev'
},
specReporter: {
maxLogLines: 9999,
suppressErrorSummary: false,
suppressFailed: false,
suppressPassed: false,
suppressSkipped: true,
showSpecTiming: false
},
reporters: ['progress', 'kjhtml', 'istanbul-threshold', 'spec'],
port: 9876,
colors: true,
logLevel: config.LOG_DEBUG,
autoWatch: true,
browsers: ['ChromeHeadless'],
singleRun: false,
captureTimeout: 210000, // it was already there
browserDisconnectTimeout: 210000,
browserDisconnectTolerance: 3,
browserNoActivityTimeout: 210000,//by default 10000
processKillTimeout: 15000
});
};
from karma-parallel.
What version of the ng-cli did you use to create your project?
from karma-parallel.
Ok here are some things I have found:
-
The msg "Waiting for remapped coverage source timed out…" is from istanbul threshold. Its strategy is to watch the output file from istanbulThresholdReporter.src. After the run completes, it watches the file and reads it to check coverage levels. I am not sure why it cannot detect the file change event after the reporter seems to write it multiple times. But that is also why it does not throw on your coverage thresholds.
-
I cannot reproduce multiple write statements from
reporter.coverage-istanbul
. I'm wondering if we are on different versions of the ng-cli. I started a new project and changed the karma-config to match yours and do not see multiple writes. I did notice that 'coverage-istanbul' is not listed as a reporter in your karma conf. Does the ng cli add it dynamically? It is possible that on your version of the cli, it happens to add it after I look for it in my framework.js files and thus my reporter is not correctly aggregating it. Not sure..
Thoughts?
from karma-parallel.
Right now I'm on version 1.6
This is a giant nightmare of packages from package.json
"dependencies": {
"@angular/animations": "^5.1.0",
"@angular/common": "^5.1.0",
"@angular/compiler": "^5.1.0",
"@angular/core": "^5.1.0",
"@angular/forms": "^5.1.0",
"@angular/http": "^5.1.0",
"@angular/platform-browser": "^5.1.0",
"@angular/platform-browser-dynamic": "^5.1.0",
"@angular/router": "^5.1.0",
"@angular/upgrade": "^5.1.0",
"@ngrx/core": "~1.2.0",
"@ngrx/store": "~4.1.1",
"chart.js": "~2.6.0",
"core-js": "^2.4.1",
"enhanced-resolve": "^3.4.1",
"font-awesome": "~4.7.0",
"immutable": "~3.8.1",
"moment": "2.18.1",
"moment-timezone": "0.5.13",
"mydatepicker": "2.0.0",
"net": "^1.0.2",
"ngx-bootstrap": "~2.0.0-beta.2",
"ngx-filter-pipe": "1.0.0",
"ngx-infinite-scroll": "~0.5.1",
"ngx-pagination": "~3.0.0",
"primeng": "^5.0.2",
"quill": "^1.3.0",
"rxjs": "^5.5.5",
"stompjs": "^2.3.3",
"web-animations-js": "^2.3.1",
"zone.js": "^0.8.18"
},
"devDependencies": {
"@angular/cli": "^1.6.0",
"@angular/compiler-cli": "^5.0.0",
"@angular/language-service": "^5.0.0",
"@ngrx/store-devtools": "~4.1.1",
"@types/bootstrap": "3.3.33",
"@types/jasmine": "^2.5.54",
"@types/jasminewd2": "~2.0.2",
"@types/jquery": "2.0.41",
"@types/node": "~6.0.60",
"@types/stompjs": "~2.3.0",
"@types/webpack": "^3.0.13",
"@types/webpack-env": "^1.13.0",
"angular2-template-loader": "^0.6.2",
"awesome-typescript-loader": "^3.1.3",
"codelyzer": "^4.0.1",
"concurrently": "^3.4.0",
"gulp": "^3.9.1",
"gulp-inline-ng2-template": "^4.0.0",
"istanbul-instrumenter-loader": "^3.0.0",
"jasmine-core": "^2.8.0",
"jasmine-spec-reporter": "^4.2.1",
"jquery": "3.2.1",
"json-loader": "^0.5.4",
"karma": "^2.0.0",
"karma-chrome-launcher": "^2.2.0",
"karma-cli": "^1.0.1",
"karma-coverage-istanbul-reporter": "^1.3.3",
"karma-istanbul-threshold": "^1.2.2",
"karma-jasmine": "^1.1.0",
"karma-jasmine-html-reporter": "^0.2.2",
"karma-parallel": "^0.1.2",
"karma-spec-reporter": "^0.0.31",
"ngrx-store-freeze": "~0.2.0",
"raw-loader": "^0.5.1",
"rimraf": "^2.6.1",
"source-map-explorer": "^1.3.3",
"to-string-loader": "^1.1.5",
"ts-node": "~3.2.0",
"tslint": "~5.7.0",
"typescript": "~2.4.2",
"webpack-angular-externals": "^1.0.2",
"webpack-rxjs-externals": "^1.1.0"
}
I think the ng cli is hard coded to require coverage-istanbul
but I did have it included before explicitly (I think I forgot to re-add it when trying different configurations). I'll re-add it and see what kind of results I get.
from karma-parallel.
Making this change I have still noticed no difference...
I did notice my stuff didn't seem up to date though (might have been me being dumb the whole time)
I did update and now I'm seeing this
01 02 2018 10:12:34.015:ERROR [karma]: TypeError: Cannot read property 'name' of undefined
at handleBrowserRegister (Angular-App\Repos\karma-parallel\lib\framework.js:45:33)
at Server.emitter.on (Angular-App\Repos\karma-parallel\lib\framework.js:85:52)
at emitOne (events.js:121:20)
at Server.emit (events.js:211:7)
at Browser.init (Angular-App\Repos\second-angular\node_modules\karma\lib\browser.js:79:13)
at Socket.<anonymous> (Angular-App\Repos\second-angular\node_modules\karma\lib\server.js:309:20)
at emitOne (events.js:116:13)
at Socket.emit (events.js:211:7)
at Angular-App\Repos\second-angular\node_modules\karma\node_modules\socket.io\lib\socket.js:513:12
at _combinedTickCallback (internal/process/next_tick.js:131:7)
at process._tickCallback (internal/process/next_tick.js:180:9)
from karma-parallel.
Shoot. I fixed that last one but forgot to push it up. It fixes "Cannot read property 'name'..."
Just pushed.
from karma-parallel.
Alright, I'll try it
I'm still ending up with this as my end result... even when I completely remove the coverage folder and can still clearly see that it is being written from scratch/nothing
01 02 2018 16:53:41.428:DEBUG [reporter.coverage-istanbul]: Writing coverage reports: [ 'html', 'json' ]
TOTAL: 1273 SUCCESS
01 02 2018 16:53:45.771:DEBUG [reporter.coverage-istanbul]: Writing coverage reports: [ 'html', 'json' ]
01 02 2018 16:53:49.746:DEBUG [reporter.coverage-istanbul]: Writing coverage reports: [ 'html', 'json' ]
01 02 2018 16:53:54.280:DEBUG [reporter.coverage-istanbul]: Writing coverage reports: [ 'html', 'json' ]
01 02 2018 16:53:58.746:DEBUG [reporter.coverage-istanbul]: Writing coverage reports: [ 'html', 'json' ]
01 02 2018 16:54:02.194:DEBUG [karma]: Run complete, exiting.
01 02 2018 16:54:02.195:DEBUG [launcher]: Disconnecting all browsers
01 02 2018 16:54:02.297:WARN [launcher]: ChromeHeadless was not killed in 15000 ms, sending SIGKILL.
Waiting for remapped coverage source timed out…
this is still without including 'parallel-coverage'
reporters: ['progress', 'kjhtml', 'coverage-istanbul', 'istanbul-threshold', 'spec'],
in karma.conf.js
trying to include it still gives me the old
01 02 2018 16:56:22.591:ERROR [reporter]: Can not load "parallel-coverage"!
RangeError: Maximum call stack size exceeded
at Injector.get (Angular-App\node_modules\di\lib\injector.js:25:21)
at Injector.get (Angular-App\node_modules\di\lib\injector.js:54:19)
at Injector.get (Angular-App\node_modules\di\lib\injector.js:54:19)
at Injector.get (Angular-App\node_modules\di\lib\injector.js:54:19)
at Injector.get (Angular-App\node_modules\di\lib\injector.js:54:19)
at Injector.get (Angular-App\node_modules\di\lib\injector.js:54:19)
at Injector.get (Angular-App\node_modules\di\lib\injector.js:54:19)
at Injector.get (Angular-App\node_modules\di\lib\injector.js:54:19)
at Injector.get (Angular-App\node_modules\di\lib\injector.js:54:19)
at Injector.get (Angular-App\node_modules\di\lib\injector.js:54:19)
at Injector.get (Angular-App\node_modules\di\lib\injector.js:54:19)
at Injector.get (Angular-App\node_modules\di\lib\injector.js:54:19)
at Injector.get (Angular-App\node_modules\di\lib\injector.js:54:19)
at Injector.get (Angular-App\node_modules\di\lib\injector.js:54:19)
at Injector.get (Angular-App\node_modules\di\lib\injector.js:54:19)
at Injector.get (Angular-App\node_modules\di\lib\injector.js:54:19)
01 02 2018 16:56:22.593:DEBUG [karma]: Registered a load error of type reporter with name parallel-coverage
from karma-parallel.
Ok, I was finally able to reproduce your issue and I think I've tracked it down:
As I had suspected, ng-cli was adding the 'coverage-istanbul' reporter after my plugin was wrapping the reporters. This led to the coverage reporter not getting properly wrapped. I have moved the logic to remove the reporters from the framework to the middleware. This executes a bit later and is able to intercept reporters added by other frameworks.
Also, I fixed the max call stack issue.
At your leisure, please give it another shot. Thanks for your help in testing this!
from karma-parallel.
Should I still explicitly include parallel-coverage
I have included it, but it doesn't seem to break anything. It seems like this is working.
I've also tested that it passes and fails, in several cases where it should. It looks like it's working great! This is awesome. Thank you for your hard work in putting all of this together!
Do you think you'll release a beta version of this soon or anything? I would love to actually start making use of this in our CLI system
from karma-parallel.
It does not matter if you include it or not. It will automatically add it as needed. (For that reason, it will not be in the docs. Most people won't ever know about it.)
I'm glad its finally working! I've benefited a lot from the community and I've finally had a good chance to try to give back.
Yes, I will cleanup and release a version here soon.
from karma-parallel.
Awesome! I'm glad everything is working too.
Thank you so much again! I look forward to the release version!
from karma-parallel.
Fixed in 0.2.0
from karma-parallel.
@joeljeske in yarn it is still available 3 prev versions only, 0.2.0 is not available yet. It is issue in my local environment only or you have the same?
`>yarn upgrade [email protected]
yarn upgrade v1.3.2
[1/4] Resolving packages...
Couldn't find any versions for "karma-parallel" that matches "0.2.0"
? Please choose a version of "karma-parallel" from this list: (Use arrow keys)
0.1.2
0.1.1
0.1.0`
from karma-parallel.
That’s odd. I was under the impression that yarn was nothing more than a npm client; meaning there is not an additional registry in which I must publish. Maybe try blowing the cache and/or doing a clean yarn add?
from karma-parallel.
@joeljeske yes, today morning it started working :-). It looks like it was some local cache issue or smth like this. Thanks for the reply :-).
I am confirming it just works, CC report is good one! But what I noticed during my testing is the following: when ng test is finished the process doesn't return control, it is still hanging and as a result next steps of the build process can't start; after waiting of 1 hour it is cancelled. If I run ng test command not as a part of build process, but manually the result is the same: tests finished, CC report is created but the console is still opened and control is still not returned; I need to close console manually. It is not a working solution for automated build process where after successful tests should be run next steps.
In the prev version of the package 0.1.2 it did worked as expected with the same configuration: tests run, CC report was created but for one browser instance only, control was returned.
Here is how I run ng test manually:
ng test --watch=false --code-coverage=true --single-run=true --progress=true --colors=true --log-level=debug --reporters=progress,junit,coverage-istanbul --config=karma.conf.headless.js
Here is my karma.conf.headless.js config:
// Karma configuration file, see link for more information
// https://karma-runner.github.io/1.0/config/configuration-file.html
module.exports = function (config) {
config.set({
basePath: '',
frameworks: ['parallel', 'jasmine', '@angular/cli'],
parallelOptions: {
executors: 4, // Defaults to cpu-count - 1
shardStrategy: 'round-robin'
// shardStrategy: 'description-length'
},
plugins: [
require('karma-jasmine'),
require('karma-chrome-launcher'),
require('karma-jasmine-html-reporter'),
require('karma-coverage-istanbul-reporter'),
require('karma-junit-reporter'),
require('@angular/cli/plugins/karma'),
require('karma-parallel')
],
client: {
clearContext: false // leave Jasmine Spec Runner output visible in browser
},
coverageIstanbulReporter: {
reports: ['html', 'lcovonly', 'cobertura'],
fixWebpackSourcePaths: true,
// Most reporters accept additional config options. You can pass these through the `report-config` option
'report-config': {
// all options available at: https://github.com/istanbuljs/istanbuljs/blob/aae256fb8b9a3d19414dcf069c592e88712c32c6/packages/istanbul-reports/lib/html/index.js#L135-L137
html: {
// outputs the report in ./coverage/html
subdir: 'html'
}
},
thresholds: {
// thresholds for all files
global: {
statements: 95,
lines: 95,
branches: 73,
functions: 95
},
}
},
angularCli: {
environment: 'dev'
},
files: [
],
reporters: ['progress', 'kjhtml'],
port: Math.floor((Math.random() * 500) + 9500),
colors: true,
logLevel: config.LOG_INFO,
autoWatch: false,
browsers: ['ChromeHeadless'],
singleRun: true,
concurrency: Infinity,
customLaunchers: {
ChromeHeadless: {
base: 'Chrome',
flags: [
'--headless',
'--disable-gpu',
// Without a remote debugging port, Google Chrome exits immediately.
`--remote-debugging-port=${Math.floor((Math.random() * 500) + 9000)}`,
'--no-sandbox'
// '--max_old_space_size=4096'
],
debug: false
}
},
// How long does Karma wait for a browser to reconnect (in ms).
// With a flaky connection, it is pretty common that the browser disconnects, but the actual test execution is still running without any problems.
// Karma does not treat a disconnection as an immediate failure and will wait for browserDisconnectTimeout (ms).If the browser reconnects during that time, everything is fine.
// Default: 2000
browserDisconnectTimeout: 6000,
// The number of disconnections tolerated.
// The disconnectTolerance value represents the maximum number of tries a browser will attempt in the case of a disconnection.
// Usually, any disconnection is considered a failure, but this option allows you to define a tolerance level when there is a flaky network link between the Karma server and the browsers.
// Default: 0
browserDisconnectTolerance: 3,
// How long will Karma wait for a message from a browser before disconnecting from it (in ms).
// If, during test execution, Karma does not receive any message from a browser within browserNoActivityTimeout(ms), it will disconnect from the browser.
// Default: 10000
browserNoActivityTimeout: 60000,
// Timeout for capturing a browser (in ms).
// The captureTimeout value represents the maximum boot-up time allowed for a browser to start and connect to Karma.
// If any browser does not get captured within the timeout, Karma will kill it and try to launch it again and, after three attempts to capture it, Karma will give up.
// Default: 60000
captureTimeout: 120000,
// Karma will report all the tests that are slower than given time limit (in ms).This is disabled by default (since the default value is 0).
reportSlowerThan: 1000
// More info about disconnected issue is here:
// https://github.com/karma-runner/karma-chrome-launcher/issues/154
// https://github.com/karma-runner/karma/issues/2652
});
};
from karma-parallel.
Could you open a new issue?
from karma-parallel.
@joeljeske done #5
from karma-parallel.
I was trying to run my test parallel karma-parallel but not getting the consolidated junit and code coverage report.
My karma.config.js has
parallelOptions: {
executors: 4, // Defaults to cpu-count - 1
shardStrategy: 'round-robin',
// shardStrategy: 'description-length'
aggregatedReporterTest: '/junit/i'
}
Can anyone help me with sample karma.config.js code to get the consolidated reports.
from karma-parallel.
It looks like you are specifying a string instead of a regex here
aggregatedReporterTest: '/junit/i'
instead of
aggregatedReporterTest: /junit/i
from karma-parallel.
@joeljeske Thanks, now i am getting consolidated junit report.
Similar way i tried to get code coverage report but noticed its not executing all the test .Also not able to identify whether code coverage report is consolidated or not.
Karma.config:
parallelOptions: {
executors: 4, // Defaults to cpu-count - 1
shardStrategy: 'round-robin',
// shardStrategy: 'description-length'
aggregatedReporterTest: /coverage/i
},
plugins: [
require('karma-jasmine'),
require('karma-chrome-launcher'),
require('karma-jasmine-html-reporter'),
require('karma-coverage-istanbul-reporter'),
require('@angular/cli/plugins/karma'),
require('karma-htmlfile-reporter'),
require('karma-junit-reporter'),
require('karma-phantomjs-launcher'),
require('karma-parallel')
],
reporters: config.angularCli && config.angularCli.codeCoverage
? ['progress', 'coverage-istanbul']
: ['progress', 'kjhtml', 'html', 'junit'],
run ng test
ng test --single-run=true --browser=ChromeHeadless --code-coverage=true
output:
junit: here all test cases were executed
Executed 122 of 122 SUCCESS (11 mins 1.151 secs / 11 mins 1.016 secs)
HeadlessChrome 63.0.3239 (Windows 7.0.0): Executed 101 of 101 (1 FAILED) (8 mins 53.288 secs / 8 mins 53.141 secs)
HeadlessChrome 63.0.3239 (Windows 7.0.0): Executed 121 of 121 (1 FAILED) (11 mins 1.914 secs / 11 mins 1.771 secs)
HeadlessChrome 63.0.3239 (Windows 7.0.0): Executed 77 of 77 (6 FAILED) (6 mins 33.557 secs / 6 mins 33.016 secs)
TOTAL: 8 FAILED, 413 SUCCESS
TOTAL: 8 FAILED, 413 SUCCESS
TOTAL: 8 FAILED, 413 SUCCESS
code-coverage: not ran all the test and got exit
Executed 67 of 122 SUCCESS (11 mins 21.314 secs / 5 mins 34.993 secs)
HeadlessChrome 63.0.3239 (Windows 7.0.0): Executed 37 of 77 SUCCESS (6 mins 49.287 secs / 2 mins 41.514 secs)
HeadlessChrome 63.0.3239 (Windows 7.0.0): Executed 95 of 121 SUCCESS (11 mins 20.396 secs / 8 mins 29.122 secs)
HeadlessChrome 63.0.3239 (Windows 7.0.0): Executed 64 of 101 SUCCESS (8 mins 42.517 secs / 4 mins 41.518 secs)
TOTAL: 263 SUCCESS
TOTAL: 263 SUCCESS
from karma-parallel.
On version 0.2.9, I'm having the issue that my tests are meeting their required global coverage in normal, non parallel runs, but when I switch to karma-parallel, they are slightly below. Is it possible that I have something configured incorrectly?
from karma-parallel.
+1
from karma-parallel.
If you are having an issue, please create a new bug instead of using a old issue. This ticket was for the initial development of code coverage support.
from karma-parallel.
Related Issues (20)
- karma-parallel do not return exit code 1 anymore on too low code coverage HOT 7
- Exception when using "karma run" HOT 7
- Should fail the test cases split to a browser that is disconnected HOT 4
- Aggregated Log Output HOT 10
- No support for Circle CI HOT 2
- Support Karma 4 HOT 1
- When browser disconnects, remaining tests are skipped HOT 5
- Random failures with uncaught errors and missing tests
- Jasmine test timeout is ignored HOT 1
- Random test failures during CI process
- How to enable running tests in parallel on CI
- Publish new version HOT 2
- No calculation of all passed tests on kubernetis -job
- Script Error while executing karma tests HOT 1
- Only supporting Jasmine ?
- Karma parallel cannot find any tests after starting Chrome
- Need help to get code coverage HOT 3
- `Parallel` throws away global beforeEach in multi-browser mode
- Is there a way to let what files to run in what threads/browser instances ?
- failSpecWithNoExpectations to true in jasmine and karma-parallel add the extra tests HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from karma-parallel.