Comments (20)
There is no way to handle direct call to die() or exit() in a tested class.
The only solution is to use a dedicated public method to stop the script and mock it in the test.
from atoum.
Just to be sure, I don't want to test the call to die(), I want to prevent someone to call a die() somewhere and still my CI is green.
from atoum.
But how can you return a global Success status though the status line prints 3/5 ?
from atoum.
"Success" means "no fail, no error and no exception".
Currently, it does not mean that all test methods was executed.
from atoum.
Maybe adding a "--strict" option a la Behat, where skipped test returns Error could solve this issue ?
from atoum.
Yep I see.
When you say "Currently", do you mean that you consider it as a bug (I do) ?
In fact there can be multiple exit scenarii that you cannot intercept like fatal errors, that lead to such a case.
from atoum.
When you say "Currently", do you mean that you consider it as a bug (I do) ?
Not sure that it's a bug, see below.
In fact there can be multiple exit scenarii that you cannot intercept like fatal errors, that lead to such a case.
atoum can handle fatal error and it's a failure.
atoum can also handle segfault without any problem, but it's not a failure, because it's a PHP problem (bug in the Zend Engine), not a code problem (not a bug in the tested class).
I think that it's a semantic problem.
What means exactly "SUCCESS" ?
In my current opinion, "SUCCESS" = 0 fail, 0 error, 0 exception.
Segfault is a problem, but it's not a test failure.
Direct call to die() or exit() are in fact similar to segfault : the php script stop, and it's not a test failure.
So, perhaps there is not enough visibility in the report about the fact that some test methods stop "abnormaly".
try this :
It's a failure ? Not sure...
Currently, atoum say :
atoum version DEVELOPMENT by Frédéric Hardy (/Users/fch/Atoum/repository)
PHP path: /usr/local/bin/php
PHP version:
=> PHP 5.3.8 (cli) (built: Sep 21 2011 23:14:37)
=> Copyright (c) 1997-2011 The PHP Group
=> Zend Engine v2.3.0, Copyright (c) 1998-2011 Zend Technologies
=> with Xdebug v2.1.1, Copyright (c) 2002-2011, by Derick Rethans
mageekguy\atoum\tests\units\adapter...
[.___________________________________________________________][0/1]
=> Test duration: 0.01 second.
=> Memory usage: 0.00 Mb.
Total test duration: 0.01 second.
Total test memory usage: 0.00 Mb.
Code coverage value: 0.00%
=> Class mageekguy\atoum\adapter: 0.00%
==> mageekguy\atoum\adapter::__call(): 0.00%
==> mageekguy\atoum\adapter::invoke(): 0.00%
Running duration: 0.16 second.
Success (1 test, 1 method, 0 assertion, 0 error, 0 exception) !
from atoum.
According to me, Success means : every test passed.
One test that did not run is a failure. What is the purpose of this test if when it does not run, the global status is still ok ?
I write tests for them to be run :)
from atoum.
I write tests for them to be run :)
But it can't run with a die() or a exit() in it...
So, if there is a die() in it, it's certainly because you don't want run it, so it's not a failure.
from atoum.
if all tests was run and passed => Success
if one or more tests wasn't run or not passed => Fail (or Error, or Warning, or whatever)
from atoum.
In fact we put a die in it to debug something. And we saw that the test still pass. That's why we opened this issue in the first place :D
The test should have failed. That surprised us it did not.
from atoum.
Currently, a die() in a test method is equivalent to an empty test method.
And an emtpy test method is not a failure.
So i have a problem, because this case is like the schrodinger cat : it may be failure, or not.
from atoum.
I see your problem :)
But what is the point of an empty test method ? According to me it should be considered as a failure.
from atoum.
But what is the point of an empty test method ? According to me it should be considered as a failure.
Why ?
No assert = no failure = success.
from atoum.
No assert = useless = failure :)
Different point of view :D
from atoum.
No assert = useless = failure :)
Different point of view :D
Yes, just need to know if all users are the same point of view than you.
FYI, It's not a problem to change this behaviour.
Just need to add a "status" in the score to handle these cases (segfault, direct call to die()/exit(), empty test method).
Currently, there are "fails", "errors" and "exceptions" in score.
But i have no idea for the name of the new status.
If you have a suggestion...
from atoum.
"problem" ?
from atoum.
"not completed" ?
from atoum.
+1
from atoum.
I have find during the name, It will be "uncompleted".
from atoum.
Related Issues (20)
- Mocking an Throwable interface produces an error HOT 1
- php 8 support ... HOT 1
- Package not up to date on packagist HOT 1
- php 7.4 : includer fail HOT 1
- php 7.4 failed test because of error HOT 1
- strign contains in array has a strange behavior
- [bug] Error executing tests
- rename namespace to atoum without alias
- How to access private and protected properties HOT 2
- HTML coverage not produced using --use-dot-report HOT 2
- php8 test-it failing (8.0.1 -> ... incompatibility on two tests) HOT 4
- [mock] reset calls count HOT 2
- Question about execution order HOT 4
- How to use atoum on a project with no classes? HOT 3
- Silent failures (when ini_set('display_errors', 0)) HOT 2
- Where can I get the phar file for version 4.0.3? HOT 1
- Atoum 4.1 to publish HOT 3
- Question about calling($mock)->MyMethod[…]
- Atoum 4.1 error extends class HOT 1
- Fatal error with Php 8.1.2 HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from atoum.