Giter Site home page Giter Site logo

Comments (20)

mageekguy avatar mageekguy commented on June 16, 2024

There is no way to handle direct call to die() or exit() in a tested class.
The only solution is to use a dedicated public method to stop the script and mock it in the test.

stop('Stop because i want !'); } } } namespace vendor\project\tests\units; class foo extends \mageekguy\atoum\test { public function testSoSomethingAndCallStop() { $this->mockTestedClass(); ``` $foo = new \mock\vendor\project\foo(); $foo->getMockController()->stop = function() {}; $foo->doSomethingAndCallStop(); $this->assert ->mock($foo)->call('stop')->withArguments('Stop because i want !')->once() ; } ``` } ?>

from atoum.

tentacode avatar tentacode commented on June 16, 2024

Just to be sure, I don't want to test the call to die(), I want to prevent someone to call a die() somewhere and still my CI is green.

from atoum.

ratibus avatar ratibus commented on June 16, 2024

But how can you return a global Success status though the status line prints 3/5 ?

from atoum.

mageekguy avatar mageekguy commented on June 16, 2024

"Success" means "no fail, no error and no exception".
Currently, it does not mean that all test methods was executed.

from atoum.

tentacode avatar tentacode commented on June 16, 2024

Maybe adding a "--strict" option a la Behat, where skipped test returns Error could solve this issue ?

from atoum.

ratibus avatar ratibus commented on June 16, 2024

Yep I see.
When you say "Currently", do you mean that you consider it as a bug (I do) ?
In fact there can be multiple exit scenarii that you cannot intercept like fatal errors, that lead to such a case.

from atoum.

mageekguy avatar mageekguy commented on June 16, 2024

When you say "Currently", do you mean that you consider it as a bug (I do) ?

Not sure that it's a bug, see below.

In fact there can be multiple exit scenarii that you cannot intercept like fatal errors, that lead to such a case.

atoum can handle fatal error and it's a failure.
atoum can also handle segfault without any problem, but it's not a failure, because it's a PHP problem (bug in the Zend Engine), not a code problem (not a bug in the tested class).

I think that it's a semantic problem.
What means exactly "SUCCESS" ?
In my current opinion, "SUCCESS" = 0 fail, 0 error, 0 exception.
Segfault is a problem, but it's not a test failure.
Direct call to die() or exit() are in fact similar to segfault : the php script stop, and it's not a test failure.

So, perhaps there is not enough visibility in the report about the fact that some test methods stop "abnormaly".

try this :

It's a failure ? Not sure...
Currently, atoum say :

atoum version DEVELOPMENT by Frédéric Hardy (/Users/fch/Atoum/repository)
PHP path: /usr/local/bin/php
PHP version:
=> PHP 5.3.8 (cli) (built: Sep 21 2011 23:14:37)
=> Copyright (c) 1997-2011 The PHP Group
=> Zend Engine v2.3.0, Copyright (c) 1998-2011 Zend Technologies
=> with Xdebug v2.1.1, Copyright (c) 2002-2011, by Derick Rethans
mageekguy\atoum\tests\units\adapter...
[.___________________________________________________________][0/1]
=> Test duration: 0.01 second.
=> Memory usage: 0.00 Mb.
Total test duration: 0.01 second.
Total test memory usage: 0.00 Mb.
Code coverage value: 0.00%
=> Class mageekguy\atoum\adapter: 0.00%
==> mageekguy\atoum\adapter::__call(): 0.00%
==> mageekguy\atoum\adapter::invoke(): 0.00%
Running duration: 0.16 second.
Success (1 test, 1 method, 0 assertion, 0 error, 0 exception) !

from atoum.

ratibus avatar ratibus commented on June 16, 2024

According to me, Success means : every test passed.
One test that did not run is a failure. What is the purpose of this test if when it does not run, the global status is still ok ?
I write tests for them to be run :)

from atoum.

mageekguy avatar mageekguy commented on June 16, 2024

I write tests for them to be run :)

But it can't run with a die() or a exit() in it...
So, if there is a die() in it, it's certainly because you don't want run it, so it's not a failure.

from atoum.

marmotz avatar marmotz commented on June 16, 2024

if all tests was run and passed => Success
if one or more tests wasn't run or not passed => Fail (or Error, or Warning, or whatever)

from atoum.

ratibus avatar ratibus commented on June 16, 2024

In fact we put a die in it to debug something. And we saw that the test still pass. That's why we opened this issue in the first place :D
The test should have failed. That surprised us it did not.

from atoum.

mageekguy avatar mageekguy commented on June 16, 2024

Currently, a die() in a test method is equivalent to an empty test method.
And an emtpy test method is not a failure.
So i have a problem, because this case is like the schrodinger cat : it may be failure, or not.

from atoum.

ratibus avatar ratibus commented on June 16, 2024

I see your problem :)
But what is the point of an empty test method ? According to me it should be considered as a failure.

from atoum.

mageekguy avatar mageekguy commented on June 16, 2024

But what is the point of an empty test method ? According to me it should be considered as a failure.

Why ?
No assert = no failure = success.

from atoum.

ratibus avatar ratibus commented on June 16, 2024

No assert = useless = failure :)
Different point of view :D

from atoum.

mageekguy avatar mageekguy commented on June 16, 2024

No assert = useless = failure :)
Different point of view :D

Yes, just need to know if all users are the same point of view than you.

FYI, It's not a problem to change this behaviour.
Just need to add a "status" in the score to handle these cases (segfault, direct call to die()/exit(), empty test method).
Currently, there are "fails", "errors" and "exceptions" in score.
But i have no idea for the name of the new status.
If you have a suggestion...

from atoum.

marmotz avatar marmotz commented on June 16, 2024

"problem" ?

from atoum.

ratibus avatar ratibus commented on June 16, 2024

"not completed" ?

from atoum.

marmotz avatar marmotz commented on June 16, 2024

+1

from atoum.

mageekguy avatar mageekguy commented on June 16, 2024

I have find during the name, It will be "uncompleted".

from atoum.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.