peridot-php/peridot

Event driven BDD test framework for PHP

The highly extensible, highly enjoyable, PHP testing framework.

Read more at peridot-php.github.io or head over to the wiki.

Building PHAR

Peridot's phar is built using Box. Once box is installed, the phar can be built using the following command from the project directory:

box build

Generating Peridot's documentation

Peridot API documentation is generated using apigen. Once apigen is installed, run the following command from the project directory:

apigen generate

This will output documentation to the docs/ directory.

Running Peridot's tests

Peridot's test suite can be run using Peridot:

$ bin/peridot

And a sample of output:

Release

We use Robo for releases.

robo release [version] [site-path]

using assert for expectations

Peridot sets ASSERT_CALLBACK via assert_options in order to throw exceptions when using the native assert function. Peridot specs are considered passing if they don't throw an exception; if you are using assert for expectations and you find your specs are triggering false positives, you may need to update zend.assertions to 1 in your php.ini. This is set to -1 by default in PHP 7+

If you aren't too keen on using assert for testing, there are a ton of assertion/expectation libraries out there. Anything that throws exceptions in response to a failed assertion will do. The peridot-php org has authored Leo, and this library offers a richer assertion vocabulary for testing.

peridot's Issues

Generic error interface

Related to #117 - it would probably be a good thing to have a generic error interface that can be used.

I imagine it would mirror what is already available to PHP exceptions, but create an interface for objects that are not actually exceptions.

Integrate scrutinizer ci and coveralls.io

We should integrate these into the project:

• scrutinizer-ci - code quality
• coveralls - coverage stats and history

PHP Errors

We should suppress non fatal errors and make them available to reporters.

Create a distributable executable (phar)

• initially, just build it
• build with phing?

Multiline exception messages are not indented correctly

I had a crack at integrating a mocking library I'm working on with Peridot, and ran into this small issue. Here's the output:

  Phony
    ✓ should record passing mock assertions
    1) should record failing mock assertions


  1 passing (28 ms)
  1 failing

  1) Phony should record failing mock assertions:
     Expected call on PhonyMock_TestClassA_0[1]->testClassAMethodA with arguments like:
    <'a'>, <'b'>
Never called.
      (stack trace)

So, basically I would expect this section:

     Expected call on PhonyMock_TestClassA_0[1]->testClassAMethodA with arguments like:
    <'a'>, <'b'>
Never called.

to be rendered like:

     Expected call on PhonyMock_TestClassA_0[1]->testClassAMethodA with  arguments like:
         <'a'>, <'b'>
     Never called.

Configurable suite loading

Make it happen.

Add CHANGELOG

A change log would be handy.

Set assert_options to throw exceptions rather than warnings

Exceptions are much are more useful for us. For now set in bootstrap and possibly make it default behavior with option to turn off?

<?php
assert_options(ASSERT_CALLBACK, function($script, $line, $message) {
    throw new \Exception($message);
});

Write Docs!

md in the repo? wiki? GH page?

Build process

A build process would be rad. Something that updates Console\Version, docs, and the phar and gets them where they need to go.

More modular peridot

I think a good goal for Peridot 2.0 would be to further modularize peridot if possible.

Not entirely sure on execution, but it would be great if peridot-php/peridot was a sum of core modules (core, cli, runners, concurrency?)

I think core plugins and modules would need to agree on some convention for this work - particularly with plugins, runners, etc.

Runner getter and setter on Console\Command

In preparation for #120 - we should at least create a getter and setter for the runner object used in Console\Command.

Single EventEmitter

We should ditch the event emitter trait and use a single event emitter. Simplifies event handling and should make plugin/extension development easier.

Mockery::close() not work in afterEach()

    beforeEach(function() {
        $this->wapper = Mockery::mock('Wrapper');
        $this->writer = new Writer($this->wapper);
    });

    it('should write a line', function() {
        $this->wapper->shouldReceive('wrap')->once();
        $this->writer->write();
        Mockery::close();
    });

This spec will failure as my expected, because wrap() not implement.
But if I move Mockery::close() in afterEach(), it's not failure, it's not my expected.

Can not be read the autoload.php in hhvm 3.5.0 nightly

Following such an error message is displayed.
The cause seems to have happened because the path of __DIR__ is different.

[RuntimeException]                                                           
Error Output: You need to set up the project dependencies using the following commands:
curl -s http://getcomposer.org/installer | php                               
php composer.phar install

I result that tried in 3.4.0 and 3.5.0 nightly-build It was following results.

Code

#!/usr/bin/env php
<?php

$autoloaders = [
    __DIR__ . '/../../../autoload.php',
    __DIR__ . '/../vendor/autoload.php',
    __DIR__ . '/vendor/autoload.php'
];

HHVM 3.4.0

<project-path>/vendor/peridot-php/peridot/bin/../../../autoload.php
<project-path>/vendor/peridot-php/peridot/bin/../vendor/autoload.php
<project-path>/vendor/peridot-php/peridot/bin/vendor/autoload.php

HHVM 3.5.0 nightly build

<project-path>/vendor/bin/../../../autoload.php
<project-path>/vendor/bin/../vendor/autoload.php
<project-path>/vendor/bin/vendor/autoload.php

Errors should be captured, and output independent of results

• have the Runner set error handler to gather up notices
• emit an "error" event and pass the notice to handlers
• update our standard reporters to spit out error information in a nice, pretty way.

Make the CLI Windows friendly

spec setups should run in order defined

This should be correct behavior for tests:

beforeEach(function() {
    $this->thing = "thing";
});

beforeEach(function() {
    $this->dependsOnThing = $this->thing . " is depended on";
});

Currently they run in reverse order.

Implement code coverage analysis

Profile Peridot

Should probably profile this bad brother to check performance bottlenecks.

Would not hurt to run suites against other spec frameworks as well.

Create a contributing doc

Remove application from peridot.configure event

Application as an argument is not useful in the peridot.configure event. It fires too late for most purposes. Application work should be handled in the peridot.start event always.

Time and memory usage

We should probably make time and memory usage stats available to the reporters.

Logging

Peridot should minimally support JUnit XML and TAP log formats.

CLI for peridot

We need a command line executable for peridot. For now should only take a single argument that is a path to a specific spec file or a directory of specs. Eventually the runner should accept some configuration via a file (yml? json?) or as command line switches for things like loading conventions, reporter to user, etc...

peridot specs/
peridot specs/runner.spec.php

Omitting a test definition (callable) should default tests to "pending" state

We should be cool and make functionless tests default to pending if a function is omitted.

describe("My rad thing", function() {
    it("should do a rad thing"); //pending

    it("should do another rad thing"); //pending
});

Improve grep

Right now the --grep mechanism only serves as a filename filter. It might make sense to add a different option for file names and allow --grep to match on the contents of test descriptions.

I would propose the following:

• --grep passes a regular expression to be evaluated against it descriptions
• add a --test-suffix option similar to PHPUnit for filtering on file names

Need a travis build

Implement a configuration file

Always invoke teardown functions when test fails

need to invoke teardown to proceed with remaining tests

We need a Wiki

We need a wiki demonstrating some testing with Peridot - good practices, leveraging some of BDD's strengths.

We should also make a point of stressing some of the features that distinguish Peridot - primarily events, plugins, reporters, and especially scopes.

Support pending specs

We should support some variety of pending specs - probably via x style functions:

<?php
xdescribe('A pending suite', function() {
   xit("pending spec", function() {
   });

   xcontext("pending context", function() {
   });
});

Setting properties on $this in describe is not working

This is broken again:

describe('A sandwich', function() {
    $this->sandwich = ['delicious' => true];

    it('should be delicious', function() {
        assert($this->sandwich['delicious'], "should be delicious"); //Exception: Scope property not found
    });
});

Move runner initialization to Console\Command instead of Application

There is no need for Console\Application to handle creation of the runner. It should be moved entirely to Console\Command.

This would make changing the runner at runtime more flexible.

Support nested suites and context fn

We need to allow nested suites and a context function.

<?php
describe("Thing", function() {
    describe("Nested thing", function() {
        context("The nested thing is this right now", function() {
        });
    });
});

Contexts are functionally equivalent to suites.

Add phpdoc comments (existing and moving forward)

We should probably start documenting things with phpdoc comments, as well as add them to existing code.

Please consider the use of Facades

See Laravel for more details.

Result reporter (pretty results)

We need a way to report failures and maybe successes? As an option? We should keep in mind that reporting should be a HIGHLY configurable part of peridot.

• Pretty output of successes - Green dots and number of tests run?
• Tree output of failures - as red?
• Output trace information for exceptions
• Convert errors to exceptions (fatal not possible)
• Create event driven system (EventEmitter? Evenement, etc....)

Avoid static php timer calls.

Since we use the static PHP_Timer methods in the reporter, there is a chance of overwriting the data by calling PHP_Timer methods elsewhere.

It would probably make more sense to just store microtimes and use the PHP_Timer formatting function (or write our own).

Rename spec.* events to test.*

For consistency with rename of classes and files, spec.* events should be test.*

Loader should only run once for an application

currently the application loads suites twice on successive runs. It should only load once.

Times based off of DateFormat?

Is there any reason to use the PHP_Timer class? Are there any issues with just using PHP's DateTime class and formatting it? Never a bad thing to eliminate a dependency.

Integrate Symfony\Console in CLI

Suite loader

A class responsible for loading specs

• Load a directory of specs by convention i.e - *.spec.php
• Return loaded file paths as array

Add beforeEach and afterEach facades for tear down and setup fns

$spec->addSetupFn and $spec->addTearDownFn need beforeEach() and afterEach() facades

• beforeEach facade
• afterEach facade

Add file path to tests

A test should be able to see its associated file. Something like:

$suite->getFile();  // /path/to/test.spec.php

Handle setUp function errors

what can actually be processed after a setup failure?

Configurable scope

Make it happen.

Support pending suites - i.e remove need for noop functions when creating pending suites.

Currently - a pending suite, or a suite that does nothing is created by passing a noop function to the suite:

$suite = new Suite('does nothing', function () { });

It would be nice to have this work as it does in tests.

$suite = new Suite('does nothing');

Fatal errors should exit with a proper error code.

Right now, they are exiting with code 0 which make travis think the tests are successful.

E.g. https://travis-ci.org/peridot-php/peridot/jobs/37307276 (hhvm build)