pherkin/test-bdd-cucumber-perl

Test::BDD::Cucumber - Cucumber in Perl

NAME

Test::BDD::Cucumber - Feature-complete Cucumber-style testing in Perl

SYNOPSIS

# Driving tests using the 'pherkin' binary that comes with the distribution
$ pherkin -l -b t/

# Or choose a subset of tests to be run by selecting all scenarios tagged 'slow'
$ pherkin -l -b --tags @slow t/

# Or all those /not/ tagged 'slow'
$ pherkin -l -b --tags ~@slow

# Fail on missing steps (by default prints as orange output and succeeds tests)
$ pherkin -l -b --strict t/


# Driving tests using 'prove' integration
$ prove --source Feature --ext=.feature examples/

# Driving parallel tests using 'prove'
$ prove -r --source Feature -j 9 --ext=.feature t/

DESCRIPTION

Cucumber for Perl, integrated with Test2, Test::More and prove.

The implementation supports the following Gherkin keywords in feature files: Feature, Scenario, Scenario Outline, Examples, Given, When, Then, And and But. Additionally, Scenario can be used as a synonym for Scenario Outline (with Examples). This best maps to Gherkin version 6.0.13, but without support for its new Rule and Example keywords.

This implementation supports the same languages as Gherkin 15.0.0 - that is, it supports exactly the same translated keywords.

Behaviour of this module is similar to that, but sometimes different from the real Cucumber, the plan is to move use the same parser and behaviour.

GETTING STARTED

This module comes with a few introductory tutorials.

• A Cucumber feature writing tutorial

  for those new to Cucumber and BDD testing

• A Step writing tutorial

  to get you started writing the code run for each Given, Then, When step

• A guide on integrating with your test suite

• An architecture overview

  for those who want to extend or hook into feature file execution

• Documentation of the command-line tool App::pherkin

If you have problems getting started, you can talk to the author(s) here:

BUGS AND LIMITATIONS

For current bugs, check the issue tracer at GitHub: https://github.com/pherkin/test-bdd-cucumber-perl/issues

One thing need specific mentioning:

• Due to the use of its own parser, differences probably exist in the interpretation of feature files when comparing to Cucumber.

  Also see the issue for tracking this topic.

PROJECT RESOURCES

• Source code repository at https://github.com/pherkin/test-bdd-cucumber-perl

• Bug tracker at https://github.com/pherkin/test-bdd-cucumber-perl/issues

• Mailing list at mailto:[email protected]

• Chat (Gitter) at https://gitter.im/pjlsergeant/test-bdd-cucumber-perl

• Chat (IRC) at irc://irc.freenode.net/#perl

• Website at https://pherkin.pm

SEE ALSO

Gherkin - A Gherkin parser and compiler

AUTHORS

Peter Sergeant [email protected]

Erik Huelsmann [email protected]

Ben Rodgers [email protected]

LICENSE

Copyright 2019-2023, Erik Huelsmann
Copyright 2011-2019, Peter Sergeant; Licensed under the same terms as Perl

test-bdd-cucumber-perl's Issues

Summary at the end of 'pherkin' run

It would be practical to have a summary at the end of a pherkin run, similar to the 'prove' summary. Summary information could be: a listing of scenario files with their failing scenarios and succeeding @todo scenarios.

Broken by Test::Builder alphas

Hola,

It appears that my work in Test::Builder will be breaking Test::BDD::Cucumber. I took a look to see if I could figure out how I broke it, but I am afraid I don't understand Test::BDD:Cucumber well enough. Would you mind pulling down the latest Test::Builder alpha (https://metacpan.org/release/EXODIST/Test-Simple-1.301001_056) and helping me figure out what the problem is?

A lot of stuff has changed under the hood of Test::Builder, so once we figure out the problem we can determine if it is a bug in my code, a missing feature, or something else.

Handling of Background sections

Background sections are meant to be run before each of the scenarios. Who knew? (conflicting answers from #cucumber, too :-/)

This needs fixing, but is going to be a PITA from the point of TAP output

Raw access to the stash hash is confusing

It's possible to create your own entries in the stash, which should be strongly discouraged. Warning about that loudly is probably the best way forward

ASCII codes are poorly supported under Windows

Switch from ASCII codes (TERM::ANSIColor?) to something like Win32::Console::ANSI (I'm not sure if this is the answer) when perl is being used in Win32. I completely hate asking for this, I hate having to work in a windows environment. But I do. So, I'm asking.

Use of Ouch sucks

Ouch is a totally unneeded dependency, and should be removed

outline_stash is confusingly named

Call it something else, as it doesn't relate to the stash

Failure in Tutorial feature

$ pherkin

Simple tests of Digest.pm
As a developer planning to use Digest.pm
I want to test the basic functionality of Digest.pm
In order to have confidence in it

Scenario: Check MD5
  Given a usable Digest class
  Given a Digest MD5 object
  When I've added "foo bar baz" to the object
  And I've added "bat ban shan" to the object
  Then the hex output is "bcb56b3dd4674d5d7459c95e4c8a41d5"
    step defined at features/basic.feature line 14.
    ok 1 - Starting to execute step: the hex output is "bcb56b3dd4674d5d7459c95e4c8a41d5"
    not ok 2 - Unknown output type foo bar baz
    #   Failed test 'Unknown output type foo bar baz'
    #   at pherkin/features/step_definitions/basic_steps.pl line 27.
    1..2
  Then the base64 output is "1B2M2Y8AsgTpgAmY7PhCfg"

Scenario: Check SHA-1
  Given a usable Digest class
  Given a Digest SHA-1 object
  When I've added "foo" to the object
  Then the hex output is "0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33"

In the basic steps pl
Then qr/the (.+) output is "(.+)"/, sub {
my $method = {base64 => 'b64digest', 'hex' => 'hexdigest' }->{ $1 } ||
do { fail("Unknown output type $1"); return };
is( S->{'object'}->$method, $2 );
};

$1 is set to 'foo bar baz' instead of base64 or hex

This is perl, v5.10.1 (*) built for x86_64-linux-thread-multi

Use Method::Signatures::Simple

Method::Signatures::Simple has less dependencies than Method::Signatures, and is a bit more lightweight

Skipped test is not displayed, and raises a warning

Hi,

with 0.25, an undefined (and thus skipped) test is not displayed in the pherkin output. Instead, I see:

Can't use an undefined value as an ARRAY reference at /usr/share/perl5/Test/BDD/Cucumber/Harness/TermColor.pm line 121.

Improve Inconsistent number of rows in table error

The "Inconsistent number of rows in table" should state which line/feature file the error was detected in

Documentation regarding stash is incorrect

http://search.cpan.org/~sargie/Test-BDD-Cucumber-0.50/lib/Test/BDD/Cucumber/StepContext.pm#stash

S-> is actually equivalent to $context->stash->{'scenario'}, so the lines aren't equivalent. Also I got rid of the step stash.

[PATCH] spelling fixes

In Debian we are currently applying the following patch to
Test-BDD-Cucumber.
We thought you might be interested in it too.

Description: spelling fixes
Origin: vendor
Author: gregor herrmann <[email protected]>
Last-Update: 2016-02-13

The patch is tracked in our Git repository at
https://anonscm.debian.org/cgit/pkg-perl/packages/libtest-bdd-cucumber-perl.git/plain/debian/patches/spelling.patch

Thanks for considering,
gregor herrmann,
Debian Perl Group

Build failing on older Perls

eg:

Test-BDD-Cucumber-0.45:

• darwin-thread-multi-2level / 5.8.7:
  • FAIL http://www.cpantesters.org/cpan/report/a5df2e90-d207-11e5-a03a-26833848fe5a

Transform that returns undef gives a warning

Use of uninitialized value in substitution iterator at /home/bdr/perl5/perlbrew/perls/perl-5.10.1/lib/site_perl/5.10.1/Test/BDD/Cucumber/StepContext.pm line 199.

The fix is to turn off warnings for the block.

ParserOld.pm shipped in the 0.52 dist: by mistake?

Hi Peter,

while packaging 0.52 for Debian I saw autopkgtest failures due to ParserOld.pm: it's shipped in the tarball, and depends on File::Slurp while this dependency was removed elsewhere. I don't see this file in Git.

Is this file included on purpose? Should we distribute it in Debian?

Output should keep exact scenario keywording

Scenario: Fish
    Given a thing
    And a thing

Should print out like that, not as

Scenario: Fish
    Given a thing
    Given a thing

Step-debugging problem: "Bizarre copy of ARRAY" replaces stacktrace

When I run the tests through pherkin (which I prefer over prove, because in our setup then I can control better which tests are being run), I'm getting the following output. Notice that there's no stacktrace whatsoever.

    Scenario: Viewing setup.pl
      Given a user named "Admin" with a password "a6m1n"
      When I navigate to the setup login page
        step defined at t/66-cucumber/01-basic/login.feature line 12.
        ok 1 - Starting to execute step: I navigate to the setup login page
        not ok 2 - Test compiled
        #   Failed test 'Test compiled'
        #   at /home/ehuelsmann/perl5/lib/perl5/Test/BDD/Cucumber/Executor.pm line 445.
        # Bizarre copy of ARRAY in list assignment at /home/ehuelsmann/perl5/lib/perl5/Devel/StackTrace.pm line 61.
        1..2

When I run the same test script through prove, I get the following output:

#   Failed test '         When I navigate to the setup login page'
#   at /home/ehuelsmann/perl5/lib/perl5/Test/BDD/Cucumber/Harness/TestBuilder.pm line 87.
#   in step at t/66-cucumber/01-basic/login.feature line 12.
# ok 1 - Starting to execute step: I navigate to the setup login page
# not ok 2 - Test compiled
# 
# #   Failed test 'Test compiled'
# #   at /home/ehuelsmann/perl5/lib/perl5/Test/BDD/Cucumber/Executor.pm line 445.
# # Cannot assign a value to a read-only accessor at reader Weasel::Session::page (defined at /home/ehuelsmann/perl5/lib/perl5/Weasel/Session.pm line 87) line 3
# #     Weasel::Session::page('Weasel::Session=HASH(0x8cefb10)', 'PageObject::Setup::Login=HASH(0x9827620)') called at t/lib/PageObject.pm line 24
# #     PageObject::open('PageObject::Setup::Login', 'driver', 'Weasel::Session=HASH(0x8cefb10)') called at t/66-cucumber/01-basic/step_definitions/pageobject_steps.pl line 63
# #     Test::BDD::Cucumber::StepFile::__ANON__('Test::BDD::Cucumber::StepContext=HASH(0x9e3a058)') called at /home/ehuelsmann/perl5/lib/perl5/Test/BDD/Cucumber/Executor.pm line 535
# #     eval {...} at /home/ehuelsmann/perl5/lib/perl5/Test/BDD/Cucumber/Executor.pm line 535
# #     Test::BDD::Cucumber::Executor::dispatch('Test::BDD::Cucumber::Executor=HASH(0x9652328)', 'Test::BDD::Cucumber::StepContext=HASH(0x9e3a058)', 'ARRAY(0x9839048)', 0, 0) called at /home/ehuelsmann/perl5/lib/perl5/Test/BDD/Cucumber/Executor.pm line 445
# #     Test::BDD::Cucumber::Executor::find_and_dispatch('Test::BDD::Cucumber::Executor=HASH(0x9652328)', 'Test::BDD::Cucumber::StepContext=HASH(0x9e3a058)', 0, 0) called at /home/ehuelsmann/perl5/lib/perl5/Test/BDD/Cucumber/Executor.pm line 336
# #     Test::BDD::Cucumber::Executor::execute_scenario('Test::BDD::Cucumber::Executor=HASH(0x9652328)', 'HASH(0x9656c10)') called at /home/ehuelsmann/perl5/lib/perl5/Test/BDD/Cucumber/Executor.pm line 178
# #     Test::BDD::Cucumber::Executor::execute('Test::BDD::Cucumber::Executor=HASH(0x9652328)', 'Test::BDD::Cucumber::Model::Feature=HASH(0x9853048)', 'Test::BDD::Cucumber::Harness::TestBuilder=HASH(0x9651f20)') called at t/66-cucumber.t line 66
#1..2

Which does have a full stacktrace and shows the actual error that's happening. (Now, I'm not even after the full stack trace, which would be nice, but I need to have the error, of course...)

Installation should require recent Term::ANSIColor

older versions doesn't contain necessary colors so user will get errors like:

Invalid attribute name bright_white at /opt/perl-5.10.1/lib/site_perl/5.10.1/Test/BDD/Cucumber/Harness/TermColor.pm line 136

When I upgraded Term::ANSIColor from 2.00 to 3.02 problem disappeared.

Can not get tag option correctly sometime

Hi,
I got an error now when I use -t option.
The module Getopt::Long considered '-t' as the abbreviation of 'theme' so that it report to the pherkin an option of ' scheme: @wip'

I'm not sure its Getopt::Long's bug or pherkin's bug or my fault. I will go on studying it when I'm free.

Here is the command:

perl -MBddHelper  /home/git/regentmarkets/cpan/local/bin/pherkin -l -t @wip features/payment_PortfolioTableUpdate.feature 

Here is the error:

Unknown color theme [@wip] at /home/git/regentmarkets/cpan/local/lib/perl5/Test/BDD/Cucumber/Harness/TermColor.pm line 105.

And here is the output of Getopt::Long when I opened its debug flag:

��Getopt::Long 2.42 called from package "Test::BDD::Cucumber::Harness::TermColor".
  argv: (-l -t @wip features/payment_PortfolioTableUpdate.feature)
  autoabbrev=1,bundling=0,getopt_compat=1,gnu_compat=0,order=1,
  ignorecase=1,requested_version=0,passthrough=1,genprefix="(--|-|\+)",longprefix="(--)".
=> link "c" to SCALAR(0x40db418)
=> $opctl{c} = ARRAY(0x9c0d208) ["s","c","<undef>",$,1,1]
   $opctl{theme} = ARRAY(0x9c0d208) ["s","c","<undef>",$,1,1]
=> arg "-l"
=> find "-l"
=> split "-"+"l"
=> 0 hits () with "l" out of 2
=> saving "-l" (not an option, may permute)
=> arg "-t"
=> find "-t"
=> split "-"+"t"
=> 1 hits (theme) with "t" out of 2
=> option "t" -> "theme"
=> found ["s","c","<undef>",$,1,1] for "theme"
=> cname for "theme" is "c"
=> ref($L{c}) -> SCALAR
=> $$L{c} = "@wip"
=> arg "features/payment_PortfolioTableUpdate.feature"
=> find "features/payment_PortfolioTableUpdate.feature"
=> saving "features/payment_PortfolioTableUpdate.feature" (not an option, may permute)
=> restoring "-l" "features/payment_PortfolioTableUpdate.feature"
Unknown color theme [@wip] at /home/git/regentmarkets/cpan/local/lib/perl5/Test/BDD/Cucumber/Harness/TermColor.pm line 105.

Here is the module version:

$ perl -MGetopt::Long -e 'prinetopt::Long::VERSION,"\n"'
2.42
$ perl -MTest::BDD::Cucumber::Harness::TermColor -e 'print $Test::BDD::Cucumber::Harness::TermColor::VERSION,"\n"'                                                                  
0.37

And the OS:

Debian 3.2.65-1+deb7u1 x86_64 GNU/Linux

Transforms

Hi,

The current version is missing transforms (https://github.com/cucumber/cucumber/wiki/Step-Argument-Transforms). They are useful and help develop better tests.

Wide character in print at /usr/local/share/perl/5.22.1/Test2/Formatter/TAP.pm

Hi Peter!

I wrote a feature file in Russian (Muiti byte string). And then pherkin command issued then print out ”Wide character in print”.

test.feature:

Feature: Тест
    тест тест тест

 Scenario: выполнить тест
    Given тест

step_definitions/test.steps:

#!perl

use strict;
use warnings;
use utf8;

use Test::More;
use Test::BDD::Cucumber::StepFile;

Given qr/тест/, sub {
    ok(1, 'Test');
};

result:

$ pherkin

  Тест
    тест тест тест

    Scenario: выполнить тест
Wide character in print at /usr/local/share/perl/5.22.1/Test2/Formatter/TAP.pm line 113.
      Given тест

What do you mean by "quoting in tables is broken"?

Some specific test you are referring to or piece of code? I might be able to help.

FindBin::libs dep

Very broken module, we don't need it, best to remove

Get a proper landing website

Something like https://sqitch.org/ or https://weasel.pm/

Configuration and extensions such as with Behat?

Hi,

Over at the LedgerSMB project, we've started using pherkin to write BDD tests. While we can run our tests as part of 'make test' using your example t/900-* script, I'm using the pherkin binary all the time to run just the BDD tests, or even part thereof.
In our own t/900-* runner script, I can stop at every feature and run pre- and post-processing hooks. However, when I'm using the pherkin script binary, that's not supported out of the box. In Behat (PHP's Cucumber), Behat reads a configuration file and uses that to trigger/initialize extensions.

Is there an option to do the same here, so I:

a. don't need to write that code into my t/* scripts; and
b. this will be supported by any testing method -- regardless of the method of execution being pherkin or t/*

Thanks for your consideration!

Erik.
PS: If you think this could be a good idea and you have some ideas how to arrange for it, I might be able to send you a PR. I'll gladly discuss further here or over e-mail (or in another discussion forum, mailing list?)

Loader::load takes an unused $tag_scheme (tag_spec?) argument

Basically, this finding invalidates #99: The tag_scheme (tag_spec?) passod to Loader::load is passed through various APIs to end up in Model::Document::_construct(). There nothing happens with it. Also, this parameter isn't documented on any of the APIs that take it as an argument.

I'm proposing to close #99, create PR to eliminate the tag_scheme argument (unless it will be used in the future, then document that) and create a new PR documenting (through examples and the t/900- code) the use of the tagspec on the executor.

Wide character in print at /usr/lib/perl5/site_perl/5.14/Test/Builder.pm line 1826.

I wrote a feature file in Japanese (Muiti byte string).
And then pherkin command issued then print out ”Wide character in print”.
I have tryed change code TestBuiler.pm to use Test::More::UTF8 instead of
Test::More, but became not to pass test.

Feature file missing abort prove

It is probably true also for *.t files but prove will abort if no feature file exist in the specified directories, even it recurse is asked and if some are present in sub-directories.

Tag support incomplete and broken

From a bug report by Craig Caroon:

1. Executor::execute() does not filter by tags

I discovered that, although the documentation says otherwise, the execute() method on Test::BDD::Cucumber::Executor does NOT in fact accept an optional TagSpec parameter and then filter the scenarios to be run by those tags.

1. Scenario level tags are NOT recognized for the first Scenario in a feature file UNLESS there exists a Background section.

Without the Background section, the first set of Scenario level tags are gobbled up by the code that parses off the Conditions of Satisfaction, '_extract_conditions_of_satisfaction'.

Amend authors section

Should probably mention @ehuelsmann

Test files are inconsistently named

Some delimit with _, and some with -

What do you mean by "Pherkin isn't really fit for purpose yet"

Does that mean it lacks some important features or what?

Add IRC details to docs

freenode #pherkin

Using 'After' like this in feature file results in parser error.

-- Parse Error --

Malformed step line
at [test.feature] line 9
thrown by: [/usr/lib/perl5/site_perl/5.8.8/Test/BDD/Cucumber/Parser.pm] line 264

-- [test.feature] --

7| When I visit the billing ledger for company and service
8| Then the billing ledger data should be displayed for company and service
9* After check teardown is successful for company service

Basically I am trying to have testcase teardown unique to every testcase.

Use Gherkin parser as soon as it becomes available

The cucumber project started "Gherkin3", a project which allows to consistently parse Gherkin .feature files across languages (https://github.com/cucumber/gherkin).

Target message protocol from upstream Cucumber project

There are a number of parsing and output protocols in the upstream Cucumber project that if we targeted properly, we could use more of their tooling

Validating HTML/XML

In some of our steps we check HTML and pass in the HTML in the data table.

| html |
|

Text

|

After upgrading to version 0.50 these steps throw an error because it treats the <> brackets as placeholders. I've tried escaping them with backslashes, wrapping the whole string in single and double quotes but it keeps throwing the same error. Is there a way to get around this? I could encode the HTML and update the step to handle this but if there is a way for me not to have to update step code and definitions that would be preferable.

Colourised output hampers colour-blind people

Would be nice to have an option for rspec-style ".#F" output or just prefixing the green/red/orange outputs with handy unicode characters would be good.

Highlighting of test parameters with embedded double quotes

 Scenario: Foo
    Given a Foo object
    When foo is "bar" and bar is "quux"
    Then the output is "iddqd"

What am I supposed to do if foo value has embedded double quotes? I can change regexp to use ' as delimiters, but terminal highlighting seems to use a parser completely unrelated to the regular expressions in step definitons.

Is it feasible to change the parser to use @+/@- offsets to highlight submatches instead?

Not easy enough to redispatch steps

It should be easy to redispatch steps from inside a step definition:

Given "something", sub {
  C->run( Given => "I did setup step 1");
  C->run(Given => "I did setup step 2");
  S->{'foo'} = S->{'bar'}->baz;
}

Is it possible detect conflicts in step-definition?

Hello,

Is it possible detect (or raise error) if one step matches multiple subroutines in *_steps.pl files?

Light colour scheme

"The colour scheme used by Test::BDD::Cucumber::Harness::TermColor does not work well on light terminals."

https://rt.cpan.org/Public/Bug/Display.html?id=85601

Incorrect documentation for Test::BDD::Cucumber::Model::Feature

There is incorrect documentation in sections NAME, DESCRIPTION of Model/Feature.pm

due this error: CPAN generates incorrect documentation and Feature.pm refers to file Document.pm

Testrules location

Prove should also look into directories specified on the command line for testrules.yml
Currently, only t/ is searched, along with what is specified by HARNESS_RULESFILE environment variable.

Test skipped due to failure in previous step

In a feature test with examples, failure in one step should only skip remaining steps of the current example.
Currently, a failure in an example skip all other examples.

Allow Harness Plugin to parse command line options again

Hello Peter,

is it possible to add the Parameter 'pass_through' to the Getopt::Long::Configure call in App::pherkin?
This would be helpfull in Test::BDD::Cucumber::Harness::Html to allow parsing command line arguments again.

-Markus

Unneeded dependency on Digest::SHA1?

Hi,

while packaging the latest release for Debian, I noticed that commit e1e81c5 introduces a dependency on Digest::SHA1. I don't see this module used anywhere in the code. Am I missing something?

Cheers!

includes option in config file being ignored

As reported by @ehuelsmann, apparently the includes option is being ignored in configuration profiles. To investigate!

pherkin doesn't exit with non-zero exit code on failed test

It would be nice to have this feature in order to use it in automatic builds/Makefiles.