gabrielfalcao/lettuce

Behavior-driven-development tool for python, inspired by Cucumber for Ruby ⛺

lettuce's Issues

Can define callbacks to run before and after all

It can be defined in any file that will be imported by lettuce.

But lettuce documentation should tell to use it at terrain.py

examples:
from lettuce.terrain import before
from lettuce.terrain import after

@before.all
def populate_database(scenario):
    from my_models import Person
    Person(name="John Doe").save()

@after.all
def reset_database(scenario):
    from my_models import Person
    Person.delete_all()

Can define callbacks to run before and after each feature

It can be defined in any file that will be imported by lettuce.

But lettuce documentation should tell to use it at terrain.py

examples:
from lettuce.terrain import before
from lettuce.terrain import after

@before.each_feature
def populate_database(scenario):
    from my_models import Person
    Person(name="John Doe").save()

@after.each_feature
def reset_database(scenario):
    from my_models import Person
    Person.delete_all()

Lettuce should complain about more than 1 feature in a file

If one defines more than one feature in a text file, lettuce should warn, show the syntax error and exit

Environment should be easy to set

Environment (setup, teardown and global scope)

** AKA thread-local stuff **

A global python file will be imported, maybe something like:

terrain.py

which contains stuff like:

from lettuce.terrain import before_each_scenario
from lettuce.terrain import after_each_scenario
@before_each_scenario
def populate_database(scenario):
    from my_models import Person
    Person(name="John Doe").save()

@after_each_scenario
def reset_database(scenario):
    from my_models import Person
    Person.delete_all()

And maybe, something that make easy stuff like, creating a single instance of a browser driver.
from lettuce.terrain import world
from webdriver_firefox.webdriver import WebDriver

world.browser = WebDriver()

So that, when and where ever I decide to use the browser driver, I do not need to instantiate it again, just do:

from lettuce.terrain import world
@step(u'I go to google home page')
def browse_google(context):
    world.browser.get("http://google.com")

And so on

Lettuce always try to load a environment file

Lettuce always try to load terrain.py which can be located ONLY in the dir which lettuce runs

Example:
gabriel@durga:~/Projects/my-project$ lettuce features

Will try to import: ~/Projects/my-project/terrain.py

Language error

Here is my languages.py code - http://pastebin.com/k7FEnkCL

Example scenario

language: pl

Właściwość: Wyszukiwanie

Scenariusz: Znajdywanie wpisów, wydarzeń, miejsc, grup i filmów o cesarskim cięciu
Zakładając że jestem na stronie głównej
Jeżeli wpisze w wyszukiwarke: "cesarskie ciecie"
Wtedy otrzymam "20" ostatnich wpisów, wydarzeń, miejsc, grup, filmów zawierających frazę: "cesarskie cięcie"

And exception

tests$ lettuce

Traceback (most recent call last):
File "/usr/bin/lettuce", line 5, in
pkg_resources.run_script('lettuce==0.1rc12', 'lettuce')
File "/usr/lib/python2.5/site-packages/pkg_resources.py", line 448, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/usr/lib/python2.5/site-packages/pkg_resources.py", line 1166, in run_script
execfile(script_filename, namespace, namespace)
File "/usr/lib/python2.5/site-packages/lettuce-0.1rc12-py2.5.egg/EGG-INFO/scripts/lettuce", line 53, in
main()
File "/usr/lib/python2.5/site-packages/lettuce-0.1rc12-py2.5.egg/EGG-INFO/scripts/lettuce", line 48, in main
result = runner.run()
File "/usr/lib/python2.5/site-packages/lettuce-0.1rc12-py2.5.egg/lettuce/init.py", line 109, in run
results.append(feature.run())
File "/usr/lib/python2.5/site-packages/lettuce-0.1rc12-py2.5.egg/lettuce/core.py", line 706, in run
callback(self)
File "/usr/lib/python2.5/site-packages/lettuce-0.1rc12-py2.5.egg/lettuce/plugins/colored_shell_output.py", line 154, in print_feature_running
write_out("\033[1;37m%s\n" % line)
File "/usr/lib/python2.5/site-packages/lettuce-0.1rc12-py2.5.egg/lettuce/plugins/colored_shell_output.py", line 47, in write_out
wrt(wp(what))
File "/usr/lib/python2.5/site-packages/lettuce-0.1rc12-py2.5.egg/lettuce/plugins/colored_shell_output.py", line 26, in wrt
sys.stdout.write(what)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u0142' in position 8: ordinal not in range(128)


Best regards,
PS

Support for tags

Tagging is cool, I can run scenarios that suit different ocasions

Scenario must contain its feature

In a nutshell:

scenario = feature.scenarios[0]
assert scenario.feature == feature

Step definitions should receive the step as first argument

So that developer can take advantage of some context.

Example:
from lettuce import step
from lettuce.core import Step

@step(r'someone does something')
def somedosome(step):
    assert isinstance(step, Step)

Create a django app within lettuce, to manage lettuce within django :)

In a few words, the programmer will be able to do something like:

settings.py:

...
INSTALLED_APPS = (
    ...
    "lettuce.django"
    ...
)

It will give us:

python manage.py lettuce

that runs lettuce within python, so that we will be able to use django models within lettuce steps:

from lettuce import step
from myapp.models import Poll

@step("Given I have the following polls:")
def have_items(step):
    for poll in step.data_list:
        Poll.objects.create(name=poll['name'])

Lettuce may also put Django's builtin server running at 0.0.0.0:9000, in a thread, so that lettuce users will be able to use browser test tools such as selenium, webdriver, windmill and so on

lettuce should propose regex groups for undefined steps with strings within double-quotes

For example:

Feature: double-quoted snippet proposal
  Scenario: Propose matched groups
    Given I have "stuff here" and "more @#$%ˆ& bizar sutff h3r3"

should be proposed with:

@step(r'Given I have "(.*)" and "(.*)"')
def given_i_have_group1_and_group2(step):
    pass

Traceback must be shown as code is runs

This rocks on cucumber

Syntax checking

Lettuce must complain about features without name

more output options - html output would be awesome!

more output options - html output would be awesome!

Lettuce should load all features files from a given folder recursively

And when lettuce is ran without parameters, this directory will be the current directory "."

lettuce should find step definitions even without __init__.py file within step_definitions folder

Why ?

It will be a python kick-ass feature.

Mostly python libraries and programs at all have this issue.
It is not a big problem, since it suits python's zen that says: "Explicit is better than implicit"

Althrough, programmers use to forget to create init.py files within dirs, and I've seen so many programmers (specially rookies) loosing half an hour, and even hours debugging after this simple problem.

How to ?

Quite simple:
Lettuce will create a empty init.py file within step_definitions folder under any dir that contains *.feature files.

The init.py file created by lettuce must contain:

# -*- coding: utf-8 -*-
# This file was automatically created by lettuce
# http://github.com/gabrielfalcao/lettuce/

AttributeError: 'thread._local' object has no attribute <attr here>

I have two features. So, before implement the second, the global variable world runs fine. But, when implement the second feature (only add @step(r'desc') def ...) the lettuce raise this exception.

In the first feature I have a attribute world.username. This works fine. Implement the second feature, raise AttributeError: 'thread._local' object has no attribute 'username'

Steps should be aware of its definitions

This ticket is a child of #8

A step must be aware of its definition.

Example:
step.defined_at.file == "/full/path/to/step_definitions/my_steps.py"
steo.defined_at.line == 1

Shouldn't stop testing on Exceptions

When an exception is raised, other features and scenarios won't be tested.
Lettuce should mark the feature as failed and continue testing.

There could be an option to stop on the first failed feature, like Django's --failfast option:
http://docs.djangoproject.com/en/dev/ref/django-admin/#djadminopt---failfast

lettuce should propose regex groups for undefined steps with strings within single-quotes

The same thing of ticket #37, but for single-quotes

Example:

Feature: double-quoted snippet proposal
  Scenario: Propose matched groups
    Given I have 'stuff here' and 'more @#$%ˆ& bizar sutff h3r3'

Snippet would be:

@step(r'Given I have \'(.*)\' and \'(.*)\'')
def given_i_have_group1_and_group2(step):
    pass

Can define callbacks to run before and after each scenario

It can be defined in any file that will be imported by lettuce.

But lettuce documentation should tell to use it at terrain.py

examples:
from lettuce.terrain import before
from lettuce.terrain import after

@before.each_scenario
def populate_database(scenario):
    from my_models import Person
    Person(name="John Doe").save()

@after.each_scenario
def reset_database(scenario):
    from my_models import Person
    Person.delete_all()

Support for step transforms

It allows tranforming matched group strings into any kind of object.
The description at cucumber's wiki is here

Example:

from sure import that
from lettuce import step, transform
from django.contrib.auth.models import User

@transform(r'the user "(.*)"')
def transform_user(username):
    user_object = User.objects.get(username=username)
    return user_object

@step(r'I clone permissions from the user "(.*)" to the user "(.*)"'):
def clone_permissions_from_to(step, one, two):
    assert that(one).is_a(User)
    assert that(two).is_a(User)

    two.clone_permissions_from(one)

Handle KeyboardInterruption

Lettuce should show the current test results already ran when user does a control-c or something that raises KeyboardInterruption.

It is so cool, developer will be able to cancel the whole feature set to run when some fail

Steps must contain its scenario

In a nutshell:

step = scenario.steps[0]
assert step.scenario == scenario

Refactor lettuce.plugins.shell_output and lettuce.plugins.colored_shell_output

Those modules must be refactored to use couleur.

shell_output must print with colored markup, something like:

sys.stdout.write("{red}1 failed")

and shell_output will setup couleur to ignore those markups:

couleur.stdout_filter.setup(ignore_markups=True)

thus colored_shell_output will do:

1. import shell_output plugin

2. setup couleur to process stdout markups:

   couleur.stdout_filter.setup(ignore_markups=False)

Step, Scenario and Feature objects should implement __repr__ nicely

So that, when debugging, se could preview its contents:

repr(step1)
<Step "When I do something" defined at steps.py:14>

Step definitions should receive matched groups as params

And must work for both named and unnamed groups.

Example with unnamed groups:

from lettuce import step
@step(r'Given I match this group "(.*)"')
def given_unnamed(group):
    pass

Example with named groups:

from lettuce import step
@step(r'Given I match this group "(?P<foobar>.*)"')
def given_unnamed(foobar):
    pass

Example with both:

from lettuce import step
@step(r'Given I (match|search) this group "(?P<name>.*)"')
def given_unnamed(action, name):
    pass

Can't install in Python 2.5

pip install lettuce works correctly on Python 2.6, but doesn't on Python 2.5. Apparently this is due to a known bug in distutils that mishandles unicode strings. In this case, it's your name, Gabriel Falcão, that triggers the problem.

A way to work this around would be explicitly encoding the string as follows:

author=u'Gabriel Falcão'.encode("UTF-8")

Not sure if this has side effects though. Any ideas?

Lettuce should load all python files under the given directory

And when lettuce is ran without parameters, this directory will be the current directory "."

Features should have description

They must be parsed from the feature, it must be tested in a full-featured feature, example:

Feature: Do many things at once
    In order to automate tests
    As a automation freaky
    I want to use scenario outlines

    Scenario Outline: Add two numbers
        Given I have entered <input_1> into the calculator
        And I have entered <input_2> into the calculator
        When I press <button>
        Then the result should be <output> on the screen

    Examples:
        | input_1 | input_2 | button | output |
        | 20      | 30      | add    | 50     |
        | 2       | 5       | add    | 7      |
        | 0       | 40      | add    | 40     |

bold output strings that match groups

lettuce should identify parts of strings that matches regex groups at step definitions, and output it as bold.

for example:

Given I fill textbox "username" with "gabriel"

and the step definition:

@step('I fill textbox "(.*)" with "(.*)"')
def fill_txtbox(step, name, value):
    pass

it should output like this;

Given I fill textbox "username" with "gabriel"
there is a discussion about it at #3

Refactor and raise test coverage to 100%

The code must be refactored and reach 100% of test coverage, both unit and functional.
Once it's achieved, never let the coverage decrease

Mode to print a summary like the following:

Mode to print a summary like the following:
"passed X tests out of Y" (if all pass) - exiting with the return code 0
if a test fails:
"passed X tests out of Y, test Z failed" - exiting with return code 1
if multiple tests fail:
"passed X tests out of Y, tests Z,P failed" - exiting with return code 1

The output needs to be on one line - this is required for use in nagios.

Documentation!

Lettuce should have a awesome documentation, inspired on Django's.

Would be awesome to put the documentation available on github too

Language support

Lettuce should support different languages, exactly the same way cucumber does.

We could even borrow cucumber language file, once its license is compatible with Lettuce one

http://github.com/aslakhellesoy/cucumber/blob/master/lib/cucumber/languages.yml

Can define callbacks to run before and after each step

It can be defined in any file that will be imported by lettuce.

But lettuce documentation should tell to use it at terrain.py

examples:
from lettuce.terrain import before
from lettuce.terrain import after

@before.each_step
def populate_database(scenario):
    from my_models import Person
    Person(name="John Doe").save()

@after.each_step
def reset_database(scenario):
    from my_models import Person
    Person.delete_all()

Unicode error on python setup.py install

pats@patrycjuszszydlo:~/Pulpit/Pobieranie/lettuce$ sudo python setup.py install
running install
running bdist_egg
running egg_info
creating lettuce.egg-info
writing lettuce.egg-info/PKG-INFO
Traceback (most recent call last):
  File "setup.py", line 38, in 
    packages=get_packages()
  File "/usr/lib/python2.5/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/usr/lib/python2.5/distutils/dist.py", line 974, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python2.5/distutils/dist.py", line 994, in run_command
    cmd_obj.run()
  File "/usr/lib/python2.5/site-packages/setuptools/command/install.py", line 76, in run
    self.do_egg_install()
  File "/usr/lib/python2.5/site-packages/setuptools/command/install.py", line 96, in do_egg_install
    self.run_command('bdist_egg')
  File "/usr/lib/python2.5/distutils/cmd.py", line 333, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python2.5/distutils/dist.py", line 994, in run_command
    cmd_obj.run()
  File "/usr/lib/python2.5/site-packages/setuptools/command/bdist_egg.py", line 167, in run
    self.run_command("egg_info")
  File "/usr/lib/python2.5/distutils/cmd.py", line 333, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python2.5/distutils/dist.py", line 994, in run_command
    cmd_obj.run()
  File "/usr/lib/python2.5/site-packages/setuptools/command/egg_info.py", line 170, in run
    writer(self, ep.name, os.path.join(self.egg_info,ep.name))
  File "/usr/lib/python2.5/site-packages/setuptools/command/egg_info.py", line 379, in write_pkg_info
    metadata.write_pkg_info(cmd.egg_info)
  File "/usr/lib/python2.5/distutils/dist.py", line 1076, in write_pkg_info
    self.write_pkg_file(pkg_info)
  File "/usr/lib/python2.5/distutils/dist.py", line 1094, in write_pkg_file
    file.write('Author: %s\n' % self.get_contact() )
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe3' in position 20: ordinal not in range(128)
pats@patrycjuszszydlo:~/Pulpit/Pobieranie/lettuce$

Best regards,
PS

ImportErrors within steps are swallowed

If one put some wrong import within some step definition file, it is swallowed by lettuce.

Scenario outlines does not work within a feature

Example:

Feature: Do many things at once
    In order to automate tests
    As a automation freaky
    I want to use scenario outlines

    Scenario Outline: Add two numbers
        Given I have entered <input_1> into the calculator
        And I have entered <input_2> into the calculator
        When I press <button>
        Then the result should be <output> on the screen

    Examples:
        | input_1 | input_2 | button | output |
        | 20      | 30      | add    | 50     |
        | 2       | 5       | add    | 7      |
        | 0       | 40      | add    | 40     |

Self documenting would be AWESOME!

What if at command line I could type something like:

gabriel@durga:~/Projects/my-project$ lettuce harvest
Steps defined here:
    /the client '(.*)' rents '(.*)'/
    Step example:
        "When the client 'John Doe' rents 'Iron man 2"
    Defined At:
        features/step_definitions/movie_rental_steps.py:14

It could be done by simply doing at features/step_definitions/movie_rental_steps.py:
from lettuce import step
@step(r"the client '(.)' rents '(.)'")
def client_rents(self, client, movie):
"When the client 'John Doe' rents 'Iron man 2"
pass

mailing list?

do you have a mailing list to communicate on yet? or a google groups?

Features, Scenarios and Steps must know about its definitions

Features, Scenarios and Steps must know the file path and line numbers that implement them.

Features:
feature.described_at.file == "/full/path/to/definition.feature"
feature.described_at.line == 17

Scenarios:
scenario.described_at.file == "/full/path/to/definition.feature"
scenario.described_at.line == 30

Step:
step.described_at.file == "/full/path/to/definition.feature"
steo.described_at.line == 25
step.defined_at.file == "step_definitions/my_steps.py"
steo.defined_at.line == 2

It should show output as in cucumber

Colored output on shell

Feature description is WHITE
Step currently running is GRAY
Steps that have failed, get RED
Steps that have passed, get GREEN as Lettuce leaves
Steps not yet defined, get YELLOW

Important

It is very useful to show the referenced python file of each step, referencing the line number. Exactly as in

Lettuce should ignore accents in letters of step definitions

It sucks and is too misleading when you define a step, and set some phrase with accents, but it doesn't match with the step it was supposed to match.

It can be a optional parameter, that says so, comes set as true by default.

review and simplify documentation

The documentation are way messy.
It must be improved before the official release 0.1

Can define callbacks to run before and after all

It can be defined in any file that will be imported by lettuce.

But lettuce documentation should tell to use it at terrain.py

examples:
from lettuce.terrain import before
from lettuce.terrain import after

@before.all
def populate_database(scenario):
    from my_models import Person
    Person(name="John Doe").save()

@after.all
def reset_database(scenario):
    from my_models import Person
    Person.delete_all()

Lettuce should have a mutable, thread-safe global context

So that we could set global variables and use it along step definitions
Something like:
from lettuce.terrain import world
from webdriver_firefox.webdriver import WebDriver

world.browser = WebDriver()

It should be simple to define steps

A python file that has methods with @lettuce.step decorator:

from lettuce import *
@step(u'I have the following tables:')
def have_following_tables(context, dicts):
    assert {"Item": "Value"} in dicts

And assertion errors must be enough to fail steps, as well

world should have immutable attributes

If set some unexistent attribute in world, and it starts with _, the it must become immutable
lettuce will use it to store its global stuff:

world._runner
world._shell

Changing them must raise a exception like:
AttributeError('lettuce.world attributes starting with "_" are immutable')

hooks for django

before and after apps:

@before.each_app
def do_something_before(app_module):
    pass

@after.each_app
def do_something_after(app_module):
    pass

before and after the "harvest"

@before.harvest
def do_something_before(dict_of_local_variables):
    pass

@after.harvest
def do_something_after(list_of_results):
    pass

before and after runserver

@before.runserver
def do_something_before(server_instance):
    pass

@after.runserver
def do_something_before(server_instance):)
    pass

threading support

handling request

@before.handle_request
def something_before(httpd):
    pass

@after.handle_request
def something_after(httpd):
    pass

running server

@before.within_runserver
def something_before(thread_instance):
    pass

@after.within_runserver
def something_after(thread_instance):
    pass