getgauge/gauge

Light weight cross-platform test automation

gauge's Issues

Support to execute multiple scenarios of same spec

gauge specs/helloworld.spec:0 specs/helloworld.spec:1

executes only one first scenario.

Move protobuf import from google.code to github

Hybrid Execution - Allow a test suite with manual and automated tests to be executed

Windows installer should also trigger installation of html-report plugin.

Table not printed with step in console when passed as arg to Concept

Execution of Specifications should be in an order e.g alphabetical

When a spec file is run, all specs should be run in a particular order. Currently its just random.

Pass API port to gauge via command line flag instead of an environment variable

Eg:

gauge --daemonize --api-port 7777

Have a mechanism to ignore a set of scenarios or specifications using tag expressions

Add API message from Gauge Core to get parse errors in spec files.

This is required for IDE plugins to highlight parse errors in specs.

Create installable distros including windows installers from buildscript.

Gauge autocomplete fails if parse/compile fails

Even if there is a problem in building the language implementation, Gauge should ideally return steps to autocomplete.

Gauge should attempt a best-effort approach in responding to API messages, rather than breaking all features.

Spec not marked as failure if step in nested concepts fails

Adding parameter info to getAllStepResponse

Currently allStepsResponse returns a list of strings with placeholders for parameters eg login as {} . We have to change that to login as if it is defined like that in the language end.

If there are multiple usages in the spec side then take the first one and add parameter info even if it is static.

Support installation via homebrew on mac os x

Support installation via homebrew on mac os x.

GetSteps API returns concepts (treated as steps) when they have been used in spec files.

The GetAllSteps API call should return only steps since GetAllConcepts API returns all the concepts.

Download should be tar.gz or any other open format rather than zip for Linux

I just noticed that download links are all .zip files. But the website says instructions for tar.gz.

Expose API message from gauge core with list of concepts with filepath and line number.

This is required for IDE plugins to navigate to a particular Concept definition

Executing gauge specs from outside the project location

We try to get the project root when we are inside the project by going one level up till we get a manifest.json. But if we are outside the project, the search strategy won't work. So will have to handle execution from outside the project also.

Tags not retained after formatting

Provide a hook for screenshots

Currently only the language runner can take screenshots on failure. Need to add a hook to allow taking custom screenshots.

Spaces not handled in Windows

gauge "C:/Users/Ken Mugrage/IdeaProjects/untitled2/specs/hello_world.spec"
Failed to start a runner. exec: "C:\Users\Ken": file does not exist

Process finished with exit code 1

Avoid repeated invoking of runners when executing specs

Gauge invokes instances of runner whenever it wants to scan and run specs separately. This causes the runner to perform redundant start up actions( i.e. build target projects) twice. These operations can become expensive, thus slowing down total execution.

Possible actions:

• Gauge caches the instance of runner until it is done with it.
• Add a flag to signal the runner that it can omit the startup actions.

Specs execute even if the before suite hook fails

Cross platform screenshot for Ruby language runner

Currently ruby runner takes screenshots only on MacOS. Need to add support for linux and windows.

Resolve issue where the protoConcept step arguments are not populated with the actual parameter values.

Gauge step formatting

If there is a step in spec file like:

• Verify that project contains "Step having "" and """

After formatting this becomes,

• Verify that project contains "Step having "" and """
  This becomes an undefined step.

Support for execution of a single scenario

IntelliJ support to execute single scenario.

Support for tagged execution

Ability to execute specifications filtered by Tag

gauge --tags prod,tag2,tag3 specs

Will execute all specs or scenarios tagged with the tags passed (prod, tag2, tag3).

Escaping characters in a table cell value especially '<' and '>'

If a table cell value is '' it is still being treated as a dynamic cell and parser fails.
The dynamic character meaning assigned to "< >" characters should be ignored when they are escaped in table cells.

Print concepts in console on execution

On execution currently the steps inside the concept are printed on the console without any information of which concept is executing.

Add a platform not supported message if the plugin.json does not have install details for a particular platform.

When installing a plugin with gauge --install if the plugin is not supported for the particular platform then an error message should be displayed.

This is particularly important for plugins like gauge-csharp which do not work across platforms.

Html-report aligning issue with stacktrace and screenshot

When stacktrace is wide, the screenshot isn't shown. so will have to wrap it.

Support for Installation of plugins directly from the zip files instead of downloading it each time.

Make special parameters a plugin

Currently the file and table special parameters are added into gauge. The ability to write a special parameter plugin needs to be provided.

Add support to update plugins

Something like

gauge --update csharp

should fetch the latest csharp plugin compatible with the current gauge version.

Add Refactoring of Steps in the spec files.

Add refactoring via command line for all spec files in the project.

Example usage

gauge --refactor "step <a> and <b>" "step with <b>"

Extract text out of code

use properties file or i18n packages

Gauge does not provide steps to IDE plugins in autocomplete which are used in only concepts

After refactoring steps tags are not retained in spec files

Check for concepts with circular concept referencing

Concepts parsing should fail if any nested concept references any higher level concept.
Example

# Top level concept
* nested concept

## nested concept
* Top level concept

Make Java compilation directory configurable

Currently it compiles all the files under src. This might be alright for 90% of the projects. But if they are using gauge together with the application code, then there will too many files which needs to be compiled unnecessarily. So have a configuration in .properties file which will say the directory to be compiled. So only tests directory can be specified.

It was compiling only src/test/java directory. But it was removed by the commit 9d38589. See the commit message for more information.

Concept execution continues even if some step fails in it

In a nested concept, even if some step fails, it marks it failed and continues the execution.

Have Timeout mechanism for refactoring if runner is taking too long to respond

Add timeout support for long running tests

Sometimes the underlying step implementation waits for a third party to respond. Rather than having the tests hang, have a way to introduce timeout on actions.

Write logs to log files especially from the api service.

Parsing fails when number of rows is more than number of columns in tables (inline or data table)

Configurable timeout for message reading

If a message request takes longer time to respond than a specified property(lets say message.read.timeout). This helps to identify potential infinite loop or unimplemented message type at the runner end.

Cannot run multiple gauge instances parallely

This is particularly important as every IDE project would invoke gauge --daemonize to communicate over the Gauge API.

If there are two projects foo and bar, the first one to receive a gauge --daemonize works well and is listening for messages.

The second one throws an error such as:

> gauge --daemonize
fatal error: all goroutines are asleep - deadlock!

goroutine 16 [semacquire]:
sync.runtime_Semacquire(0xc082000b88)
        c:/go/src/pkg/runtime/sema.goc:199 +0x37
sync.(*WaitGroup).Wait(0xc082006e20)
        c:/go/src/pkg/sync/waitgroup.go:129 +0x152
main.runInBackground()
        c:/Users/srikantv/projects/gauge/tmp/src/gauge/gauge.go:113 +0x175
main.main()
        c:/Users/srikantv/projects/gauge/tmp/src/gauge/gauge.go:44 +0x3b

goroutine 19 [finalizer wait]:
runtime.park(0x456920, 0xa090f0, 0xa07229)
        c:/go/src/pkg/runtime/proc.c:1369 +0xac
runtime.parkunlock(0xa090f0, 0xa07229)
        c:/go/src/pkg/runtime/proc.c:1385 +0x42
runfinq()
        c:/go/src/pkg/runtime/mgc0.c:2644 +0xdd
runtime.goexit()
        c:/go/src/pkg/runtime/proc.c:1445

spec Items (context) is not getting resovled in the reports

If the context has some parameters it should be resolved.

Add support for running specs in parallel