garris/backstopjs

Catch CSS curve balls.

backstopjs's Issues

Expose mismatch threshold as a configurable option

Right now it's possible for subtle regressions to be missed by the reporting tool. This is a shame because the great value in a tool like this is catching regressions that are easily missed by humans.

The default value right now is:

var misMatchThresh = 1;

See #21

refactor

hey,

I've found your repo and it looks very interesting... almost what i need :)

I know at the moment there are no test, probably because testing gulp tasks is hard, but if i propose a refactor how likely is a pull-request going to accepted? I ask because the last pull request is sitting still.

I think it makes sense to pull out as much code as possible into an index.js to allow people require this like a node module and exposing an API. the gulp file would then also require this file keeping it backward compatible.

A second pull request would be to make the configuration to be slightly more flexible and I propose a change to allow the user to choose where to save the screenshots i.e. not within bower_components.

The final thing would be to enhance the tasks to respond with exit code depending on if there was an error or not. Hopefully this could also be applied to the comparison tool too.

We need these things are about to start the work anyway i was just wondering how likely you are to accept these changes considering there are no tests yet?

thanks!

Work outside /node_modules possible?

Hello,

I am enjoying backstop.js but would enjoy it even more if we didn't have to work inside the /node_modules/ folder. This folder I do not check into the repository generally.

In a team setting I'd like to be able to simply npm install the module and work with it from the root of the project.. much like most other node modules. This would help for team based development. Is this possible and if so can I see examples?

I'm thinking this kind of scenario:
/node_modules/backstopjs/
/bitmaps_reference/ (configurable)
/bitmaps_test/ (configurable)
/backstop.json
Gulpfile.js

It looks like I could maybe hack the /node_modules/gulp/utils/paths.js file's paths but that's too hairy for me and I'd be manipulating the module and again,..I'd rather not checkin the node_modules folder.

Any ideas?

are all CSS selectors that can be used in backstop.json file?

For eg +, >, '~'.

I tried using these and I found none of them are actually working.

update gulpfile.js

gulp.run() has been deprecated. Use task dependencies or gulp.watch task triggering instead.

...

events.js:72
throw er; // Unhandled 'error' event
^
Error: spawn ENOENT
at errnoException (child_process.js:1001:11)
at Process.ChildProcess._handle.onexit (child_process.js:792:34)

If bower directory is different from bower_components, angular APP has wrong JS paths

I have following folder structure:

• .bowerrc - contains:

{
  "directory": "patternlab-source/bower_components"
}

• patternlab-source/bower_components/backstopjs/compare/patternlab-source/bower_components/angular
• patternlab-source/bower_components/backstopjs/compare/patternlab-source/bower_components/angular-route
• ...

Paths in Angular App do not match folder structure - screenshot . To fix the issue path should contain custom folder defined in .bowerrc file. The path to angular JS will look like:

http://localhost:3001/compare/patternlab-source/bower_components/...

Delete

Implement global settings node in config

Add global property settings in config.

Namely:

• readyEvent
• delay
• threshold
• serverShutDownTime
• cookiesJsonPath
• casperScriptPath

Based on popular requests.

ENOENT error

When i try to run this i get an error, i followed the steps and im not sure what im missing.

To get the error I went to the specified folder and ran gulp echo, this is the message received:

[11:51:14] Using gulpfile ~\Projects\git\myCoolProject\bower_components\backstopjs\gulpfile.js
[11:51:14] Starting 'echo'...
[11:51:14] Finished 'echo' after 2.22 ms

events.js:72
throw er; // Unhandled 'error' event
^
Error: spawn ENOENT
at errnoException (child_process.js:1001:11)
at Process.ChildProcess._handle.onexit (child_process.js:792:34)

Thanks!

installation issues

Not sure why I am getting this. Fresh install

$ npm install --save-dev backstopjs
/
> [email protected] install /Users/dan/Documents/Stringify/old_site/node_modules/backstopjs/node_modules/bunyan/node_modules/dtrace-provider
> node-gyp rebuild

  ACTION binding_gyp_libusdt_target_build_libusdt .
Building libusdt for x86_64
rm -f *.gch
rm -f *.o
rm -f libusdt.a
rm -f test_usdt
rm -f test_usdt32
rm -f test_usdt64
rm -f test_mem_usage
gcc -O2 -arch x86_64   -c -o usdt.o usdt.c
usdt.c:212:9: warning: implicit declaration of function 'usdt_dof_section_free' is invalid in C99
      [-Wimplicit-function-declaration]
        usdt_dof_section_free(&strtab);
        ^
usdt.c:238:9: warning: implicit declaration of function 'usdt_dof_file_free' is invalid in C99
      [-Wimplicit-function-declaration]
        usdt_dof_file_free(provider->file);
        ^
2 warnings generated.
gcc -O2 -arch x86_64   -c -o usdt_dof_file.o usdt_dof_file.c
gcc -arch x86_64 -o usdt_tracepoints.o -c usdt_tracepoints_x86_64.s
gcc -O2 -arch x86_64   -c -o usdt_probe.o usdt_probe.c
gcc -O2 -arch x86_64   -c -o usdt_dof.o usdt_dof.c
gcc -O2 -arch x86_64   -c -o usdt_dof_sections.o usdt_dof_sections.c
rm -f libusdt.a
ar cru libusdt.a usdt.o usdt_dof_file.o usdt_tracepoints.o usdt_probe.o usdt_dof.o usdt_dof_sections.o
ranlib libusdt.a
  TOUCH Release/obj.target/libusdt.stamp
  CXX(target) Release/obj.target/DTraceProviderBindings/dtrace_provider.o
In file included from ../dtrace_provider.cc:1:
../dtrace_provider.h:69:44: error: no type named 'Arguments' in namespace 'v8'; did you mean
      'v8::internal::Arguments'?
    static v8::Handle<v8::Value> New(const v8::Arguments& args);
                                           ^~~~~~~~~~~~~
                                           v8::internal::Arguments
/Users/dan/.node-gyp/0.12.0/deps/v8/include/v8.h:127:7: note: 'v8::internal::Arguments' declared here
class Arguments;
      ^
In file included from ../dtrace_provider.cc:1:
../dtrace_provider.h:70:45: error: no type named 'Arguments' in namespace 'v8'; did you mean
      'v8::internal::Arguments'?
    static v8::Handle<v8::Value> Fire(const v8::Arguments& args);
                                            ^~~~~~~~~~~~~
                                            v8::internal::Arguments
/Users/dan/.node-gyp/0.12.0/deps/v8/include/v8.h:127:7: note: 'v8::internal::Arguments' declared here
class Arguments;
      ^
In file included from ../dtrace_provider.cc:1:
../dtrace_provider.h:87:44: error: no type named 'Arguments' in namespace 'v8'; did you mean
      'v8::internal::Arguments'?
    static v8::Handle<v8::Value> New(const v8::Arguments& args);
                                           ^~~~~~~~~~~~~
                                           v8::internal::Arguments
/Users/dan/.node-gyp/0.12.0/deps/v8/include/v8.h:127:7: note: 'v8::internal::Arguments' declared here
class Arguments;
      ^
In file included from ../dtrace_provider.cc:1:
../dtrace_provider.h:88:49: error: no type named 'Arguments' in namespace 'v8'; did you mean
      'v8::internal::Arguments'?
    static v8::Handle<v8::Value> AddProbe(const v8::Arguments& args);
                                                ^~~~~~~~~~~~~
                                                v8::internal::Arguments
/Users/dan/.node-gyp/0.12.0/deps/v8/include/v8.h:127:7: note: 'v8::internal::Arguments' declared here
class Arguments;
      ^
In file included from ../dtrace_provider.cc:1:
../dtrace_provider.h:89:52: error: no type named 'Arguments' in namespace 'v8'; did you mean
      'v8::internal::Arguments'?
    static v8::Handle<v8::Value> RemoveProbe(const v8::Arguments& args);
                                                   ^~~~~~~~~~~~~
                                                   v8::internal::Arguments
/Users/dan/.node-gyp/0.12.0/deps/v8/include/v8.h:127:7: note: 'v8::internal::Arguments' declared here
class Arguments;
      ^
In file included from ../dtrace_provider.cc:1:
../dtrace_provider.h:90:47: error: no type named 'Arguments' in namespace 'v8'; did you mean
      'v8::internal::Arguments'?
    static v8::Handle<v8::Value> Enable(const v8::Arguments& args);
                                              ^~~~~~~~~~~~~
                                              v8::internal::Arguments
/Users/dan/.node-gyp/0.12.0/deps/v8/include/v8.h:127:7: note: 'v8::internal::Arguments' declared here
class Arguments;
      ^
In file included from ../dtrace_provider.cc:1:
../dtrace_provider.h:91:48: error: no type named 'Arguments' in namespace 'v8'; did you mean
      'v8::internal::Arguments'?
    static v8::Handle<v8::Value> Disable(const v8::Arguments& args);
                                               ^~~~~~~~~~~~~
                                               v8::internal::Arguments
/Users/dan/.node-gyp/0.12.0/deps/v8/include/v8.h:127:7: note: 'v8::internal::Arguments' declared here
class Arguments;
      ^
In file included from ../dtrace_provider.cc:1:
../dtrace_provider.h:92:45: error: no type named 'Arguments' in namespace 'v8'; did you mean
      'v8::internal::Arguments'?
    static v8::Handle<v8::Value> Fire(const v8::Arguments& args);
                                            ^~~~~~~~~~~~~
                                            v8::internal::Arguments
/Users/dan/.node-gyp/0.12.0/deps/v8/include/v8.h:127:7: note: 'v8::internal::Arguments' declared here
class Arguments;
      ^
../dtrace_provider.cc:23:17: error: calling a protected constructor of class 'v8::HandleScope'
    HandleScope scope;
                ^
/Users/dan/.node-gyp/0.12.0/deps/v8/include/v8.h:816:13: note: declared protected here
  V8_INLINE HandleScope() {}
            ^
../dtrace_provider.cc:25:55: error: cannot initialize a parameter of type 'v8::Isolate *' with an lvalue of type
      'v8::Handle<v8::Value> (const v8::internal::Arguments &)'
    Local<FunctionTemplate> t = FunctionTemplate::New(DTraceProvider::New);
                                                      ^~~~~~~~~~~~~~~~~~~
/Users/dan/.node-gyp/0.12.0/deps/v8/include/v8.h:3456:16: note: passing argument to parameter 'isolate' here
      Isolate* isolate,
               ^
../dtrace_provider.cc:26:58: error: 'New' is a private member of 'v8::PersistentBase<v8::FunctionTemplate>'
    constructor_template = Persistent<FunctionTemplate>::New(t);
                                                         ^
/Users/dan/.node-gyp/0.12.0/deps/v8/include/v8.h:572:23: note: declared private here
  V8_INLINE static T* New(Isolate* isolate, T* that);
                      ^
../dtrace_provider.cc:26:28: error: too few arguments to function call, expected 2, have 1; did you mean
      'DTraceProbe::New'?
    constructor_template = Persistent<FunctionTemplate>::New(t);
                           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                           DTraceProbe::New
../dtrace_provider.h:69:34: note: 'DTraceProbe::New' declared here
    static v8::Handle<v8::Value> New(const v8::Arguments& args);
                                 ^
../dtrace_provider.cc:27:25: error: member reference type 'Persistent<v8::FunctionTemplate>' is not a pointer; maybe
      you meant to use '.'?
    constructor_template->InstanceTemplate()->SetInternalFieldCount(1);
    ~~~~~~~~~~~~~~~~~~~~^~
                        .
../dtrace_provider.cc:27:27: error: no member named 'InstanceTemplate' in 'v8::Persistent<v8::FunctionTemplate,
      v8::NonCopyablePersistentTraits<v8::FunctionTemplate> >'
    constructor_template->InstanceTemplate()->SetInternalFieldCount(1);
    ~~~~~~~~~~~~~~~~~~~~  ^
../dtrace_provider.cc:28:25: error: member reference type 'Persistent<v8::FunctionTemplate>' is not a pointer; maybe
      you meant to use '.'?
    constructor_template->SetClassName(String::NewSymbol("DTraceProvider"));
    ~~~~~~~~~~~~~~~~~~~~^~
                        .
../dtrace_provider.cc:28:27: error: no member named 'SetClassName' in 'v8::Persistent<v8::FunctionTemplate,
      v8::NonCopyablePersistentTraits<v8::FunctionTemplate> >'
    constructor_template->SetClassName(String::NewSymbol("DTraceProvider"));
    ~~~~~~~~~~~~~~~~~~~~  ^
../dtrace_provider.cc:28:48: error: no member named 'NewSymbol' in 'v8::String'
    constructor_template->SetClassName(String::NewSymbol("DTraceProvider"));
                                       ~~~~~~~~^
../dtrace_provider.cc:30:31: error: no viable conversion from 'Persistent<v8::FunctionTemplate>' to
      'v8::Handle<v8::FunctionTemplate>'
    NODE_SET_PROTOTYPE_METHOD(constructor_template, "addProbe", DTraceProvider::AddProbe);
                              ^~~~~~~~~~~~~~~~~~~~
/Users/dan/.node-gyp/0.12.0/deps/v8/include/v8.h:196:26: note: candidate constructor (the implicit copy constructor)
      not viable: no known conversion from 'Persistent<v8::FunctionTemplate>' to 'const
      v8::Handle<v8::FunctionTemplate> &' for 1st argument
template <class T> class Handle {
                         ^
/Users/dan/.node-gyp/0.12.0/deps/v8/include/v8.h:213:32: note: candidate template ignored: could not match 'Handle'
      against 'Persistent'
  template <class S> V8_INLINE Handle(Handle<S> that)
                               ^
/Users/dan/.node-gyp/0.12.0/src/node.h:244:72: note: passing argument to parameter 'recv' here
inline void NODE_SET_PROTOTYPE_METHOD(v8::Handle<v8::FunctionTemplate> recv,
                                                                       ^
../dtrace_provider.cc:31:31: error: no viable conversion from 'Persistent<v8::FunctionTemplate>' to
      'v8::Handle<v8::FunctionTemplate>'
    NODE_SET_PROTOTYPE_METHOD(constructor_template, "removeProbe", DTraceProvider::RemoveProbe);
                              ^~~~~~~~~~~~~~~~~~~~
/Users/dan/.node-gyp/0.12.0/deps/v8/include/v8.h:196:26: note: candidate constructor (the implicit copy constructor)
      not viable: no known conversion from 'Persistent<v8::FunctionTemplate>' to 'const
      v8::Handle<v8::FunctionTemplate> &' for 1st argument
template <class T> class Handle {
                         ^
/Users/dan/.node-gyp/0.12.0/deps/v8/include/v8.h:213:32: note: candidate template ignored: could not match 'Handle'
      against 'Persistent'
  template <class S> V8_INLINE Handle(Handle<S> that)
                               ^
/Users/dan/.node-gyp/0.12.0/src/node.h:244:72: note: passing argument to parameter 'recv' here
inline void NODE_SET_PROTOTYPE_METHOD(v8::Handle<v8::FunctionTemplate> recv,
                                                                       ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
make: *** [Release/obj.target/DTraceProviderBindings/dtrace_provider.o] Error 1
gyp ERR! build error
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:269:23)
gyp ERR! stack     at ChildProcess.emit (events.js:110:17)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (child_process.js:1067:12)
gyp ERR! System Darwin 14.3.0
gyp ERR! command "node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /Users/dan/Documents/Stringify/old_site/node_modules/backstopjs/node_modules/bunyan/node_modules/dtrace-provider
gyp ERR! node -v v0.12.0
gyp ERR! node-gyp -v v1.0.3
gyp ERR! not ok
npm WARN optional dep failed, continuing [email protected]
[email protected] node_modules/backstopjs
├── [email protected]
├── [email protected]
├── [email protected] ([email protected])
├── [email protected] ([email protected], [email protected], [email protected], [email protected], [email protected])
├── [email protected] ([email protected], [email protected], [email protected], [email protected])
├── [email protected] ([email protected], [email protected], [email protected])
├── [email protected] ([email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected])
└── [email protected] ([email protected])

Using HTTP verbs with snapshot requests

Is there currently any way to test URLs with, say, a POST request? If not, this could be really handy for a lot of folks. It could also represent a part of the solution to the dynamic data issue. For example, my app loads a bunch of dynamic data that changes often. If I could make a POST request with params, I could pass a stub of static data to ensure the reference and the test are using consistent content.

validation of new features: readyEvent and delay

I implemented loadEvent and delay features. Documentation is in the readme.

If anyone happens to get a minute I'd love to get some feedback on those.

Thanks! GS

Accept YAML configuration

YAML has a couple of advantages over JSON.

The main selling point for me is the ability to comment out most of my viewports when I want to use backstopjs for a quick sanity-check.

I think the best would be that the app works if there is a backstop.yaml file in lieu of a backstop.json.

It doesn't look like a big change. If nobody has time to allocate to this I might be able to create a pull-request someday.

Is it possible to see old report comparisons or even archive them

Hi i want to use backstopJS for company development use and i am interested in some tasks which are partially answered here.

At first i think i would be good if the folders for bitmap references and tests can be directed to a user customized folder in the customer-project-folder with using a "config.json" in which the directory of the backstop.json file for defining tests can be set too. I hope this will be possible soon because this would be a high benefit for the tool.

But my question now is, is it possible to see older testruns for archiving development process and track bugsources better.
I think this now is not possible because TEST always compares the newest references with the actual taken snapshots of the webpages. In normal cases this is totally okay, but for comparing old testruns there maybe an archive folder needed which snapshots the actual references and the actual taken test_pictures for comparing them in the future. Maybe with showing a list of ran tests on the index page of the tool when starting REPORT?

If i can help you here just tell me how.

Thanks and regards

Non-conventional code formatting in generated config file

The config file uses tabs instead of spaces. I think that two spaces is more common and widely accepted among front-end developers.

Also, the commas are on the beginning of the next line, when typically I find them on the end of the line.

This is how I would like to format it:

{
  "viewports": [
    {
      "name": "phone",
      "viewport": {
        "width": 320,
        "height": 480
      }
    }, {
      "name": "tablet_v",
      "viewport": {
        "width": 568,
        "height": 1024
      }
    }, {
      "name": "tablet_h",
      "viewport": {
        "width": 1024,
        "height": 768
      }
    }
  ],
  "grabConfigs": [
    {
      "testName": "http://getbootstrap.com",
      "url": "http://getbootstrap.com",
      "hideSelectors": [],
      "removeSelectors": [
        "#carbonads-container"
      ],
      "selectors": [
        "header", "main", "body .bs-docs-featurette:nth-of-type(1)", "body .bs-docs-featurette:nth-of-type(2)", "footer", "body"
      ],
      "readyEvent": null,
      "delay": 500
    }
  ]
}

Zoom images in report

If screenshot is quite large, it is difficult to understand what has been changed. It would be great to zoom images on click. What do you think?

Allow `bitmaps_reference` location to be configurable outside the backstopjs folder

Would be great to allow this file location to be outside the backstopjs folder if possible. I would like to be able to check in the reference folder without having to do the following in the .gitignore file:

node_modules/backstopjs/*
!node_modules/backstopjs/bitmaps_reference/
!node_modules/backstopjs/bitmaps_reference/*

Ability to use Slimerjs?

Is there a way to use slimerjs? in the casper options?

Lazyload and font icons

It seems like backstopJS cannot process font-icons? I am also struggling with images that are lazy loaded. They are not shown at all (https://github.com/aFarkas/lazysizes).

This is the page I am testing (local): http://ggw-prototype.goalgorilla.com/components/teasers.html (WIP)

Does anyone have a clue how to solve this? I want to keep the markup as close to the production code as possible, for obvious reasons.

The snapshot looks like this (missing the preview and the icons are not displayed properly)

Version 0.4.0 stop working

Hey,

I did not review all the changes between 0.3.0 and 0.4.0, but seems the basic command gulp reference is throwing:

[15:21:09] Finished 'reference' after 5.04 ms
CasperJS: CREATING NEW REFERENCE FILES
CasperJS: CasperError: No steps defined, aborting
CasperJS: /usr/lib/node_modules/casperjs/modules/casper.js:1505 in run
/home/user/projects/myCoolProject/node_modules/backstopjs/capture/genBitmaps.js:187

Testing script failed with code: 1

Does anyone know which could be the problem?

In the meantime, I'm going to continue working with 0.3.0 till I get time to debug this issue.

Thanks!

Allow Click or something else client side

I would want to add some code to be executed before the screenshot being taken.
I started to to add the following code into the backstop.json

"testFile" : "styleguide-buttons.js" 

But I'm stuck with the part of executing code from file and pause the generation process (for a click or something).

if(scenario.testFile){
  var testFile = require('./test_files/'+scenario.testFile);
  casper.waitFor(function check(){
    testFile.dosomething(casper)
  });
  casper.log(testFile);
}

Refactor into a Command Line Tool

See discussion:

#31 (comment)
#31 (comment)

1. Define public interface (commands, file locations, configurations).
2. Outline refactoring steps to get there.

Improve code pattern consistency

Hi @garris

Would you like a pull request with a .editorconfig and JSHint tests to improve JS code?

Things like, ensure I used single quotes, ensure I used 4 spaces tabs, remove trailing spaces, ensure I do not let unused variables in the code etc...

I think this repository will receive a lot of pull request and some tests with Travis CI can save a lot of your time.

Thanks.

Install BackstopJS via NPM

The benefit of installing BackstopJS via NPM is that it's dependencies would be installed recursively.

npm install --save-dev garris/BackstopJS

This not only records BackstopJS as a dependency in my project's package.json and installs it into node_modules/, but it also installs BackstopJS dependencies, recorded in node_modules/BackstopJS/package.json, into node_modules/BackstopJS/node_modules.

I'm not sure I would even keep the bower.json at that point. If a user has bower they have npm.

Testing hover states

It's common for menus etc to be displayed on hover/focus of certain elements.

Is it possible to include this in tests?

If so, please provide documentation on this.

Maybe I need to read the Casper/Phantom docs...

Empty markup returned when testing HTTPS pages.

If your configuration is set up to hit a page over HTTPS, then you get the following markup:

CasperJS: CREATING NEW REFERENCE FILES
CasperJS: LOG> CASPER IS RUNNING
CasperJS: LOG> PHANTOM IS RUNNING
CasperJS: 
==================
Current location is https://www.twitter.com
==================

CasperJS: 

<html><head></head><body></body></html>
CasperJS: 
======================
echoFiles has completed 
=======================


Echo files completed.

This isn't an issue with BackstopJS exactly, but how PhantomJS handles pages running under HTTPS. Following advice on StackOverflow here, it can be fixed by passing --ssl-protocol=tlsv1 in with the tests array prior to spawning the CasperJS process (e.g. echo.js).

Obviously this doesn't make sense to add this in globally, but perhaps we could add an option for which SSL protocol to use (if any) in the backstop.json config file? I guess it'd be most suitable in the scenarios section?

I.e.

...
 "scenarios": [
    {
      "label": "Twitter",
      "url": "https://www.twitter.com",
      "ssl-protocol": "tlsv1",
      "hideSelectors": []
    }
 ]

I'm happy to work on a pull request for this, but does the suggested approach make sense?

wrong language set on Github

FYI I'm pretty sure you have your language set to Objective-C instead of JS. I found this repo trending in the Objective-C list here: https://github.com/trending?l=objective-c

Figured I'd point it out as it may hurt discoverability for you :)

spelling error

Nice tool! It says:

"automates CSS regression tesing of your responsive web UI by comparing DOM screenshots at various viewport sizes."

Should say "testing". :)

font rendering issues?

I'm curious to see if anyone else is having trouble getting fonts to render correctly in their screenshots.

I initially thought my problem was that I async everything (async typekit embed, loadCSS for everything else), so I added a readyEvent and a long delay. This appeared to fire correctly (as far as I can tell) but didn't fix the issue of fonts not rendering properly.

So as a test I went and made everything synchronous again. No loadCSS, and the default sync typekit script. Fonts still do not render properly.

Anyone else have this issue? Anything I might be doing wrong?

edit: actually, after more testing, it appears to not even render properly at all, like a significant chunk of the css isn't getting parsed

screen capture for dynamically generated content

Hi,

I'm not sure whether this has been addressed or implemented already as part of the feature of backstopjs. Nevertheless I would raise it otherwise if it hasn't been done.

It would be great to see how you can extend Backstop feature further to capture screenshots for any dynamically generated content such as AJAX calls or animated image/text carousel pages.

Cause what I often find is with these scenarios, (since started using it) I would like to see the dynamic content in its ready/paused state for backstop to take quick snapshots and I didn't have much success on its current build.

Is that the case?

Let me know if you wish me to clarify further.

BackstopJS git workflow

Hello guys,

Super cool and useful project 👍

Can you share the most efficient way for adding bitmaps_reference folder in version control.

Should the reference images be committed on the repo?

Hoping to write this question in the right place:

I'm trying to work with BackstopJS and understanding how it should be used at it's best without misunderstood some point of the regression testing workflow.

I've installed BacktopJS, made some test to see if it's working and it works nice.

Now I'm on a team of back-end developers which can make front-end changes and I would learn if I should commit on the repo the reference images or I need to keep tests green.

In my project I basically add 1 HTML template every day, so I'm supposed to create a reference of images every day.

My doubt is: what about old images, are them deleted and generated again?

In that case how can I ensure I don't loose old valid template image references?

I'm not sure about this point of the testing workflow and how I should handle old reference images.

change localhost/compare url default auto-shutdown time

After I run grunt backstop:test, I noted this line

$ grunt backstop:test
Running "backstop:test" (backstop) task
.......................................
.......................................
.......................................
Server launched in background with PID: 7448
NOTE: Sever will auto-shutdown (default time 15 mins). See documentation for more info.

[11:13:59] Starting 'openReport'...

Opening report ->  http://localhost:3001/compare/
[11:13:59] Finished 'openReport' after 5.16 ms

It says that server will auto-shutdown within 15 mins by default.

How can I modify this to increment the auto-shutdown time. Where is the documentation for this?

Scan URLs with less specific rules

Hey, I thought it might be a good idea to let Backstop scan routes of a website using pattern matching (Regex or simple wildcards). Specifying every single page by hand can be tedious and less maintainable in the future.

What do you think?

Scenario urls: Crawl a set list of pages behind a login

Is it possible to have BackStop crawl a set of pages rather than having to handcode each individual scenario?

Here's the code that defines a scenario in BackStop:

"scenarios": [
{
"label": "http://getbootstrap.com",
"url": "http://getbootstrap.com",
"hideSelectors": [],
"removeSelectors": [
"#carbonads-container"
],
.....

Is it possible to pass in a function in place of a hardcoded url here? So for example:

"scenarios": [
{
"label": "http://getbootstrap.com",
"url": crawlSite(),
"hideSelectors": [],
"removeSelectors": [
"#carbonads-container"
],

The crawlSite() function would be using click() functions to target links used in possibly CasperJS or PhantomJS as a means to crawl the site, where you would use either of those to initiate browser clicks to proceed to the next page in order to compare the reference screenshots. Additionally, the "gulp reference" command would have to crawl the entire site (using a custom PhantomJS/CasperJS script) to make the comparisons against.

Any suggestions?

Docker image?

Hey!

I'm going to setup this as a docker image for simplicity, if you want I can contrib the basic configuration and documentation.

Just let me know,
Thanks!

Spawning child processes on windows

When spawning child processes on Windows (Win7x64) gulp crashes because it cannot find the commands.
gulpfile.js:78

spawn('bower',['install'],{cwd:comparePath});

& gulpfile.js:193

var casperChild = spawn('casperjs', tests);

By checking the process.platform variable first and calling either 'bower.cmd' on windows or 'bower' on MacOS, this problem is solved.

var bowerProcess = (process.platform === "win32" ? "bower.cmd" : "bower");
spawn(bowerProcess,['install'],{cwd:comparePath}).on('error', errorHandler);

&

var casperProcess = (process.platform === "win32" ? "casperjs.cmd" : "casperjs");
var casperChild = spawn(casperProcess, tests);

Hanging on "CasperJS: CREATING NEW REFERENCE FILES"

Everything seems to be good, but it never completes "CasperJS: CREATING NEW REFERENCE FILES" and doesn't generate any images.

Any ideas?

We're on Ubuntu 14.04 and the setup instructions went smoothly.

Uncaught fails?

Hey,

I was testing v0.3.0 today and I noticed that some tests that were caught by resemble.js weren't being marked as fails by the app? Screenshots attached:

Looks like we're almost ready to release the NPM install version.

@jehoshua02 @lewisnyman I think we will want to merge the new work on master into the npm_install branch and validate. With all the clean up you guys put in I think we could call this 1.0. Sound like a good next step?

Use BackstopJS for login-required pages

Hi, I think it'd be nice if I can use BackstopJS for login-required pages.
So I tried to add tiny patch for BackstopJS to handle Cookie.
master...kyanny:addCookie
How do you think? Do you have a plan to support Cookie-based login for BackstopJS?

Doesn't work with phantomJS 2.0

Hello there,

I tried to run gulp reference with phantomJS 2.0 installed, and I got

CasperJS: CasperJS needs PhantomJS v1.x

CasperJS:   /usr/local/lib/node_modules/casperjs/bin/bootstrap.js:91 in __die

Testing script failed with code: 1

Looks like an error occured. You may want to try running `$ gulp echo`. This will echo the requested test URL output to the console. You can check this output to verify that the file requested is indeed being received in the expected format.

Also i tried with phantomJS 1.9.7, and all worked fine

Can you do something for this?
Thanks.

Remove the need for empty configuration objects

Hi, this tool is great! Very usable and simple. Thank you.

One thing I saw that could make it more usable is to reduce the amount of config that is required or assumed, for simpler use cases.

Simple use case
I want to capture the whole page, and hide/remove nothing on the page.

This means that the hideSelectors and removeSelectors are redundant, but removing them from grabConfigs causes this error:
`CasperJS: TypeError: 'undefined' is not an object (evaluating 'grabConfig.hideSelectors.forEach')

In theory we could also provide a default value for selectors of 'body', which would remove the need for that in a simple use case. We'd only need to provide the name and the url.

What do you think?

CasperJS and PhantomJS are BackstopJS Dependencies.

CasperJS and PhantomJS are BackstopJS dependencies. Please consider adding them to package.json and/or bower.json.

Make configurations more flexible

Hey @garris

I have found 2 problems in my progress of make this project more easy to install using docker:

1. The location of the file backstop.json should be configurable using ENV variables or JSON file.
2. From my perspective, the server auto-shutdown should be removed completely from the code, we could handle that using a bash script or anything external, no need to force that by default.

What do you think?

Can I generate and share a report?

• Use case: *
  I'm run some tests on the latest version of code and I've found some failures. Instead of fixing the myself I want to share them with my team so they can fix the problems.

Is there a way to do that now? It seems like I need a server running access the report, there's no way of generating a static report?

Option to control mismatch percent for specific element?

Hi,

Is there any way to change mismatch percent for specific element from backstop.json file ?

No mention of reference or test commands in README

Hello!

I noticed that there are no instructions on how to run the tests in the readme, it looks like they were removed in b03aa5a

Is that intentional?

Start task fails to restart web server

On Windows, once stopped, web server is not starting anymore.

I have found that start task use kill -0 PID to check if process exists. It results in the error: "Command failed: 'kill' is not recognized as an internal or external command, operable program or batch file." when I run it on Windows machine.

I would suggest use either process.kill or npm module https://www.npmjs.com/package/is-running (which actually wraps process.kill).

Testing invisible elements should not result in a screenshot of the whole page

One of my selectors defines an element, which is not visible for smaller viewports. BackstopJS accidentally now takes a screenshot of the whole page.

I think it should just do not create a screenshot, what do you think?