python-needle / needle Goto Github PK
View Code? Open in Web Editor NEWAutomated tests for your CSS.
Home Page: https://needle.readthedocs.io/
License: Other
needle's People
Contributors
Stargazers
Watchers
Forkers
rosscdh mfelsche repos-css acdha wildone treyhunner jphalip mimundome vokkim dalejung mbertheau ajay05 rustoceans frewsxcv avorio justiniso kimpetersend1 sri85 sterago cgoldberg hizhaoxiaoyang xerxesnoble jmbowman yileye devenlu skudasov copyleftdev avinash9 moellering az-pz jorenko oksumoron sawt00th webslinger joshuamckenty sanjaydatasciencedojo cisco-rich vicemedia sonofvanu pawelkuznik qualityenablement partosxp iq-scm mindojoneedle's Issues
Support for Selenium 3
setup.py defines a requirement for selenium <3
Are there any plans to remove this restriction?
Test various mobile device and browser
Hi, is it possible to test various mobile devices and browser?
If so, how can I do that?
Thank you!
WebDriverException on clean install
I try to run simple test from /test directory and I get an error like
~/needle/tests$ nosetests3 test_driver.py --with-save-baseline
...EE
======================================================================
ERROR: test_get_dimensions (tests.test_driver.TestWebElement)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/damian/needle/tests/test_driver.py", line 47, in test_get_dimensions
self.assertEqual(e.get_dimensions(), {
File "/usr/local/lib/python3.5/dist-packages/needle/driver.py", line 44, in get_dimensions
location = self.location
File "/home/damian/.local/lib/python3.5/site-packages/selenium/webdriver/remote/webelement.py", line 402, in location
old_loc = self._execute(Command.GET_ELEMENT_LOCATION)['value']
File "/home/damian/.local/lib/python3.5/site-packages/selenium/webdriver/remote/webelement.py", line 494, in _execute
return self._parent.execute(command, params)
File "/home/damian/.local/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 236, in execute
self.error_handler.check_response(response)
File "/home/damian/.local/lib/python3.5/site-packages/selenium/webdriver/remote/errorhandler.py", line 192, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: GET /session/cd8c0384-21b8-49a4-8493-e7dd682e33cf/element/85d8038e-6e97-4c88-b62f-672cca2f95be/location did not match a known command
General external comparison tool support
We should probably have a simple configurable way to use external comparison tools like the ImageMagick/GraphicsMagick convert utilities. The current support for perceptualdiff could be replaced with this, which would also provide a way to customize arguments.
Android Exception
Hi there,
I some more exceptions for you for Android devices.
For one I had to comment out the section of resizing the window as Android Driver does not support that. Furthermore there seems to be a problem with getting the element dimensions to crop the screenshot.
======================================================================
ERROR: test_footer (test_basic_android_s3.basicTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "D:\xampp\htdocs\mop\relaunch\__inc\tests\run\test_basic_android_s3.py",
line 46, in test_footer
self.assertScreenshot("#footer", "basic.view1.ANDROID.S3.footer")
File "D:\Python27\lib\site-packages\needle\cases.py", line 128, in assertScree
nshot
pass
File "D:\Python27\lib\contextlib.py", line 24, in __exit__
self.gen.next()
File "D:\Python27\lib\site-packages\needle\cases.py", line 187, in compareScre
enshot
element.get_screenshot().save(baseline_file)
File "D:\Python27\lib\site-packages\needle\driver.py", line 64, in get_screens
hot
d['left'],
TypeError: 'NoneType' object has no attribute '__getitem__'
-------------------- >> begin captured stdout << ---------------------
None
--------------------- >> end captured stdout << ----------------------
-------------------- >> begin captured logging << --------------------
selenium.webdriver.remote.remote_connection: DEBUG: POST http://khaireddinhussei
1:[email protected]:80/wd/hub/session {"desiredCapabilities": {"d
evice": "Samsung Galaxy S III", "platform": "ANDROID", "browserName": "android",
"javascriptEnabled": true, "browserstack.local": true}}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://khaireddinhussei
1:[email protected]:80/wd/hub/session/2cc0ef9706acd072dbf23cf8ba7
079d6c51f8e4e/url {"url": "http://localhost/mop/relaunch/home.htm", "sessionId":
"2cc0ef9706acd072dbf23cf8ba7079d6c51f8e4e"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://khaireddinhussei
1:[email protected]:80/wd/hub/session/2cc0ef9706acd072dbf23cf8ba7
079d6c51f8e4e/execute {"sessionId": "2cc0ef9706acd072dbf23cf8ba7079d6c51f8e4e",
"args": [], "script": "window.scrollTo(0,$('#footer').offset().top);"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://khaireddinhussei
1:[email protected]:80/wd/hub/session/2cc0ef9706acd072dbf23cf8ba7
079d6c51f8e4e/execute {"sessionId": "2cc0ef9706acd072dbf23cf8ba7079d6c51f8e4e",
"args": [], "script": "window.scrollTo(0,0)"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://khaireddinhussei
1:[email protected]:80/wd/hub/session/2cc0ef9706acd072dbf23cf8ba7
079d6c51f8e4e/element {"using": "css selector", "sessionId": "2cc0ef9706acd072db
f23cf8ba7079d6c51f8e4e", "value": "#footer"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://khaireddinhussei
1:[email protected]:80/wd/hub/session/2cc0ef9706acd072dbf23cf8ba7
079d6c51f8e4e/execute {"sessionId": "2cc0ef9706acd072dbf23cf8ba7079d6c51f8e4e",
"args": [], "script": "return typeof(jQuery)"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://khaireddinhussei
1:[email protected]:80/wd/hub/session/2cc0ef9706acd072dbf23cf8ba7
079d6c51f8e4e/execute {"sessionId": "2cc0ef9706acd072dbf23cf8ba7079d6c51f8e4e",
"args": [{"ELEMENT": "0"}], "script": "\n var e = $(arguments[0]);\n
var offset = e.offset();\n var dimensions = {\n
'width': e.outerWidth(),\n 'height': e.outerHeight(),\n
'left': Math.floor(offset.left),\n 'top': Math.floor(o
ffset.top)\n };\n return dimensions;\n "}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: GET http://khaireddinhussei1
:[email protected]:80/wd/hub/session/2cc0ef9706acd072dbf23cf8ba70
79d6c51f8e4e/screenshot {"sessionId": "2cc0ef9706acd072dbf23cf8ba7079d6c51f8e4e"
}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
--------------------- >> end captured logging << ---------------------
======================================================================
ERROR: test_header (test_basic_android_s3.basicTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "D:\xampp\htdocs\mop\relaunch\__inc\tests\run\test_basic_android_s3.py",
line 28, in test_header
self.assertScreenshot("#header", "basic.view1.ANDROID.S3.header")
File "D:\Python27\lib\site-packages\needle\cases.py", line 128, in assertScree
nshot
pass
File "D:\Python27\lib\contextlib.py", line 24, in __exit__
self.gen.next()
File "D:\Python27\lib\site-packages\needle\cases.py", line 187, in compareScre
enshot
element.get_screenshot().save(baseline_file)
File "D:\Python27\lib\site-packages\needle\driver.py", line 64, in get_screens
hot
d['left'],
TypeError: 'NoneType' object has no attribute '__getitem__'
-------------------- >> begin captured stdout << ---------------------
None
--------------------- >> end captured stdout << ----------------------
-------------------- >> begin captured logging << --------------------
selenium.webdriver.remote.remote_connection: DEBUG: POST http://khaireddinhussei
1:[email protected]:80/wd/hub/session/2cc0ef9706acd072dbf23cf8ba7
079d6c51f8e4e/url {"url": "http://localhost/mop/relaunch/home.htm", "sessionId":
"2cc0ef9706acd072dbf23cf8ba7079d6c51f8e4e"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://khaireddinhussei
1:[email protected]:80/wd/hub/session/2cc0ef9706acd072dbf23cf8ba7
079d6c51f8e4e/execute {"sessionId": "2cc0ef9706acd072dbf23cf8ba7079d6c51f8e4e",
"args": [], "script": "window.scrollTo(0,$('#header').offset().top);"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://khaireddinhussei
1:[email protected]:80/wd/hub/session/2cc0ef9706acd072dbf23cf8ba7
079d6c51f8e4e/execute {"sessionId": "2cc0ef9706acd072dbf23cf8ba7079d6c51f8e4e",
"args": [], "script": "window.scrollTo(0,0)"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://khaireddinhussei
1:[email protected]:80/wd/hub/session/2cc0ef9706acd072dbf23cf8ba7
079d6c51f8e4e/element {"using": "css selector", "sessionId": "2cc0ef9706acd072db
f23cf8ba7079d6c51f8e4e", "value": "#header"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://khaireddinhussei
1:[email protected]:80/wd/hub/session/2cc0ef9706acd072dbf23cf8ba7
079d6c51f8e4e/execute {"sessionId": "2cc0ef9706acd072dbf23cf8ba7079d6c51f8e4e",
"args": [], "script": "return typeof(jQuery)"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://khaireddinhussei
1:[email protected]:80/wd/hub/session/2cc0ef9706acd072dbf23cf8ba7
079d6c51f8e4e/execute {"sessionId": "2cc0ef9706acd072dbf23cf8ba7079d6c51f8e4e",
"args": [{"ELEMENT": "1"}], "script": "\n var e = $(arguments[0]);\n
var offset = e.offset();\n var dimensions = {\n
'width': e.outerWidth(),\n 'height': e.outerHeight(),\n
'left': Math.floor(offset.left),\n 'top': Math.floor(o
ffset.top)\n };\n return dimensions;\n "}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: GET http://khaireddinhussei1
:[email protected]:80/wd/hub/session/2cc0ef9706acd072dbf23cf8ba70
79d6c51f8e4e/screenshot {"sessionId": "2cc0ef9706acd072dbf23cf8ba7079d6c51f8e4e"
}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
--------------------- >> end captured logging << ---------------------
======================================================================
ERROR: test_menu (test_basic_android_s3.basicTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "D:\xampp\htdocs\mop\relaunch\__inc\tests\run\test_basic_android_s3.py",
line 37, in test_menu
self.assertScreenshot("#navigation", "basic.view1.ANDROID.S3.menu")
File "D:\Python27\lib\site-packages\needle\cases.py", line 128, in assertScree
nshot
pass
File "D:\Python27\lib\contextlib.py", line 24, in __exit__
self.gen.next()
File "D:\Python27\lib\site-packages\needle\cases.py", line 187, in compareScre
enshot
element.get_screenshot().save(baseline_file)
File "D:\Python27\lib\site-packages\needle\driver.py", line 64, in get_screens
hot
d['left'],
TypeError: 'NoneType' object has no attribute '__getitem__'
-------------------- >> begin captured stdout << ---------------------
None
--------------------- >> end captured stdout << ----------------------
-------------------- >> begin captured logging << --------------------
selenium.webdriver.remote.remote_connection: DEBUG: POST http://khaireddinhussei
1:[email protected]:80/wd/hub/session/2cc0ef9706acd072dbf23cf8ba7
079d6c51f8e4e/url {"url": "http://localhost/mop/relaunch/home.htm", "sessionId":
"2cc0ef9706acd072dbf23cf8ba7079d6c51f8e4e"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://khaireddinhussei
1:[email protected]:80/wd/hub/session/2cc0ef9706acd072dbf23cf8ba7
079d6c51f8e4e/execute {"sessionId": "2cc0ef9706acd072dbf23cf8ba7079d6c51f8e4e",
"args": [], "script": "window.scrollTo(0,$('#navigation').offset().top);"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://khaireddinhussei
1:[email protected]:80/wd/hub/session/2cc0ef9706acd072dbf23cf8ba7
079d6c51f8e4e/execute {"sessionId": "2cc0ef9706acd072dbf23cf8ba7079d6c51f8e4e",
"args": [], "script": "window.scrollTo(0,0)"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://khaireddinhussei
1:[email protected]:80/wd/hub/session/2cc0ef9706acd072dbf23cf8ba7
079d6c51f8e4e/element {"using": "css selector", "sessionId": "2cc0ef9706acd072db
f23cf8ba7079d6c51f8e4e", "value": "#navigation"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://khaireddinhussei
1:[email protected]:80/wd/hub/session/2cc0ef9706acd072dbf23cf8ba7
079d6c51f8e4e/execute {"sessionId": "2cc0ef9706acd072dbf23cf8ba7079d6c51f8e4e",
"args": [], "script": "return typeof(jQuery)"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://khaireddinhussei
1:[email protected]:80/wd/hub/session/2cc0ef9706acd072dbf23cf8ba7
079d6c51f8e4e/execute {"sessionId": "2cc0ef9706acd072dbf23cf8ba7079d6c51f8e4e",
"args": [{"ELEMENT": "2"}], "script": "\n var e = $(arguments[0]);\n
var offset = e.offset();\n var dimensions = {\n
'width': e.outerWidth(),\n 'height': e.outerHeight(),\n
'left': Math.floor(offset.left),\n 'top': Math.floor(o
ffset.top)\n };\n return dimensions;\n "}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: GET http://khaireddinhussei1
:[email protected]:80/wd/hub/session/2cc0ef9706acd072dbf23cf8ba70
79d6c51f8e4e/screenshot {"sessionId": "2cc0ef9706acd072dbf23cf8ba7079d6c51f8e4e"
}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 3 tests in 185.073s
FAILED (errors=3)
Here is my test:
# encoding: utf-8
from needle.cases import NeedleTestCase
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
import needle.driver
import yaml
import time
class basicTest(NeedleTestCase):
engine_class = 'needle.engines.perceptualdiff_engine.Engine'
base_url = 'http://localhost/mop/relaunch/'
@classmethod
def get_web_driver(cls):
desired_cap = {'browserName': 'android', 'platform': 'ANDROID', 'device': 'Samsung Galaxy S III',"javascriptEnabled": True}
desired_cap['browserstack.local'] = True
return needle.driver.NeedleRemote(
command_executor='http://[snip]:[snip]@hub.browserstack.com:80/wd/hub',
desired_capabilities=desired_cap)
def test_header(self):
self.driver.get(self.base_url + 'home.htm')
time.sleep(0.5)
self.driver.execute_script("window.scrollTo(0,$('#header').offset().top);")
time.sleep(0.5)
self.driver.execute_script("window.scrollTo(0,0)")
self.assertScreenshot("#header", "basic.view1.ANDROID.S3.header")
def test_menu(self):
self.driver.get(self.base_url + 'home.htm')
time.sleep(0.5)
self.driver.execute_script("window.scrollTo(0,$('#navigation').offset().top);")
time.sleep(0.5)
self.driver.execute_script("window.scrollTo(0,0)")
self.assertScreenshot("#navigation", "basic.view1.ANDROID.S3.menu")
def test_footer(self):
self.driver.get(self.base_url + 'home.htm')
time.sleep(0.5)
self.driver.execute_script("window.scrollTo(0,$('#footer').offset().top);")
time.sleep(0.5)
self.driver.execute_script("window.scrollTo(0,0)")
self.assertScreenshot("#footer", "basic.view1.ANDROID.S3.footer")
Tests don't run propery
Expected result
I clone, install needle and then run tox. All tests should pass.
Actual result
[...] FAIL: test_existing_baseline_not_overwritten (tests.test_plugin.NeedleCaptureOverwriteTest)
[...] FAIL: test_baseline_is_saved (tests.test_plugin.NeedleCaptureTest)
[...] FAIL: test_screenshot_is_cleanedup (tests.test_plugin.NeedleCleanupOnSuccessTest)
and many more. I have appended the full output as a file.
Reproduce
git clone https://github.com/python-needle/needle.git
cd needle/
mkvirtualenv needle
pip install .
pip install tox
tox
Remarks
I did not find any further documentation on how to prepare the testing environment so this may be due to some missing configuration steps. If that is the case I'd like to point this issue into a "missing documentation" issue.
Environment
pip freeze needle==0.5.0 nose==1.3.7 Pillow==5.0.0 pluggy==0.6.0 py==1.5.2 selenium==3.8.1 six==1.11.0 tox==2.9.1 virtualenv==15.1.0
Arch Linux: 4.14.15-1-ARCH
Python 3.6.4
pip 9.0.1
virtualenv 15.1.0
ChromeDriver 2.35.528139
geckodriver 0.19.0
Fix tests for Python 2.6 / or drop Python 2.6
The tests are currently failing on Travis since the recent release of Pillow 4.0.0, which is dropping Python 2.6 support: http://pillow.readthedocs.io/en/stable/releasenotes/4.0.0.html#python-2-6-and-3-2-dropped
We should either fix our test matrix, or drop Python 2.6 support ourselves too as it's not official supported by the core Python team any more.
Element screenshots incorrect when page has been scrolled
When the page has been scrolled, the element coordinates returned by NeedleWebElementMixin.get_dimensions() are relative to the top-left of the page, but the screenshot that get_screenshot() crops starts at the current scroll position. This results in the returned screenshot being of either the wrong part of the page or mostly/all blank, depending on the browser. To fix this, the current scroll position needs to be subtracted from the position, like so:
diff --git a/home/jjurack/.local/share/virtualenvs/test_web-6ViYfjms/lib/python3.6/site-packages/needle/driver-orig.py b/home/jjurack/.local/share/virtualenvs/test_web-6ViYfjms/lib/python3.6/site-packages/needle/driver.py
index cd10b49..eceafe5 100644
--- a/home/jjurack/.local/share/virtualenvs/test_web-6ViYfjms/lib/python3.6/site-packages/needle/driver-orig.py
+++ b/home/jjurack/.local/share/virtualenvs/test_web-6ViYfjms/lib/python3.6/site-packages/needle/driver.py
@@ -63,8 +63,8 @@ class NeedleWebElementMixin(object):
d = self.get_dimensions()
# Cast values to int in order for _ImageCrop not to break
- d['left'] = int(d['left'])
- d['top'] = int(d['top'])
+ d['left'] = int(d['left']) - self._parent.execute_script('return window.scrollX')
+ d['top'] = int(d['top']) - self._parent.execute_script('return window.scrollY')
d['width'] = int(d['width'])
d['height'] = int(d['height'])
Saving new updates to screenshot requires new filename
Running nosetests test_needle.py --with-needle-capture will not overwrite the existing file, so you have to create a new file altogether.
Add changelog
Versioning is lazy at the moment.
Can't import cases from needle
Hello, there is some import problem. I follow step by step tutorial, also this one http://the-creative-tester.github.io/Python-Visual-Regression-Testing/, but the problem is on the import stage.
----------------------------------------------------------------------
Traceback (most recent call last):
File "c:\dev\test\env\lib\site-packages\nose\failure.py", line 39, in runTest
raise self.exc_val.with_traceback(self.tb)
File "c:\dev\test\env\lib\site-packages\nose\loader.py", line 418, in loadTestsFromName
addr.filename, addr.module)
File "c:\dev\test\env\lib\site-packages\nose\importer.py", line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
File "c:\dev\test\env\lib\site-packages\nose\importer.py", line 92, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "C:\Python36\lib\imp.py", line 235, in load_module
return load_source(name, filename, file)
File "C:\Python36\lib\imp.py", line 172, in load_source
module = _load(spec)
File "<frozen importlib._bootstrap>", line 684, in _load
File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 678, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "C:\dev\test\needle.py", line 1, in <module>
from needle.cases import NeedleTestCase
ModuleNotFoundError: No module named 'needle.cases'; 'needle' is not a package
Python 3.6, Windows 10
PerceptualDiff and Python 3.4.1 exception
Have some prototype tests written using Needle and Python 3.4.1. When the tests run with the default image comparison setting they run fine. When I added PerceptualDiff to the tests, an ugly exception gets thrown coming out of PIL.
Traceback (most recent call last):
File "D:\Temp\TestDev\Needle_Tests_Workflow\Needle_Tests_Workflow\Needle_Tests_Workflow.py", line 42, in test_loginPage
self.assertScreenshot('body', 'body')
File "D:\Python34\lib\site-packages\needle\cases.py", line 128, in assertScreenshot
pass
File "D:\Python34\lib\contextlib.py", line 66, in exit
next(self.gen)
File "D:\Python34\lib\site-packages\needle\cases.py", line 198, in compareScreenshot
self.engine.assertSameFiles(output_file, baseline_file, threshold)
File "D:\Python34\lib\site-packages\needle\engines\perceptualdiff_engine.py", line 16, in assertSameFiles
width, height = Image.open(open(output_file)).size
File "D:\Python34\lib\site-packages\PIL\Image.py", line 2255, in open
prefix = fp.read(16)
File "D:\Python34\lib\encodings\cp1252.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
nose.proxy.UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 49: character maps to
Add Firefox to the Travis test matrix
Currently we only test with PhantomJS. For more completeness, it'd be useful to add Firefox following the steps here: https://docs.travis-ci.com/user/gui-and-headless-browsers/#Using-xvfb-to-Run-Tests-That-Require-a-GUI
black screenshot for element inside scrollable div but outside visible area
E.g.
you have a div
this div contains some elements
those elements height is higher than parent div height so your div is scrollable
if you try to assert element inside scrollable area but out of the visible area, you got a black screenshot.
this is the example code
def test_scrollable_not_scrolled(self):
self.driver.get('https://rawgit.com/abioneperhobby/needle/master/demo/scroll.html')
self.assertScreenshot('#underscroll', 'not_scrolled')
def test_scrollable_scrolled(self):
self.driver.get('https://rawgit.com/abioneperhobby/needle/master/demo/scroll.html')
self.driver.execute_script('document.getElementById("scrollable").scrollTop = document.getElementById("scrollable").scrollHeight')
self.assertScreenshot('#underscroll', 'underscrolled')
in the example i open a page with a scrollable div. This scrollable div contains a green rectangle in the bottom. If you try to make assertion , you obtain a black square. In the second test, i scroll down the div before taking screenshot. Scrolling down the div, i obtain the true screenshot.
Installing with setuptools problem
Running needle-0.3/setup.py -q bdist_egg --dist-dir /var/folders/yd/zc63l4vx2bx0_wx6tkqxb7fm0000gn/T/easy_install-s6hft8jc/needle-0.3/egg-dist-tmp-eg_vn_sg
zip_safe flag not set; analyzing archive contents...
needle.pycache.cases.cpython-34: module references file
needle.pycache.driver.cpython-34: module references file
Traceback (most recent call last):
File "setup.py", line 25, in
packages=['dao', 'ui', 'ui.nosepython', 'ui.data', 'ui.nosepython', 'ui.nosepython.data_driven', 'ui.nosepython.dlp_ui', 'ui.nosepython.model', 'ui.nosepython.flux_capacitor', 'services', 'services.data', 'services.model'],
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/distutils/core.py", line 149, in setup
dist.run_commands()
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/Users/802619/Envs/test1/lib/python3.4/site-packages/setuptools/command/install.py", line 65, in run
self.do_egg_install()
File "/Users/802619/Envs/test1/lib/python3.4/site-packages/setuptools/command/install.py", line 115, in do_egg_install
cmd.run()
File "/Users/802619/Envs/test1/lib/python3.4/site-packages/setuptools/command/easy_install.py", line 360, in run
self.easy_install(spec, not self.no_deps)
File "/Users/802619/Envs/test1/lib/python3.4/site-packages/setuptools/command/easy_install.py", line 576, in easy_install
return self.install_item(None, spec, tmpdir, deps, True)
File "/Users/802619/Envs/test1/lib/python3.4/site-packages/setuptools/command/easy_install.py", line 627, in install_item
self.process_distribution(spec, dist, deps)
File "/Users/802619/Envs/test1/lib/python3.4/site-packages/setuptools/command/easy_install.py", line 682, in process_distribution
[requirement], self.local_index, self.easy_install
File "/Users/802619/Envs/test1/lib/python3.4/site-packages/pkg_resources.py", line 631, in resolve
dist = best[req.key] = env.best_match(req, ws, installer)
File "/Users/802619/Envs/test1/lib/python3.4/site-packages/pkg_resources.py", line 871, in best_match
return self.obtain(req, installer)
File "/Users/802619/Envs/test1/lib/python3.4/site-packages/pkg_resources.py", line 883, in obtain
return installer(requirement)
File "/Users/802619/Envs/test1/lib/python3.4/site-packages/setuptools/command/easy_install.py", line 595, in easy_install
return self.install_item(spec, dist.location, tmpdir, deps)
File "/Users/802619/Envs/test1/lib/python3.4/site-packages/setuptools/command/easy_install.py", line 625, in install_item
dists = self.install_eggs(spec, download, tmpdir)
File "/Users/802619/Envs/test1/lib/python3.4/site-packages/setuptools/command/easy_install.py", line 822, in install_eggs
return self.build_and_install(setup_script, setup_base)
File "/Users/802619/Envs/test1/lib/python3.4/site-packages/setuptools/command/easy_install.py", line 1028, in build_and_install
self.run_setup(setup_script, setup_base, args)
File "/Users/802619/Envs/test1/lib/python3.4/site-packages/setuptools/command/easy_install.py", line 1013, in run_setup
run_setup(setup_script, args)
File "/Users/802619/Envs/test1/lib/python3.4/site-packages/setuptools/sandbox.py", line 50, in run_setup
lambda: execfile(
File "/Users/802619/Envs/test1/lib/python3.4/site-packages/setuptools/sandbox.py", line 100, in run
return func()
File "/Users/802619/Envs/test1/lib/python3.4/site-packages/setuptools/sandbox.py", line 52, in
{'file':setup_script, 'name':'main'}
File "/Users/802619/Envs/test1/lib/python3.4/site-packages/setuptools/compat.py", line 78, in execfile
exec(compile(source, fn, 'exec'), globs, locs)
File "setup.py", line 61, in
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/distutils/core.py", line 149, in setup
dist.run_commands()
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/Users/802619/Envs/test1/lib/python3.4/site-packages/setuptools/command/bdist_egg.py", line 203, in run
os.path.join(archive_root,'EGG-INFO'), self.zip_safe()
File "/Users/802619/Envs/test1/lib/python3.4/site-packages/setuptools/command/bdist_egg.py", line 239, in zip_safe
return analyze_egg(self.bdist_dir, self.stubs)
File "/Users/802619/Envs/test1/lib/python3.4/site-packages/setuptools/command/bdist_egg.py", line 347, in analyze_egg
safe = scan_module(egg_dir, base, name, stubs) and safe
File "/Users/802619/Envs/test1/lib/python3.4/site-packages/setuptools/command/bdist_egg.py", line 381, in scan_module
code = marshal.load(f)
ValueError: bad marshal data (unknown type code)
Creating baseline images only if not already present
Hi,
I'm trying to work out a good workflow for using needle alongside a full site crawl.
So far my plan is this:
- Whenever the developer makes a change they delete the baseline images that the expect to change.
- They then generate new baselines for these
- They check the new baseline images by eye before running a full crawl test to see if there are any unexpected changes in other images.
The one missing feature I need to add to facilitate this is the ability to create a baseline only if it doesn't already exist. I can imagine this being useful for other reasons.
I'm happy to issue a pull request for this but wanted to discuss it first to see if the idea is generally acceptable - and if you're happy with my planned approach.
I was simply going to add another environment var and matching class property. Probably: NEEDLE_OVERWRITE_BASELINE and self.overwrite_baseline. These would default to True to match the existing behaviour.
How does that sound?
[firefox] - single test - multiple assertScreenshot - if one fails, the diff image show one of the not failed assertion
i have a complex single page web application.
i want to make a visual regression splitting the web page in different small screenshot (for example left menu, main menu, right menu, footer and navigation).
i make the baseline and screenshot are good.
i make a regression check and there is no difference.
now, if i modify something in the left menu, the test fails, but i obtain images of area that are not different on FIREFOX, but it works on CHROME
this is my code
def test_home_mainMenu(self):
self.driver.delete_all_cookies()
self.driver.get('https://'+configuration.clientId+'.thron.com/#/login/'+configuration.token)
time.sleep(15)
self.assertScreenshot('.main-menu', 'main-menu')
self.assertScreenshot('.apps-list-container', 'apps-list-container')
self.assertScreenshot('.menu-right-container', 'menu-right-container')
self.assertScreenshot('.notifications', 'notifications')
self.assertScreenshot('.secondary-menu-contents', 'sx-menu-library')
self.assertScreenshot('.secondary-menu-calendar', 'calendar-live')
self.assertScreenshot('.main-primary', 'contents-area')
and i obtain all the correct screenshot
but now i make the test again making a change on sx-menu-library.png , closing the folder before assertScreenshot starts (this explain why there is the time.sleep(15))
so from
to
test correcly fails,
but, as you can see, the screenshot created show a difference where there is no difference
and this is the log
C:\src\automatictestingtool\NeedleVisual>nosetests homepage.py
F
FAIL: test_home_mainMenu (homepage.THRONVisualRegression)
Traceback (most recent call last):
File "C:\src\automatictestingtool\NeedleVisual\homepage.py", line 15, in test_
home_mainMenu
self.assertScreenshot('.notifications', 'notifications')
File "c:\python27\lib\site-packages\needle\cases.py", line 131, in assertScree
nshot
pass
File "c:\python27\lib\contextlib.py", line 24, in exit
self.gen.next()
File "c:\python27\lib\site-packages\needle\cases.py", line 202, in compareScre
enshot
self.engine.assertSameFiles(output_file, baseline_file, threshold)
File "c:\python27\lib\site-packages\needle\engines\pil_engine.py", line 25, in
assertSameFiles
% (output_file, baseline_file, distance))
AssertionError: The new screenshot 'C:\src\automatictestingtool\NeedleVisual\scr
eenshots\notifications.png' did not match the baseline 'C:\src\automatictestingt
ool\NeedleVisual\screenshots\baseline\notifications.png' (by a distance of 198.6
1)
-------------------- >> begin captured logging << --------------------
requests.packages.urllib3.connectionpool: INFO: Starting new HTTP connection (1)
: thronqa-view.4me.it
requests.packages.urllib3.connectionpool: DEBUG: "POST /api/xsso/resources/ident
itymanager/login HTTP/1.1" 200 56
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:58835/
hub/session {"desiredCapabilities": {"platform": "ANY", "browserName": "firefox"
, "version": "", "marionette": false, "javascriptEnabled": true}}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:58835/
hub/session/7ca39783-4f52-4288-8afe-44a99c6b574c/window/current/position {"y": 0
, "x": 0, "windowHandle": "current", "sessionId": "7ca39783-4f52-4288-8afe-44a99
c6b574c"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:58835/
hub/session/7ca39783-4f52-4288-8afe-44a99c6b574c/window/current/size {"width": 1
024, "windowHandle": "current", "sessionId": "7ca39783-4f52-4288-8afe-44a99c6b57
4c", "height": 768}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:58835/
hub/session/7ca39783-4f52-4288-8afe-44a99c6b574c/execute {"sessionId": "7ca39783
-4f52-4288-8afe-44a99c6b574c", "args": [], "script": "return {width: document.bo
dy.clientWidth, height: document.body.clientHeight};"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:58835/
hub/session/7ca39783-4f52-4288-8afe-44a99c6b574c/window/current/size {"width": 1
037, "windowHandle": "current", "sessionId": "7ca39783-4f52-4288-8afe-44a99c6b57
4c", "height": 768}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: DELETE http://127.0.0.1:5883
5/hub/session/7ca39783-4f52-4288-8afe-44a99c6b574c/cookie {"sessionId": "7ca3978
3-4f52-4288-8afe-44a99c6b574c"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:58835/
hub/session/7ca39783-4f52-4288-8afe-44a99c6b574c/url {"url": "https://thronqa.th
ron.com/#/login/b8171bd2-69c3-4f26-9e71-c05aa0726c47", "sessionId": "7ca39783-4f
52-4288-8afe-44a99c6b574c"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:58835/
hub/session/7ca39783-4f52-4288-8afe-44a99c6b574c/element {"using": "css selector
", "sessionId": "7ca39783-4f52-4288-8afe-44a99c6b574c", "value": ".main-menu"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: GET http://127.0.0.1:58835/h
ub/session/7ca39783-4f52-4288-8afe-44a99c6b574c/element/{faf2a1af-be85-4b3d-96fb
-441d9621939f}/location {"sessionId": "7ca39783-4f52-4288-8afe-44a99c6b574c", "i
d": "{faf2a1af-be85-4b3d-96fb-441d9621939f}"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: GET http://127.0.0.1:58835/h
ub/session/7ca39783-4f52-4288-8afe-44a99c6b574c/element/{faf2a1af-be85-4b3d-96fb
-441d9621939f}/size {"sessionId": "7ca39783-4f52-4288-8afe-44a99c6b574c", "id":
"{faf2a1af-be85-4b3d-96fb-441d9621939f}"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: GET http://127.0.0.1:58835/h
ub/session/7ca39783-4f52-4288-8afe-44a99c6b574c/screenshot {"sessionId": "7ca397
83-4f52-4288-8afe-44a99c6b574c"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
PIL.PngImagePlugin: DEBUG: STREAM IHDR 16 13
PIL.PngImagePlugin: DEBUG: STREAM IDAT 41 8192
PIL.PngImagePlugin: DEBUG: STREAM IHDR 16 13
PIL.PngImagePlugin: DEBUG: STREAM IDAT 41 4875
PIL.PngImagePlugin: DEBUG: STREAM IHDR 16 13
PIL.PngImagePlugin: DEBUG: STREAM IDAT 41 4875
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:58835/
hub/session/7ca39783-4f52-4288-8afe-44a99c6b574c/element {"using": "css selector
", "sessionId": "7ca39783-4f52-4288-8afe-44a99c6b574c", "value": ".apps-list-con
tainer"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: GET http://127.0.0.1:58835/h
ub/session/7ca39783-4f52-4288-8afe-44a99c6b574c/element/{b0806653-ccfe-4755-bc37
-bf95d7b59c96}/location {"sessionId": "7ca39783-4f52-4288-8afe-44a99c6b574c", "i
d": "{b0806653-ccfe-4755-bc37-bf95d7b59c96}"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: GET http://127.0.0.1:58835/h
ub/session/7ca39783-4f52-4288-8afe-44a99c6b574c/element/{b0806653-ccfe-4755-bc37
-bf95d7b59c96}/size {"sessionId": "7ca39783-4f52-4288-8afe-44a99c6b574c", "id":
"{b0806653-ccfe-4755-bc37-bf95d7b59c96}"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: GET http://127.0.0.1:58835/h
ub/session/7ca39783-4f52-4288-8afe-44a99c6b574c/screenshot {"sessionId": "7ca397
83-4f52-4288-8afe-44a99c6b574c"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
PIL.PngImagePlugin: DEBUG: STREAM IHDR 16 13
PIL.PngImagePlugin: DEBUG: STREAM IDAT 41 8192
PIL.PngImagePlugin: DEBUG: STREAM IHDR 16 13
PIL.PngImagePlugin: DEBUG: STREAM IDAT 41 136
PIL.PngImagePlugin: DEBUG: STREAM IHDR 16 13
PIL.PngImagePlugin: DEBUG: STREAM IDAT 41 136
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:58835/
hub/session/7ca39783-4f52-4288-8afe-44a99c6b574c/element {"using": "css selector
", "sessionId": "7ca39783-4f52-4288-8afe-44a99c6b574c", "value": ".menu-right-co
ntainer"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: GET http://127.0.0.1:58835/h
ub/session/7ca39783-4f52-4288-8afe-44a99c6b574c/element/{c73b4fa6-fe5d-4dc7-a214
-5e45bb0eb5b8}/location {"sessionId": "7ca39783-4f52-4288-8afe-44a99c6b574c", "i
d": "{c73b4fa6-fe5d-4dc7-a214-5e45bb0eb5b8}"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: GET http://127.0.0.1:58835/h
ub/session/7ca39783-4f52-4288-8afe-44a99c6b574c/element/{c73b4fa6-fe5d-4dc7-a214
-5e45bb0eb5b8}/size {"sessionId": "7ca39783-4f52-4288-8afe-44a99c6b574c", "id":
"{c73b4fa6-fe5d-4dc7-a214-5e45bb0eb5b8}"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: GET http://127.0.0.1:58835/h
ub/session/7ca39783-4f52-4288-8afe-44a99c6b574c/screenshot {"sessionId": "7ca397
83-4f52-4288-8afe-44a99c6b574c"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
PIL.PngImagePlugin: DEBUG: STREAM IHDR 16 13
PIL.PngImagePlugin: DEBUG: STREAM IDAT 41 8192
PIL.PngImagePlugin: DEBUG: STREAM IHDR 16 13
PIL.PngImagePlugin: DEBUG: STREAM IDAT 41 2523
PIL.PngImagePlugin: DEBUG: STREAM IHDR 16 13
PIL.PngImagePlugin: DEBUG: STREAM IDAT 41 2523
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:58835/
hub/session/7ca39783-4f52-4288-8afe-44a99c6b574c/element {"using": "css selector
", "sessionId": "7ca39783-4f52-4288-8afe-44a99c6b574c", "value": ".notifications
"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: GET http://127.0.0.1:58835/h
ub/session/7ca39783-4f52-4288-8afe-44a99c6b574c/element/{5d8563ea-3067-4dc7-8caf
-080f55194a5e}/location {"sessionId": "7ca39783-4f52-4288-8afe-44a99c6b574c", "i
d": "{5d8563ea-3067-4dc7-8caf-080f55194a5e}"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: GET http://127.0.0.1:58835/h
ub/session/7ca39783-4f52-4288-8afe-44a99c6b574c/element/{5d8563ea-3067-4dc7-8caf
-080f55194a5e}/size {"sessionId": "7ca39783-4f52-4288-8afe-44a99c6b574c", "id":
"{5d8563ea-3067-4dc7-8caf-080f55194a5e}"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: GET http://127.0.0.1:58835/h
ub/session/7ca39783-4f52-4288-8afe-44a99c6b574c/screenshot {"sessionId": "7ca397
83-4f52-4288-8afe-44a99c6b574c"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
PIL.PngImagePlugin: DEBUG: STREAM IHDR 16 13
PIL.PngImagePlugin: DEBUG: STREAM IDAT 41 8192
PIL.PngImagePlugin: DEBUG: STREAM IHDR 16 13
PIL.PngImagePlugin: DEBUG: STREAM IDAT 41 8948
PIL.PngImagePlugin: DEBUG: STREAM IHDR 16 13
PIL.PngImagePlugin: DEBUG: STREAM IDAT 41 7340
--------------------- >> end captured logging << ---------------------
Ran 1 test in 19.994s
FAILED (failures=1)
C:\src\automatictestingtool\NeedleVisual>
why?
thanks
get_distance is too slow
Hello. I like the idea and laziness of needle a lot, but when I tried it for a real project the assertScreenshot() method is just too slow. For images of about 500kb, each call to get_distance() takes just over 30 seconds.
Are there any hints to speed up needle?
Compare 2 URLs
Hey,
Is there a way to compare something.com against staging.something.com?
Thank you!
assertSameFiles of perceptualdiff engine fails when file paths contain spaces
First of all, thanks for the nice tool!
We are using it in a Jenkins CI setup, and the screenshots path happens to have spaces.
This is causing the perceptualdiff command to parse the --output parameter incorrectly, splitting it in correspondence of the space.
We've now worked around the issue by patching this line:
https://github.com/bfirsh/needle/blob/master/needle/engines/perceptualdiff_engine.py#L20
specifically replacing
%s %s %s
with
'%s' '%s' '%s'
AttributeError: 'WebElement' object has no attribute 'get_screenshot'
Hi,
I started looking into needle (instead of ApliTools) and trying to create a test but facing an issue with this command:
element.get_screenshot().save(baseline_file)
In compareScreenshot() method cases.py file
File "C:\Python27\lib\site-packages\needle\cases.py", line 238, in compareScreenshot
element.get_screenshot().save(output_file)
AttributeError: 'WebElement' object has no attribute 'get_screenshot'
My guess that the driver that am referring to is the selenium web driver, but the one used in cases.py is the needle driver!
My code:
from needle.cases import NeedleTestCase
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from time import sleep
class NebulaTest(NeedleTestCase):
def init(self):
self.output_directory = "C:\screenshots"
self.baseline_directory = "C:\screenshots\baseline"
#self.engine_class = 'needle.engines.perceptualdiff_engine.Engine'
#engine_class = 'needle.engines.imagemagick_engine.Engine'
self.driver = webdriver.Chrome()
self.test_feedback()
def test_feedback(self):
#self.set_viewport_size(width=1024, height=768)
self.driver.get('https://kampyle25.tumblr.com/')
sleep(5)
#try:
# WebDriverWait(self.driver, 10).until(EC.presence_of_element_located((By.CSS_SELECTOR, "div.kampyle_button-text")))
#finally:
# self.driver.quit()
self.assertScreenshot('div.kampyle_button-text', 'feedback')
NebulaTest()
Thanks in advance.
nose.proxy.TypeError: integer argument expected, got float
I got an error when I use IE for my explorer.
The IEDriverServer.exe (32bit) is putting at C:\Python34
Python version : 3.4.0
IE: IE9
OS:Win 7
please comment if anything needed.
Testing code:
from needle.cases import NeedleTestCase
from needle.driver import NeedleIe
class BTest(NeedleTestCase):
@classmethod
def get_web_driver(cls):
return NeedleIe()
def test_masthead(self):
self.driver.get('http://www.baidu.com/')
self.assertScreenshot('#ftCon', 'test-baidu')
Error information:
d:\Cain\dev\needle>nosetests test.py --with-save-baseline
ERROR: test_masthead (test.BBCNewsTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "d:\Cain\dev\needle\test.py", line 10, in test_masthead
self.assertScreenshot('#ftCon', 'test-baidu')
File "C:\Python34\lib\site-packages\needle\cases.py", line 127, in assertScreenshot
pass
File "C:\Python34\lib\contextlib.py", line 66, in __exit__
next(self.gen)
File "C:\Python34\lib\site-packages\needle\cases.py", line 186, in compareScreenshot
element.get_screenshot().save(baseline_file)
File "C:\Python34\lib\site-packages\PIL\Image.py", line 1532, in save
self.load()
File "C:\Python34\lib\site-packages\PIL\Image.py", line 1830, in load
self.im = self.im.crop(self.__crop)
nose.proxy.TypeError: integer argument expected, got float
-------------------- >> begin captured logging << --------------------
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:49256/session {"desiredCapabilities": {"browserName": "internet explore
r", "version": "", "javascriptEnabled": true, "platform": "WINDOWS"}, "sessionId": null}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:49256/session/f519906e-9ffd-418f-88ee-3a8499bba0bb/window/current/posit
ion {"y": 0, "sessionId": "f519906e-9ffd-418f-88ee-3a8499bba0bb", "windowHandle": "current", "x": 0}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:49256/session/f519906e-9ffd-418f-88ee-3a8499bba0bb/window/current/size
{"width": 1024, "sessionId": "f519906e-9ffd-418f-88ee-3a8499bba0bb", "windowHandle": "current", "height": 768}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:49256/session/f519906e-9ffd-418f-88ee-3a8499bba0bb/execute {"script": "
return {width: document.body.clientWidth, height: document.body.clientHeight};", "sessionId": "f519906e-9ffd-418f-88ee-3a8499bba0bb", "args": []}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:49256/session/f519906e-9ffd-418f-88ee-3a8499bba0bb/window/current/size
{"width": 1061, "sessionId": "f519906e-9ffd-418f-88ee-3a8499bba0bb", "windowHandle": "current", "height": 768}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:49256/session/f519906e-9ffd-418f-88ee-3a8499bba0bb/url {"url": "http://
www.baidu.com/", "sessionId": "f519906e-9ffd-418f-88ee-3a8499bba0bb"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:49256/session/f519906e-9ffd-418f-88ee-3a8499bba0bb/element {"using": "c
ss selector", "value": "#ftCon", "sessionId": "f519906e-9ffd-418f-88ee-3a8499bba0bb"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:49256/session/f519906e-9ffd-418f-88ee-3a8499bba0bb/execute {"script": "
return typeof(jQuery)", "sessionId": "f519906e-9ffd-418f-88ee-3a8499bba0bb", "args": []}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:49256/session/f519906e-9ffd-418f-88ee-3a8499bba0bb/execute {"script": "
\n var e = $(arguments[0]);\n var offset = e.offset();\n var dimensions = {\n 'width': e.outerWid
th(),\n 'height': e.outerHeight(),\n 'left': Math.floor(offset.left),\n 'top': Math.floor(offset.top
)\n };\n return dimensions;\n ", "sessionId": "f519906e-9ffd-418f-88ee-3a8499bba0bb", "args": [{"ELEMENT": "80fa0498
-8c23-4ef3-a62e-94c10bbda350"}]}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: GET http://127.0.0.1:49256/session/f519906e-9ffd-418f-88ee-3a8499bba0bb/screenshot {"sessionI
d": "f519906e-9ffd-418f-88ee-3a8499bba0bb"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 1 test in 6.982s
FAILED (errors=1)
Empty diff file when using PerceptualDiff
This is actually a duplicate of #61 but it got closed by the reporter.
I'm using the latest perceptualdiff, needle 0.5.0, python 3.7 (and pytest-needle 0.3.11).
When running a simple test and the new image does not match the baseline:
Traceback (most recent call last):
File ".../test.py", line 174, in test
needle.assert_screenshot('something')
File ".../lib/python3.7/site-packages/pytest_needle/driver.py", line 322, in assert_screenshot
self.engine.assertSameFiles(fresh_image_file, baseline_image, threshold)
File ".../lib/python3.7/site-packages/needle/engines/perceptualdiff_engine.py", line 38, in assertSameFiles
Image.open(diff_ppm).save(diff_png)
File ".../lib/python3.7/site-packages/PIL/Image.py", line 2818, in open
raise IOError("cannot identify image file %r" % (filename if filename else fp))
OSError: cannot identify image file '.../screenshots/something.diff.ppm'
something.diff.ppm turns out to be an empty file. When running perceptualdiff manually:
$ perceptualdiff baseline/something.png something.png --output some.ppm
FAIL: Images are visibly different
86451 pixels are different
Failed to save to 'some.ppm'
It shows an error, but as it turns out, this does work:
$ perceptualdiff baseline/something.png something.png --output some.png
FAIL: Images are visibly different
86451 pixels are different
Wrote difference image to some.png
perceptualdiff uses FreeImage since 1.0.2 (and it still does):
1.0.2 - [jt] Converted the loading and saving routines to use FreeImage
Even though FreeImage supports PPM, for some reason it doesn't work on my machine. Oddly enough gimp can write PPM and my image viewer can view it too. Anyway, all this can be avoided by requesting a png from perceptualdiff instead of a ppm and then converting it to png; and update the docs to require at least perceptualdiff >= 1.0.2.
Can i commit my new feature: whitelist?
hi,
sometime i needed to remove something inside my screenshot div because sometime some changes can be considered normal.
e.g. i have a dashboard with some graphs, i need to make a visual comparison of the dashboard but changes in graph can be normal.
so i updated the library and now assertScreenshot can take a whitelist of css selector as parameters.
can i commit my update?
thanks
Diff fails due to a few missing rows of pixels
I ran my tests on:
- Ubuntu 13.04 with a 1024 x 768 browser size
- Through pyvirtualdisplay (using Xvfb on Ubuntu 13.04) with a 1024 x 768 browser size
The height of the captured images were 3 pixels shorter during one test than during the other so the tests failed.
Fixing the cause would be ideal, but this seems like it might vary from machine-to-machine (these were both on the same machine) so mitigating the issue with an optional size difference allowance could stop the tests from failing without cause.
Visual test report
Is there any test report (or tool) which displays the differences if a test failed?
Something like: Baseline screenshot on the left - Failed screenshot on the right?
Release a new PyPI version
It would be nice if the NeedleWebDriverMixin was available when installing from PyPI.
how to handle screenshot assertion of div out of the current viewport?
Hi,
when i use needle to capture screenshot under the current viewport, screenshot becomes black.
for example, if my browser has 1000px of height but the page is scrollable because its body has a height of 2000px, and my target id is in the page, but out of the area the browser can see, the screenshot becomes black.
i attached two screenshot. the test is the same, but i put a time.sleep(10) that permits to me to scroll down in the page obtaining a screenshot
how should i handle this situation?
should i implement a "if not visible scroll browser to makes the div visible" ?
Unable to run Needle with pytest
Currently using needle 0.2.2
Got this error when trying to run needle with pytest
usage: py.test [options] [file_or_dir] [file_or_dir] [...]
py.test: error: unrecognized arguments: --with-save-baseline
AttributeError: 'NeedleWebDriver' object has no attribute 'set_window_position'
Hi,
I tried to use needle following the instructions on: http://needle.readthedocs.org/en/latest/
This is what I get:
╰─➤ nosetests test_bbc.py --with-needle-capture 1 ↵
E
======================================================================
ERROR: test suite for <class 'test_bbc.BBCNewsTest'>
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/nose/suite.py", line 208, in run
self.setUp()
File "/usr/local/lib/python2.7/dist-packages/nose/suite.py", line 291, in setUp
self.setupContext(ancestor)
File "/usr/local/lib/python2.7/dist-packages/nose/suite.py", line 314, in setupContext
try_run(context, names)
File "/usr/local/lib/python2.7/dist-packages/nose/util.py", line 469, in try_run
return func()
File "/usr/local/lib/python2.7/dist-packages/needle-0.0.1-py2.7.egg/needle/cases.py", line 52, in setUpClass
cls.driver.set_window_position(0, 0)
AttributeError: 'NeedleWebDriver' object has no attribute 'set_window_position'
-------------------- >> begin captured logging << --------------------
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:4444/wd/hub/session {"sessionId": null, "desiredCapabilities": {"browserName": "firefox"}}
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 0 tests in 2.925s
FAILED (errors=1)
Am I doing something wrong?
By the way, the docs say: "Download selenium-server-standalone-2.8.0.jar. By default, Selenium requires Firefox 4." so I tried with those versions (and recent versions too). Is this a hard requirement or are newer versions supported too?
FF35 'offset is undefined'
Hi,
I tried to investigate but without any luck. It seems strange that the error appears only if I'm using FF35, however on FF34 everything works well.
I think the problem might be here: https://github.com/bfirsh/needle/blob/master/needle/driver.py#L47
Can this be a selenium issue?
Any help is appreciated. Thanks.
File "/usr/lib/python2.7/unittest/case.py", line 331, in run
testMethod()
File "/var/lib/jenkins/jobs/pull-request-functional-tests-builder/workspace/tests/ft/b2c/tests_headers.py", line 19, in test_navbar_wrapper
threshold=0
File "/var/lib/jenkins/jobs/pull-request-functional-tests-builder/workspace/tests/ft/base_selenium.py", line 73, in assert_responsive
kwargs.get("threshold", 0)
File "/var/lib/jenkins/shiningpanda/jobs/3312707c/virtualenvs/d41d8cd9/local/lib/python2.7/site-packages/needle/cases.py", line 128, in assertScreenshot
pass
File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
self.gen.next()
File "/var/lib/jenkins/shiningpanda/jobs/3312707c/virtualenvs/d41d8cd9/local/lib/python2.7/site-packages/needle/cases.py", line 196, in compareScreenshot
element.get_screenshot().save(output_file)
File "/var/lib/jenkins/shiningpanda/jobs/3312707c/virtualenvs/d41d8cd9/local/lib/python2.7/site-packages/needle/driver.py", line 61, in get_screenshot
d = self.get_dimensions()
File "/var/lib/jenkins/shiningpanda/jobs/3312707c/virtualenvs/d41d8cd9/local/lib/python2.7/site-packages/needle/driver.py", line 55, in get_dimensions
""", self)
File "/var/lib/jenkins/shiningpanda/jobs/3312707c/virtualenvs/d41d8cd9/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 401, in execute_script
{'script': script, 'args':converted_args})['value']
File "/var/lib/jenkins/shiningpanda/jobs/3312707c/virtualenvs/d41d8cd9/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 173, in execute
self.error_handler.check_response(response)
File "/var/lib/jenkins/shiningpanda/jobs/3312707c/virtualenvs/d41d8cd9/local/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 166, in check_response
raise exception_class(message, screen, stacktrace)
'Message: u\'offset is undefined\' ;
There is no heat map/color difference pytest-needly using PIL
Team,
I’m using pytest-needle 0.3.11 using PIL engine to evaluate visual testing. Everything working as expected but in report unable to find the colour difference in the diff screenshot. In report, its showing actual screenshot, expected screenshot and difference. But unable to identify what is the difference. Is there any solution to get heat map (color) difference between two screenshot.
Thanks in advance and looking forward your support.
Regards,
Rafeek
WebDriverException: Message: u'unknown error: Maximum call stack size exceeded
With Chrome 36.0.1985.125 and driver version 2.9
ERROR:
W:\MOP\Relaunch\__inc\tests>nosetests test_looksC_chrome.py --with-save-baseline
EEEE
======================================================================
ERROR: test_view1_carousel (test_looksC_chrome.looksCTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "W:\MOP\Relaunch\__inc\tests\test_looksC_chrome.py", line 20, in test_view1_carousel
self.driver.execute_script("window.scrollTo(0,$('.looks-home-carousel').offset().top);")
File "c:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 401, in execute_script
{'script': script, 'args':converted_args})['value']
File "c:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 173, in execute
self.error_handler.check_response(response)
File "c:\Python27\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 164, in check_response
raise exception_class(message, screen, stacktrace)
WebDriverException: Message: u'unknown error: Maximum call stack size exceeded\n (Session info: chrome=36.0.1985.125)\n (Driver info: chromedriver=2.9.248315,platform=Windows NT 6.1 SP1 x86_64)'
-------------------- >> begin captured logging << --------------------
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:63037/session {"desiredCapabilities": {"platform": "ANY", "browserName": "chrome", "version": "", "javascriptEnabled": true, "chromeOptions": {"args": [], "extensions": []}}}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:63037/session/5c82fff520de155800b4d8a0fe6b5513/window/current/position {"y": 0, "x": 0, "windowHandle": "current", "sessionId": "5c82fff520de155800b4d8a0fe6b5513"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:63037/session/5c82fff520de155800b4d8a0fe6b5513/window/current/size {"width": 1024, "windowHandle": "current", "sessionId": "5c82fff520de155800b4d8a0fe6b5513", "height": 768}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:63037/session/5c82fff520de155800b4d8a0fe6b5513/execute {"sessionId": "5c82fff520de155800b4d8a0fe6b5513", "args": [], "script": "return {width: document.body.clientWidth, height: doc
ument.body.clientHeight};"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:63037/session/5c82fff520de155800b4d8a0fe6b5513/window/current/size {"width": 1040, "windowHandle": "current", "sessionId": "5c82fff520de155800b4d8a0fe6b5513", "height": 768}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:63037/session/5c82fff520de155800b4d8a0fe6b5513/window/current/size {"width": 1024, "windowHandle": "current", "sessionId": "5c82fff520de155800b4d8a0fe6b5513", "height": 768}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:63037/session/5c82fff520de155800b4d8a0fe6b5513/execute {"sessionId": "5c82fff520de155800b4d8a0fe6b5513", "args": [], "script": "return {width: document.body.clientWidth, height: doc
ument.body.clientHeight};"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:63037/session/5c82fff520de155800b4d8a0fe6b5513/window/current/size {"width": 1040, "windowHandle": "current", "sessionId": "5c82fff520de155800b4d8a0fe6b5513", "height": 768}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:63037/session/5c82fff520de155800b4d8a0fe6b5513/url {"url": "http://mop.relaunch.keh/home.htm", "sessionId": "5c82fff520de155800b4d8a0fe6b5513"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:63037/session/5c82fff520de155800b4d8a0fe6b5513/execute {"sessionId": "5c82fff520de155800b4d8a0fe6b5513", "args": [], "script": "window.scrollTo(0,$('.looks-home-carousel').offset().
top);"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
--------------------- >> end captured logging << ---------------------
======================================================================
ERROR: test_view2_carousel (test_looksC_chrome.looksCTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "W:\MOP\Relaunch\__inc\tests\test_looksC_chrome.py", line 26, in test_view2_carousel
self.set_viewport_size(width=768, height=640)
File "c:\Python27\lib\site-packages\needle\cases.py", line 114, in set_viewport_size
measured = cls.driver.execute_script("return {width: document.body.clientWidth, height: document.body.clientHeight};")
File "c:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 401, in execute_script
{'script': script, 'args':converted_args})['value']
File "c:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 173, in execute
self.error_handler.check_response(response)
File "c:\Python27\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 164, in check_response
raise exception_class(message, screen, stacktrace)
WebDriverException: Message: u'unknown error: Maximum call stack size exceeded\n (Session info: chrome=36.0.1985.125)\n (Driver info: chromedriver=2.9.248315,platform=Windows NT 6.1 SP1 x86_64)'
-------------------- >> begin captured logging << --------------------
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:63037/session/5c82fff520de155800b4d8a0fe6b5513/window/current/size {"width": 768, "windowHandle": "current", "sessionId": "5c82fff520de155800b4d8a0fe6b5513", "height": 640}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:63037/session/5c82fff520de155800b4d8a0fe6b5513/execute {"sessionId": "5c82fff520de155800b4d8a0fe6b5513", "args": [], "script": "return {width: document.body.clientWidth, height: doc
ument.body.clientHeight};"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
--------------------- >> end captured logging << ---------------------
======================================================================
ERROR: test_view3_carousel (test_looksC_chrome.looksCTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "W:\MOP\Relaunch\__inc\tests\test_looksC_chrome.py", line 36, in test_view3_carousel
self.set_viewport_size(width=480, height=320)
File "c:\Python27\lib\site-packages\needle\cases.py", line 114, in set_viewport_size
measured = cls.driver.execute_script("return {width: document.body.clientWidth, height: document.body.clientHeight};")
File "c:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 401, in execute_script
{'script': script, 'args':converted_args})['value']
File "c:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 173, in execute
self.error_handler.check_response(response)
File "c:\Python27\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 164, in check_response
raise exception_class(message, screen, stacktrace)
WebDriverException: Message: u'unknown error: Maximum call stack size exceeded\n (Session info: chrome=36.0.1985.125)\n (Driver info: chromedriver=2.9.248315,platform=Windows NT 6.1 SP1 x86_64)'
-------------------- >> begin captured logging << --------------------
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:63037/session/5c82fff520de155800b4d8a0fe6b5513/window/current/size {"width": 480, "windowHandle": "current", "sessionId": "5c82fff520de155800b4d8a0fe6b5513", "height": 320}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:63037/session/5c82fff520de155800b4d8a0fe6b5513/execute {"sessionId": "5c82fff520de155800b4d8a0fe6b5513", "args": [], "script": "return {width: document.body.clientWidth, height: doc
ument.body.clientHeight};"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
--------------------- >> end captured logging << ---------------------
======================================================================
ERROR: test_view4_carousel (test_looksC_chrome.looksCTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "W:\MOP\Relaunch\__inc\tests\test_looksC_chrome.py", line 46, in test_view4_carousel
self.set_viewport_size(width=320, height=240)
File "c:\Python27\lib\site-packages\needle\cases.py", line 114, in set_viewport_size
measured = cls.driver.execute_script("return {width: document.body.clientWidth, height: document.body.clientHeight};")
File "c:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 401, in execute_script
{'script': script, 'args':converted_args})['value']
File "c:\Python27\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 173, in execute
self.error_handler.check_response(response)
File "c:\Python27\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 164, in check_response
raise exception_class(message, screen, stacktrace)
WebDriverException: Message: u'unknown error: Maximum call stack size exceeded\n (Session info: chrome=36.0.1985.125)\n (Driver info: chromedriver=2.9.248315,platform=Windows NT 6.1 SP1 x86_64)'
-------------------- >> begin captured logging << --------------------
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:63037/session/5c82fff520de155800b4d8a0fe6b5513/window/current/size {"width": 320, "windowHandle": "current", "sessionId": "5c82fff520de155800b4d8a0fe6b5513", "height": 240}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
selenium.webdriver.remote.remote_connection: DEBUG: POST http://127.0.0.1:63037/session/5c82fff520de155800b4d8a0fe6b5513/execute {"sessionId": "5c82fff520de155800b4d8a0fe6b5513", "args": [], "script": "return {width: document.body.clientWidth, height: doc
ument.body.clientHeight};"}
selenium.webdriver.remote.remote_connection: DEBUG: Finished Request
--------------------- >> end captured logging << ---------------------
----------------------------------------------------------------------
Ran 4 tests in 5.514s
FAILED (errors=4)
Test:
# encoding: utf-8
from needle.cases import NeedleTestCase
import needle.driver
import time
class looksCTest(NeedleTestCase):
engine_class = 'needle.engines.perceptualdiff_engine.Engine'
@classmethod
def get_web_driver(cls):
return needle.driver.NeedleChrome()
def test_view1_carousel(self):
self.set_viewport_size(width=1024, height=768)
self.driver.get('http://mop.relaunch.keh/home.htm')
time.sleep(0.5)
self.driver.execute_script("window.scrollTo(0,$('.looks-home-carousel').offset().top);")
time.sleep(0.5)
self.driver.execute_script("window.scrollTo(0,0)")
self.assertScreenshot(".looks-home-carousel", "looksC.view1.carousel")
def test_view2_carousel(self):
self.set_viewport_size(width=768, height=640)
self.driver.get('http://mop.relaunch.keh/home.htm')
time.sleep(0.5)
self.driver.execute_script("window.scrollTo(0,$('.looks-home-carousel').offset().top);")
time.sleep(0.5)
self.driver.execute_script("window.scrollTo(0,0)")
self.assertScreenshot(".looks-home-carousel", "looksC.view2.carousel")
def test_view3_carousel(self):
self.set_viewport_size(width=480, height=320)
self.driver.get('http://mop.relaunch.keh/home.htm')
time.sleep(0.5)
self.driver.execute_script("window.scrollTo(0,$('.looks-home-carousel').offset().top);")
time.sleep(0.5)
self.driver.execute_script("window.scrollTo(0,0)")
self.assertScreenshot(".looks-home-carousel", "looksC.view3.carousel")
def test_view4_carousel(self):
self.set_viewport_size(width=320, height=240)
self.driver.get('http://mop.relaunch.keh/home.htm')
time.sleep(0.5)
self.driver.execute_script("window.scrollTo(0,$('.looks-home-carousel').offset().top);")
time.sleep(0.5)
self.driver.execute_script("window.scrollTo(0,0)")
self.assertScreenshot(".looks-home-carousel", "looksC.view4.carousel")
Test fails with cryptic error message if fresh image is larger then baseline image (imagemagick)
Expected Output
assertSameFiles Needle should either state that the images are similar or that they are not.
Actual Output
ValueError: ("could not convert string to float: b'idt'") thrown at /needle/engines/imagemagick_engine.py:24
- Checking the
compare_stderrin that situation yieldscompare: width or height exceeds limit [...] - Running the actual command stored
incompare_cmdin a console yields the same error - However I don't think this this an imagemagick bug since the intention
-subimage-searchis to find a smaller image inside a bigger one.
Reproduce
- create an images 1.png and 2.png where 2.png is bigger then 1.png
- run
magick compare -metric RMSE -dissimilarity-threshold 1.0 -subimage-search 1.png 2.png d.png - get
compare: width or height exceeds limit '1.png' @ error/cache.c/OpenPixelCache/3491.
Suggestions
We could order the images by size. However I guess that will not fix the problem is one image is wider but not as high as the other one.
Contribution
I tried to run the tests but many fail on my system. I'm not sure if that is a problem with the tests or with me not using the right commands. I created an issue #74
If this issue is solved I'd be happy to implement my the reordering of the images.
is this project death? should i use nose even if nose is death? is this project production usable?
hi,
thanks for this very interesting project.
- there is no commit since the last year and needle are in version 0.3 . is this project death?
- the project needs nose, but nose project is death (http://nose.readthedocs.io/en/latest/) so nose propose to use node2. What should i with needle?
- needle with nose permits to obtain good result or should i move away? any suggestion?
thanks.
Selenium Installation
Hi there,
Can you point us in the right direction how to install Selenium to work with needle?
Support for multi selectors
I noticed that when using a selector that actually returns more than one element, the screenshots are taken only for the first(?) found element?
Wouldn't it be better to have a for each to get and compare screenshots for each element in the selector?
I'd do the coding but does anyone find this useful?
Error thrown out by PIL when using PerceptualDiff (Python 3.4)
Problem which I encountered is very similar to issue #38 but a little bit different. When I'm running my script with default engine everything is fine but when I want to use PerceptualDiff following error is thrown:
Traceback (most recent call last):
File "/home/jrozycki/selenium_and_python/needle_test.py", line 15, in test_masthead
self.assertScreenshot('#nav-collapse', 'test_file')
File "/opt/python3.4/lib/python3.4/site-packages/needle/cases.py", line 148, in assertScreenshot
pass
File "/opt/python3.4/lib/python3.4/contextlib.py", line 66, in exit
next(self.gen)
File "/opt/python3.4/lib/python3.4/site-packages/needle/cases.py", line 219, in compareScreenshot
self.engine.assertSameFiles(output_file, baseline_file, threshold)
File "/opt/python3.4/lib/python3.4/site-packages/needle/engines/perceptualdiff_engine.py", line 38, in assertSameFiles
Image.open(diff_ppm).save(diff_png)
File "/opt/python3.4/lib/python3.4/site-packages/PIL/Image.py", line 2349, in open
% (filename if filename else fp))
nose.proxy.OSError: cannot identify image file '/home/jrozycki/selenium_and_python/screenshots/test_file.diff.ppm'
I'm using the newest version of needle (0.4.1) and PerceptualDiff works fine on it's own.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.