Plugin for generating HTML reports for pytest results
License: Other
pytest-html is a plugin for pytest that generates a HTML report for test results.
We welcome contributions.
To learn more, see Development
![pre-commit-ci[bot] avatar](https://avatars.githubusercontent.com/in/68672?v=4 "pre-commit-ci[bot]")
Report is now plain, which means that you cannot "overview" whole list of checks and assume which one is whole failed, and which one contains errors in steps.
Its very important for step-dependent runtime, where stepA is the function which performs authorization, purpose of stepB is to click several times in order to go to the subservice on the website, stepC contains steps for sanity checks, etc.
My tests contains around 20-30 steps, so the report looks unreadable. Is it possible to shorten levels by package/module/class/function and then collapse by click the button like "+"?
When Jenkins addressed their stored XSS vulnerability through workspace files and archived artifacts, the HTML reports served became very limited. While it's possible to relax the security in Jenkins, it's not advisable. I wonder if we could degrade the HTML report gracefully for this case.
These are the default rules now imposed by Jenkins:
Without JavaScript, we lose support for splitting the log and additional HTML onto a separate row, the default sort order of test results, and the ability for users to change the sort order of the report. We should see if it's possible to take care of the first two in the generated HTML instead of relying on JavaScript.
All CSS is currently inline, so it may make sense for us to generate a CSS file alongside the HTML report. Currently the user specifies the path to the HTML file, which means we either have to make some assumptions about the available name for the CSS file. Perhaps instead of a file, we could support a directory, to which index.html and style.css can be written.
We also lose the handling of images that prevents large data URLs to be duplicated in the HTML. I believe that using data URLs for images should still work, so we should either find an optimal way to use them for both links and displayed images, or consider saving images as separate resources.
Useful when you have thousands of tests and the browser struggles to open them all.
I'm using pytest-html with pytest-selenium. My selenium tests are doing an image diff.
I want to include the results of a failed image diff test in my pytest-html report.
However, if I just use pytest_html.extras.append(...) then my output gets mixed up with pytest-selenium's output and makes for a very incoherent report e.g.
Full report at: https://s3.amazonaws.com/bokeh-travis/112986254/tests/pytest-report.html
I have hacked around this at the moment by overriding longrepr and extras completely so that I get something like this:
Full report at: https://s3.amazonaws.com/bokeh-travis/112997379/tests/pytest-report.html
What I'd like is a hook to be able to add another row of my own custom test output.
The environment section in the HTML report is missing when we run tests in parallel using pytest-xdist. I'm hoping that @RonnyPfannschmidt can take a look and suggest how we can fix it. 😄
Hi!
Your plugin is great! But since version 1.10.0 I get all the screenshot images broken:
I am trying to add some content to the Environment section of the report
On adding the following code to my conftest.py file,
@pytest.fixture(autouse=True)
def _environment(request):
request.config._html.environment.append(('foo', 'bar'))I get the error
> request.config._html.environment.append(('foo', 'bar'))
E AttributeError: 'Config' object has no attribute '_html'
Has the path to _html changed?
Regards
Prem
Hi,
I would like to request to add "html" to addopts in setup.cfg.
Thank you
Feature enhancement: enhance extras.image to accept a path/URL as an alternative to a base64 encoded PNG.
Implementation: check the content type of the extra content. If it is not base64 then just assume it to be a path/URL. The resulting href/src for the should be the provided string value as-is (optionally: could check if path starts with file: or http(s))
Background: In my setup, I do not have base64 encoded extra content when the report gets generated. Instead, my test logic already creates an external folder where screenshots are stored during test execution. So essentially I'd like to just link to them using a given file:// or relative URL inside the report, no matter if the report is self-contained or not.
e.g. this
extra.append(pytest_html.extras.image('../screenshots/foo_bar.png'))
extra.append(pytest_html.extras.image('file:///screenshots/foo_bar.png'))
extra.append(pytest_html.extras.image('http://foo.bar/screenshots/foo_bar.png'))
would result in 3 tags in the report with their src/href set to the above values.
I know, this is a horrendous configuration; it's what we're working with.
On line 73 of pytest_html/plugin.py the key environment is not present innode.slaveoutput. As the title says, this condition is reached when executing pytest with xdist over multiple ssh hosts, that themselves are running ssh daemons with cygwin. I've been able to get around this using:
73 if 'environment' not in node.slaveoutput:
74 node.slaveoutput['environment'] = None
But I am this certain causes issues in other circumstances, e.g. when trying to access the environment attribute when generating reports. I could instead simply add 'Not Found'... Would you accept a PR with this? I can also give the xdist guys a heads up.
When JavaScript is disabled (such as via CSP) we should not display the filter checkboxes. The functionality is dependent on JavaScript, and without it they are just a distraction.
There could be unit tests to the JS code, there are none until now! I personally like Mocha to do that: https://mochajs.org/
Currently I use ANSI colors for logger levels (info, debug etcc.) and it comes up nicely in console. However this is not retained in final html reports while using pytest-html.
The results output shows something like this.
----------------------------- Captured stderr call -----------------------------
[32m[INFO][0m INFO messages[0m
[36m[DEBUG][0m DEBUG messages[0m
[31m[ERROR][0m ERROR messages[0m
Is there a way to retain the color codes shown up in console.
pytest-html is really a fantastic plugin!!
We should not include the 'reruns' in the report summary when the pytest-rerunfailures plugin is not registered.
It would be useful for users with large reports to be able to filter them by outcome. This will likely require JavaScript, but should degrade gracefully when this is not available. A suitable message should be displayed if no results match the filter applied. I imagine a list of outcomes with a checkbox, all checked by default, and unchecking a box would hide the associated results.
This is one enhancement related to issue #29
On Python 3, when I have a test file like:
import pytest
@pytest.mark.parametrize('val', ['\ud800'])
def test_foo(val):
passI get:
...
File "/home/florian/proj/qutebrowser/git/.tox/py34/lib/python3.4/site-packages/pytest_html/plugin.py", line 250, in pytest_sessionfinish
logfile.write(doc.unicode(indent=2))
UnicodeEncodeError: 'utf-8' codec can't encode character '\ud800' in position 9233: surrogates not allowed
Since I was the one contributing python 3 support, I can look at this issue in some days/weeks, but I'm rather busy currently.
Along with result, test, duration and links, it'd be great if we have a test_case description field in the test report. The values for the description field will be the docstring for that specific test case.
If we already have a way to achieve this, please let me know.
Firefox 47.0 Ubuntu 16.04 LTS
Browser is asking to stop script when trying to open big html report.
Here is the sample report with 2558 tests: https://r-ci.tk/stuff/pytest-old.html
Since #37 Pytest-html gives support to rerun, it would be nice if we could collapse all the reruns by test name
It's probably rather early to report this, but I thought I'd do so anyways.
When using pytest-html with the latest pytest version from their git repo with -rw (show warning texts) I get this:
WI2 /.../pytest_html/plugin.py:28 use pluginmanager.add_hookspecs instead of deprecated addhooks() method.
I got this weird error with 1.6:
Traceback (most recent call last):
File "/var/lib/jenkins/virtenv/428e316eff82784c964740c18463dd91/lib/python2.7/site-packages/_pytest/main.py", line 84, in wrap_session
doit(config, session)
File "/var/lib/jenkins/virtenv/428e316eff82784c964740c18463dd91/lib/python2.7/site-packages/_pytest/main.py", line 122, in _main
config.hook.pytest_runtestloop(session=session)
File "/var/lib/jenkins/virtenv/428e316eff82784c964740c18463dd91/lib/python2.7/site-packages/_pytest/core.py", line 521, in __call__
return self._docall(self.methods, kwargs)
File "/var/lib/jenkins/virtenv/428e316eff82784c964740c18463dd91/lib/python2.7/site-packages/_pytest/core.py", line 528, in _docall
firstresult=self.firstresult).execute()
File "/var/lib/jenkins/virtenv/428e316eff82784c964740c18463dd91/lib/python2.7/site-packages/_pytest/core.py", line 394, in execute
res = method(*args)
File "/var/lib/jenkins/virtenv/428e316eff82784c964740c18463dd91/lib/python2.7/site-packages/xdist/dsession.py", line 521, in pytest_runtestloop
self.loop_once()
File "/var/lib/jenkins/virtenv/428e316eff82784c964740c18463dd91/lib/python2.7/site-packages/xdist/dsession.py", line 539, in loop_once
call(**kwargs)
File "/var/lib/jenkins/virtenv/428e316eff82784c964740c18463dd91/lib/python2.7/site-packages/xdist/dsession.py", line 582, in slave_errordown
self.config.hook.pytest_testnodedown(node=node, error=error)
File "/var/lib/jenkins/virtenv/428e316eff82784c964740c18463dd91/lib/python2.7/site-packages/_pytest/core.py", line 521, in __call__
return self._docall(self.methods, kwargs)
File "/var/lib/jenkins/virtenv/428e316eff82784c964740c18463dd91/lib/python2.7/site-packages/_pytest/core.py", line 528, in _docall
firstresult=self.firstresult).execute()
File "/var/lib/jenkins/virtenv/428e316eff82784c964740c18463dd91/lib/python2.7/site-packages/_pytest/core.py", line 394, in execute
res = method(*args)
File "/var/lib/jenkins/virtenv/428e316eff82784c964740c18463dd91/lib/python2.7/site-packages/pytest_html/plugin.py", line 61, in pytest_testnodedown
node.config._environment = node.slaveoutput['environment']
AttributeError: 'SlaveController' object has no attribute 'slaveoutput'
Any idea what's going on?
Captured logs are included but captured stdout/stderr is not.
(tested with pytest 2.7.1)
If the tests are running in a familiar CI, then we should grab certain environment variables for inclusion in the environment section of the report.
Of particular interest is a URL to the CI build and to the revision being tested.
One more suggest from me. Hope I'm not impudent, well not yet :)
Docstring is a cool feature, and with it we don't have to describe out tests somewhere else istead of test itself. I would suggest to read docstring from class/function and append it to the title here:
https://github.com/davehunt/pytest-html/blob/master/pytest_html/plugin.py#L154
optionally. Optionally, because when someone wants to perform doctest (dunno if there someone which does :) ), the feature will destroy the report.
It would be nice if there was a search box where the user could search by the test tittle
Tried to add the code below, but got error - AttributeError: 'TestReport' object has no attribute 'extra'
from py.xml import html
from html import extras
def pytest_runtest_makereport(multicall, item):
report = multicall.execute()
extra = getattr(report, 'extra', [])
if report.when == 'call':
xfail = hasattr(report, 'wasxfail')
if (report.skipped and xfail) or (report.failed and not xfail):
url = TestSetup.selenium.current_url
report.extra.append(extras.url(url))
screenshot = TestSetup.selenium.get_screenshot_as_base64()
report.extra.append(extras.image(screenshot, 'Screenshot'))
html = TestSetup.selenium.page_source.encode('utf-8')
report.extra.append(extra.text(html, 'HTML'))
report.extra.append(extra.html(html.div('Additional HTML')))
report.extra = extra
return report
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File "D:\Python27\lib\site-packages_pytest\main.py", line 84, in wrap_session
INTERNALERROR> doit(config, session)
INTERNALERROR> File "D:\Python27\lib\site-packages_pytest\main.py", line 122, in _main
INTERNALERROR> config.hook.pytest_runtestloop(session=session)
...........................
pytest_runtest_makereport
INTERNALERROR> report.extra.append(extra.text(item._obj.doc.strip(), 'HTML'))
INTERNALERROR> AttributeError: 'TestReport' object has no attribute 'extra'
According to #29 , if the JS is removed, we lost the ability to load images in data URI
In the recent times many IDEs have become popular for Python development. Among them are Jetbrain Community's Pycharm and PyDev, an IDE extended from Eclipse for Python development.
Upon opening a project within, these IDEs generate certain files and directories which are specific to them, and are not project specific.
These files and directories need not be pushed on Github with the main source. So they must be included in .gitignore file.
What's the state of Python3 support with pytest-html?
In html/plugin.py I noticed this:
# Python 2.X and 3.X compatibility
if sys.version_info[0] < 3:
from codecs import openHowever python3 tests were removed in 30a37b7 and it doesn't seem to work - one reason seems to be the html package name conflicts with the html module in the stdlib:
Downloading/unpacking pytest-html
Downloading pytest-html-1.0.tar.gz
Running setup.py (path:/tmp/pip-build-v2gumy9z/pytest-html/setup.py) egg_info for package pytest-html
Traceback (most recent call last):
File "<string>", line 3, in <module>
File "/home/florian/proj/qutebrowser/git/.tox/unittests/lib/python3.4/site-packages/setuptools/__init__.py", line 12, in <module>
from setuptools.extension import Extension
File "/home/florian/proj/qutebrowser/git/.tox/unittests/lib/python3.4/site-packages/setuptools/extension.py", line 7, in <module>
from setuptools.dist import _get_unpatched
File "/home/florian/proj/qutebrowser/git/.tox/unittests/lib/python3.4/site-packages/setuptools/dist.py", line 16, in <module>
from setuptools.depends import Require
File "/home/florian/proj/qutebrowser/git/.tox/unittests/lib/python3.4/site-packages/setuptools/depends.py", line 6, in <module>
from setuptools import compat
File "/home/florian/proj/qutebrowser/git/.tox/unittests/lib/python3.4/site-packages/setuptools/compat.py", line 44, in <module>
from html.entities import name2codepoint
ImportError: No module named 'html.entities'
Would a pull request for python3 support be welcome? Then I'd like to get this working - it'll mean the package needs to be renamed however, e.g. to pytest_html.
Also, would it be okay to introduce six as a dependency if it makes things easier?
The pytest-rerunfailures plugin automatically reruns failed tests. We should include the reruns in the HTML report, including any debug information. This can help to identify if persistent failures are failing in a consistent manner. Each rerun should have its own row similar to failure but the status should be orange. The total number of reruns should be reported in the summary. Note that due to pytest-dev/pytest#1193, if pytest-xdist is used, reruns will not be included in the report.
Hi,
I managed to send the html result (say report.html) through mail, but Gmail shows the mail without all the coloring, tables etc.
Any idea?
Here's out it looks on Chrome (great) and Gmail (not so great). I guess I miss something in my e-mail header:
To: [email protected]
From: [email protected]
Content-Type: text/html; charset=UTF-8
Subject: kuka
Thanks!
Yuval.
Chrome:
Gmail:
Test assumes that you already have pytest-selenium installed into your virtualenv.
Create a simple test called test_debug_pytest_html.py
import pytest
class TestDebugPytestHtml(object):
@pytest.mark.capabilities(marionette='true')
def test_user_can_log_in(self, selenium):
selenium.get('http://www.mozilla.org')
assert selenium.title == 'We love Firefox!'Now run this test from the command py.test --driver Firefox --html test_debug_pytest_html.py
Expected result: Run test with a report written to some file name like report.html
Actual Result: The html report is appended to my python test file.
Hello folks!
Seems to be, screenshots for the report (as for selenium tests) stored right in the report file, encoded as base64 and then linked to the document. Also, driver log, browser log is stored same way. Is it possible to handle the screenshots data and send it to external source (file, or using external server by uploading as POST form-data), and return the link to the source instead of the source itself?
Hi,
I get following error when I am trying to use code snippet provided by you to add details to the HTML reports.
extra.append(extras.url('http://www.example.com/'))
AttributeError: 'module' object has no attribute 'url'
Here is the code we are trying to run-
from py.xml import html
import extras
import os.path
import url
def pytest_runtest_makereport(__multicall__, item):
report = __multicall__.execute()
extra = getattr(report, 'extra', [])
if report.when == 'call':
# always add url to report
extra.append(extras.url('http://www.example.com/'))
xfail = hasattr(report, 'wasxfail')
if (report.skipped and xfail) or (report.failed and not xfail):
# only add additional html on failure
extra.append(extra.html(html.div('Additional HTML')))
report.extra = extra
return reportA feature request - would it be possible to add makereport hooks that allow the user to configure columns to add, or even hide? For instance, add a column with the docstring for a test description.
Hi,
Could you please add a new feature that user could collapse the error or failure log? it's a little bit hard to find the useful information once there're hundreds of errors or failures in the report.
For better understanding, I would like to give an example:
Regards,
Hailin
When doing tox -e py27 locally, TestHTML.test_environment_xdist_reruns hangs for about a minute, eats 10 GB of RAM, and then fails with 12000 lines of output:
=================================== FAILURES ===================================
____________________ TestHTML.test_environment_xdist_reruns ____________________
self = <test_pytest_html.TestHTML instance at 0x7fab1c54ba28>
testdir = <Testdir local('/tmp/pytest-of-florian/pytest-12/testdir/test_environment_xdist_reruns0')>
def test_environment_xdist_reruns(self, testdir):
content = str(random.random())
testdir.makeconftest("""
import pytest
@pytest.fixture(autouse=True)
def _environment(request):
for i in range(2):
request.config._environment.append(('content', '{0}'))
""".format(content))
testdir.makepyfile('def test_fail(): assert False')
result, html = run(testdir, 'report.html', '-n', '1', '--reruns', '1')
assert result.ret
> assert 'Environment' in html
E assert 'Environment' in '<!DOCTYPE html><html>\n <head>\n <meta charset="utf-8"/>\n <title>Test Report</title>\n <style>body {\n\tfo...ation</th>\n <th>Links</th></tr></thead>\n <tbody id="results-table-body"></tbody></table></body></html>'
/home/florian/proj/pytest-html/test_pytest_html.py:322: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux2 -- Python 2.7.9, pytest-2.8.7, py-1.4.31, pluggy-0.3.1
rootdir: /tmp/pytest-of-florian/pytest-12/testdir/test_environment_xdist_reruns0, inifile:
plugins: rerunfailures-0.05, xdist-1.13.1, html-1.7
gw0 I
[gw0] node down: Traceback (most recent call last):
File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/execnet/gateway_base.py", line 841, in _local_receive
data = loads_internal(data, channel, strconfig)
File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/execnet/gateway_base.py", line 1350, in loads_internal
return Unserializer(io, channelfactory, strconfig).load()
File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/execnet/gateway_base.py", line 1152, in load
opcode = self.stream.read(1)
File "/usr/lib/python2.7/StringIO.py", line 127, in read
_complain_ifclosed(self.closed)
TypeError: 'NoneType' object is not callable
Replacing crashed slave gw0
[gw1] node down: Traceback (most recent call last):
[...]
Replacing crashed slave gw288
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/_pytest/main.py", line 90, in wrap_session
INTERNALERROR> session.exitstatus = doit(config, session) or 0
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/_pytest/main.py", line 121, in _main
INTERNALERROR> config.hook.pytest_runtestloop(session=session)
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 724, in __call__
INTERNALERROR> return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 338, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 301, in __call__
INTERNALERROR> return outcome.get_result()
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 279, in get_result
INTERNALERROR> _reraise(*ex) # noqa
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 264, in __init__
INTERNALERROR> self.result = func()
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 299, in <lambda>
INTERNALERROR> outcome = _CallOutcome(lambda: self.oldcall(hook, hook_impls, kwargs))
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 333, in <lambda>
INTERNALERROR> _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 596, in execute
INTERNALERROR> res = hook_impl.function(*args)
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/xdist/dsession.py", line 521, in pytest_runtestloop
INTERNALERROR> self.loop_once()
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/xdist/dsession.py", line 539, in loop_once
INTERNALERROR> call(**kwargs)
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/xdist/dsession.py", line 603, in slave_errordown
INTERNALERROR> self._clone_node(node)
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/xdist/dsession.py", line 665, in _clone_node
INTERNALERROR> node = self.nodemanager.setup_node(spec, self.queue.put)
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/xdist/slavemanage.py", line 49, in setup_node
INTERNALERROR> gw = self.group.makegateway(spec)
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/execnet/multi.py", line 127, in makegateway
INTERNALERROR> io = gateway_io.create_io(spec, execmodel=self.execmodel)
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/execnet/gateway_io.py", line 113, in create_io
INTERNALERROR> return Popen2IOMaster(args, execmodel)
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/execnet/gateway_io.py", line 17, in __init__
INTERNALERROR> self.popen = p = execmodel.PopenPiped(args)
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/execnet/gateway_base.py", line 178, in PopenPiped
INTERNALERROR> return self.subprocess.Popen(args, stdout=PIPE, stdin=PIPE)
INTERNALERROR> File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
INTERNALERROR> errread, errwrite)
INTERNALERROR> File "/usr/lib/python2.7/subprocess.py", line 1231, in _execute_child
INTERNALERROR> self.pid = os.fork()
INTERNALERROR> OSError: [Errno 12] Cannot allocate memory
======================== no tests ran in 80.46 seconds =========================
----------------------------- Captured stderr call -----------------------------
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/_pytest/main.py", line 90, in wrap_session
INTERNALERROR> session.exitstatus = doit(config, session) or 0
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/_pytest/main.py", line 120, in _main
INTERNALERROR> config.hook.pytest_collection(session=session)
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 724, in __call__
INTERNALERROR> return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 338, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 333, in <lambda>
INTERNALERROR> _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 596, in execute
INTERNALERROR> res = hook_impl.function(*args)
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/_pytest/main.py", line 129, in pytest_collection
INTERNALERROR> return session.perform_collect()
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/_pytest/main.py", line 565, in perform_collect
INTERNALERROR> hook.pytest_collection_finish(session=self)
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 724, in __call__
INTERNALERROR> return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 338, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 333, in <lambda>
INTERNALERROR> _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 596, in execute
INTERNALERROR> res = hook_impl.function(*args)
INTERNALERROR> File "<remote exec>", line 77, in pytest_collection_finish
INTERNALERROR> File "<remote exec>", line 23, in sendevent
INTERNALERROR> File "/home/florian/proj/pytest-html/.tox/py27/local/lib/python2.7/site-packages/execnet/gateway_base.py", line 716, in send
INTERNALERROR> raise IOError("cannot send to %r" % (self,))
INTERNALERROR> IOError: cannot send to <Channel id=1 closed>
[...]
@ronny @nicoddemus @bobsilverberg any idea what's going on here? I'm not familiar with xdist/execnet at all. The test source is here:
https://github.com/davehunt/pytest-html/blob/master/test_pytest_html.py#L310-L323
at that point its better to create a package and put the other files in
Because of CSP restrictions on inline CSS, the CSS file should be written apart
When a Test is FAILED and wasxfail is set why it is marked as XPASSED, Shouldn't it be XFAILED?
Also when wasxfail is set and test is PASSED why it is not marked as XPASSED?
def append_passed(self, report):
if report.when == 'call':
self.passed += 1
self._appendrow('Passed', report)
def append_failed(self, report):
if report.when == "call":
if hasattr(report, "wasxfail"):
self.xpassed += 1
self._appendrow('XPassed', report)
else:
self.failed += 1
self._appendrow('Failed', report)
IMO above code should be modified as
def append_passed(self, report):
if report.when == 'call':
if hasattr(report, "wasxfail"):
self.xpassed += 1
self._appendrow('XPassed', report)
else:
self.passed += 1
self._appendrow('Passed', report)
def append_failed(self, report):
if report.when == "call":
if hasattr(report, "wasxfail"):
self.xfailed += 1
self._appendrow('XFailed', report)
else:
self.failed += 1
self._appendrow('Failed', report)
Hi,
Great job for pytest-html: I like it very much!
I am customizing the "Environment" part in my html report and it works fine when running tests one by one.
However, when using pytest-xdist, this part is set to the default 2 values: Platform and Python.
Is it a but or am I using the wrong implementation?
Note: Might be related to issue #79 ...
Thanks
This is the code I have in conftest.py
#==Retrieve Test Environment==#
def get_testenv():
'''Retrieve Test Environment and store if for later use'''
testenv_ini = "testenv.ini"
testenv = ConfigParser.ConfigParser()
testenv.read(testenv_ini)
if not testenv.has_option('TEST_ENV', '2.1 Python') \
or testenv.get('TEST_ENV', '2.1 Python') != platform.python_version():
#Get Environment
if not testenv.has_section('TEST_ENV'): testenv.add_section('TEST_ENV')
#Hardware
OS = subprocess.check_output('wmic os get BuildNumber, Caption, CSDVersion, OSArchitecture')
RAM = [l.strip() for l in subprocess.check_output('wmic memorychip get capacity').splitlines() if re.match(r'\d', l)]
testenv.set('TEST_ENV', '1.1 Machine', platform.node())
testenv.set('TEST_ENV', '1.2 OS', re.sub(r'^.*OSArchitecture\s*(\d+)\s*(.*?) (.*?)\s+((?:32|64)-bit)\s*$', r'\2 \4, \3, Build \1', OS))
testenv.set('TEST_ENV', '1.3 Platform', platform.platform())
testenv.set('TEST_ENV', '1.4 Model', re.sub(r'^\w+\s*(.*?)\s*$', r'\1', subprocess.check_output('wmic computersystem get model')) )
testenv.set('TEST_ENV', '1.5 RAM', sum([int(l)/1048576 for l in RAM]))
testenv.set('TEST_ENV', '1.6 CPU', re.sub(r'^\w+\s*(.*?)\s*$', r'\1', subprocess.check_output('wmic cpu get name')))
#Python and Diff
testenv.set('TEST_ENV', '2.1 Python', platform.python_version())
testenv.set('TEST_ENV', '2.2 GNU diff', subprocess.check_output('diff -v').splitlines()[0])
with open(testenv_ini, 'w') as file:
testenv.write(file)
else:
pass #Environment file up to date
return testenv
@pytest.fixture(autouse=True)
def _environment(request):
'''Add extra information on Environment in html report'''
testenv = get_testenv()
request.config._environment = [(tool, info) for tool, info in testenv.items('TEST_ENV')]pip install pytest-html as mentioned in the README does not work, as the project doesn't seem to be on PyPI.
Hi, Dave!
Is it possible to show a screenshots for the passed tests as well?
This is my code:
def pytest_runtest_makereport(__multicall__, item):
_driver = item.funcargs['app'].driver
browser_type = _driver.capabilities['browserName']
pytest_html = item.config.pluginmanager.getplugin('html')
report = __multicall__.execute()
extra = getattr(report, 'extra', [])
if report.when == 'call':
url = _driver.current_url
extra.append(pytest_html.extras.url(url))
screenshot = _driver.get_screenshot_as_base64()
extra.append(pytest_html.extras.image(screenshot, ''))
report.extra = extra
return reportBoth passed and failed tests has an URL field,
however, only failed tests are displayed with screenshot.
BR and thanks for the great plugin :)
Michael
Hi,
I have a test which is "xpassed", reported as such by pytest and recorded as "passed" in the html report.
I have the following test file
import pytest
def test_failed():
assert 3 + 4 == 6
def test_passed():
assert 4 + 4 == 8
@pytest.mark.xfail(reason="test xfailed")
def test_xfailed():
assert False
@pytest.mark.xfail(reason="test xpassed")
def test_xpassed():
assert TrueWould it be possible to expose a flag that allows merging multiple html reports?
This is something that could probably be done with a separate script, but I think it makes sense to add this functionality to the plugin. Something like:
py.test --merge-html result1.html result2.html ... resultN.html
Cheers,
Goncalo
I love this plug in, is this included in pytest by default now?
I've incorporated in jenkins, and it works well, save for the screenshot bit.
Any help?
I would like to have an ability to suppress log section in case if I'm using additional HTML in the report and log itself does not contains any information besides of "No log output captured."
(could you just remove lines 143-145 and move additional_html.append(log) inside the IF section in the plugin.py file)
I'm using:
I get the following error on a test fail:
INTERNALERROR> data = b64encode(content.encode(charset)).decode('ascii')
INTERNALERROR> AttributeError: 'bytes' object has no attribute 'encode'
> /Users/caged/miniconda3/envs/bokeh_34/lib/python3.4/site-packages/pytest_html-1.7-py3.4.egg/pytest_html/plugin.py(74)data_uri()
-> data = b64encode(content.encode(charset)).decode('ascii')
72 def data_uri(content, mime_type='text/plain', charset='utf-8'):
73 if PY3:
74 -> data = b64encode(content.encode(charset)).decode('ascii')
75 else:
76 data = b64encode(content)
77 return 'data:{0};charset={1};base64,{2}'.format(mime_type, charset, data)When in pdb, just running b64encode(content) works great - maybe PY3 check isn't quite right? Could we check what content is instead?
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.