dequelabs/axe-core-maven-html

Tools for using axe for web accessibility testing with JUnit, Selenium, and Playwright

axe-core-maven-html

This repository contains 2 packages, which can be used for automated accessibility testing powered by axe core.

The packages are listed below:

• Selenium
• Playwright

Development

Install root dependencies:

npm install

Please refer to respective README for installation, usage, and configuration notes.

Philosophy

We believe that automated testing has an important role to play in achieving digital equality and that in order to do that, it must achieve mainstream adoption by professional web developers. That means that the tests must inspire trust, must be fast, must work everywhere and must be available everywhere.

Manifesto

1. Automated accessibility testing rules must have a zero false positive rate
2. Automated accessibility testing rules must be lightweight and fast
3. Automated accessibility testing rules must work in all modern browsers
4. Automated accessibility testing rules must, themselves, be tested automatically

Contributing

In order to contribute, you must accept the contributor licence agreement (CLA). Acceptance of this agreement will be checked automatically and pull requests without a CLA cannot be merged.

Deployment (Maintainers Only)

This package is deployed to Maven Central via OSSRH. To deploy this package, follow these instructions on StackOverflow.

Additionally add your OSSRH credentials to your ~/.m2/settings.xml file as such:

<servers>
  <server>
    <id>ossrh</id>
    <username>YOUR_OSSRH_JIRA_USERNAME</username>
    <password>YOUR_OSSRH_JIRA_PASSWORD</password>
  </server>
</servers>

axe-core-maven-html's Issues

org.openqa.selenium.WebDriverException: javascript error: axe.a11yCheck is not a function

The axe-selenium version in Readme needs to be updated. I ended up using 2.0 and ran the error from 2.0 (Related issue: dequelabs/axe-core#833) even though I was using the latest axe-min.js. Took me some effort to figure out what is happening

JVM STACKTRACE:
java.lang.Throwable: ( username: [user context not established] )
..
org.openqa.selenium.WebDriverException: javascript error: axe.a11yCheck is not a function
JavaScript stack:
TypeError: axe.a11yCheck is not a function
at eval (eval at executeAsyncScript (:457:5), <anonymous>:3:24)
at eval (eval at executeAsyncScript (:457:5), <anonymous>:3:86)
at executeAsyncScript (<anonymous>:457:26)
at <anonymous>:473:29
at callFunction (<anonymous>:361:33)
at <anonymous>:371:23
at <anonymous>:372:3

Allow option to skip iframe injection

Hi,

Is it possible to add an option to skip the following line?
https://github.com/dequelabs/axe-selenium-java/blob/master/src/main/java/com/deque/axe/AXE.java#L72

The main reason is because this call is super slow. It's currently preventing me to use this in our CI pipeline because it's adding extra 30s for each page just to execute this line:

https://github.com/dequelabs/axe-selenium-java/blob/9b2cabcd8611c29448654422bec70c18a6aa3735/src/main/java/com/deque/axe/AXE.java#L87

Update to use new default reporter

Other components rely on the new format, this integration should be updated to use that new format.

AxeReporter: Select any of the reports to be in the HTML reporter

User should be able to select from any of the four ResultTypes to be in the report.
For instance, if I want the violations, passes, and Inapplicable in my report, they should be included in the report html.
This means that in this case the Incompletes will not be in the reports.

Support axe-core 3.0

As mentioned here: dequelabs/axe-core#833

This project does not yet support axe-core 3.0.

Unexpected token

I'm running into an issue when attempting to use axe 2.1 with my selenium tests. I've borrowed the code from the SetupTest.java class. Any suggestions or examples you could share?

Code Snip:

public class AccessibilityHandler {

private static final URL scriptUrl = AccessibilityHandler.class.getResource("/axe.min.js");
private static final Logger LOGGER = LoggerFactory.getLogger(AccessibilityHandler.class);

public static void testAccessibility(WebDriver driver) {
JSONObject responseJSON;
try {
responseJSON = new AXE.Builder(driver, scriptUrl).analyze();
JSONArray violations = responseJSON.getJSONArray("violations");
if (violations.length() == 0) {
assertTrue("No violations found", true);
} else {
AXE.writeResults("Test Method", responseJSON);
}
} catch (Exception e) {
LOGGER.error("*****************an error occurred with analyze", e);
}
}
}

### Sanitized Error log:
org.openqa.selenium.WebDriverException: unknown error: Runtime.evaluate threw exception: SyntaxError: Unexpected token <
(Session info: chrome=65.0.3325.146)
(Driver info: chromedriver=2.38.552518 (183d19265345f54ce39cbb94cf81ba5f15905011),platform=Mac OS X 10.13.4 x86_64) (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 0 milliseconds
Build info: version: '3.14.0', revision: 'aacccce0', time: '2018-08-02T20:19:58.91Z'
System info: host: XXXXXXXXXXXXX, os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.13.4', java.version: '1.8.0_171'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Capabilities {acceptInsecureCerts: false, acceptSslCerts: false, applicationCacheEnabled: false, browserConnectionEnabled: false, browserName: chrome, chrome: {chromedriverVersion: 2.38.552518 (183d19265345f5..., userDataDir: /var/folders/w9/xgvfht4s1xd...}, cssSelectorsEnabled: true, databaseEnabled: false, handlesAlerts: true, hasTouchScreen: false, javascriptEnabled: true, locationContextEnabled: true, mobileEmulationEnabled: false, nativeEvents: true, networkConnectionEnabled: false, pageLoadStrategy: normal, platform: MAC, platformName: MAC, rotatable: false, setWindowRect: true, takesHeapSnapshot: true, takesScreenshot: true, unexpectedAlertBehaviour: , unhandledPromptBehavior: , version: 65.0.3325.146, webStorageEnabled: true}
Session ID: 762e154cbaa0f081ad27d49fa135b891
at sun.reflect.GeneratedConstructorAccessor15.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.openqa.selenium.remote.ErrorHandler.createThrowable(ErrorHandler.java:214)
at org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed(ErrorHandler.java:166)
at org.openqa.selenium.remote.http.JsonHttpResponseCodec.reconstructValue(JsonHttpResponseCodec.java:40)
at org.openqa.selenium.remote.http.AbstractHttpResponseCodec.decode(AbstractHttpResponseCodec.java:80)
at org.openqa.selenium.remote.http.AbstractHttpResponseCodec.decode(AbstractHttpResponseCodec.java:44)
at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:158)
at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:83)
at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:548)
at org.openqa.selenium.remote.RemoteWebDriver.executeScript(RemoteWebDriver.java:485)
at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.openqa.selenium.support.events.EventFiringWebDriver.lambda$new$1(EventFiringWebDriver.java:105)
at com.sun.proxy.$Proxy11.executeScript(Unknown Source)
at com.deque.axe.AXE.inject(AXE.java:79)
at com.deque.axe.AXE$Builder.execute(AXE.java:322)
at com.deque.axe.AXE$Builder.analyze(AXE.java:307)

Help URL is not URL pointing to help in AxeReporter

Line 153 in AxeReporter.java

Implement Marionette Driver in ExampleTest.java

Hello All!

Mozilla has taken over development of the FirefoxDriver from Selenium and have started implementing Marionette starting around Firefox 46.0.

What that means is that users may start running into:

Selenium::WebDriver::Error::WebDriverError: unable to obtain stable firefox connection in 60 seconds

Unless implementing the GeckoDriver like so:
https://developer.mozilla.org/en-US/docs/Mozilla/QA/Marionette/WebDriver

Wonderful accessibility tool I'm now leveraging! 👏

Will be shouting out in coming up user group presentation.

Help running axe in existing java/selenium test

im trying to run axe in existing java selenium test to get violations. Im using this dependency:

com.deque
axe-selenium
3.0
test

and im also using the axe.js under my test but what do i need to analyze multipule pages in one test?

Handle errors coming from axe-core

The injected JS snippets (below) do not handle the possible error coming from axe-core's run method. Scripts in question:

• https://github.com/dequelabs/axe-selenium-java/blob/914a50693058c152891202d4fb9a764c8cbcf09b/src/main/java/com/deque/axe/AXE.java#L297-L300
• https://github.com/dequelabs/axe-selenium-java/blob/914a50693058c152891202d4fb9a764c8cbcf09b/src/main/java/com/deque/axe/AXE.java#L302
• https://github.com/dequelabs/axe-selenium-java/blob/914a50693058c152891202d4fb9a764c8cbcf09b/src/main/java/com/deque/axe/AXE.java#L304
• https://github.com/dequelabs/axe-selenium-java/blob/914a50693058c152891202d4fb9a764c8cbcf09b/src/main/java/com/deque/axe/AXE.java#L316

Instead of something like:

var done = arguments[arguments.length - 1]; axe.run(arguments[0], %s, function(err, res) { done(res); });

We need to have something like:

var done = arguments[arguments.length - 1]
axe.run(context, options, function (err, res) {
  if (err) {
    handle_error_somehow(err) // <-- we need to pass the error back to Java here
  } else {
    done(res)
  }
})

In an ideal world, the err provided be axe-core would be raised as a RuntimeException from AXE.run().

Did not get all violations which are captured through chrome AXE plugin

Dear Team,

I tried using axe-selenium-java lib to get all violations on my webpage, but it does not capture all violations, however those were captured in chrome AXE plugin.

How can we capture all violations using axe-selenium-java lib? For example: we got "Banner landmark must not be contained in another landmark" violation on chrome AXE plugin, but its ignored when I used axe-selenium-java lib.

Thanks,
Manak

4.3.3 audit

• Implements recursive runPartial
  • injects and configures axe on each frame (exception: top-level iframe since axe might already be injected there from testing legacy code path)
  • calls axe.utils.getFrameContexts on each frame
  • runs axe.runPartial on each frame
  • finds each iframe using axe.utils.shadowSelect
  • calls function again with new iframe and context
  • returns null if iframe errors for any reason and does not process children iframes
• implements finishRun
  • opens new tab and navigates to blank page (using .goto or .navigate, etc.)
  • injects and configures axe
  • calls axe.finishRun and passes results from all iframes
  • closes blank page and returns driver context to original page
• calls axe.run in legacy axe versions (where axe.runPartial is not defined)
• implements all fixture tests
  • analyze - throws if axe errors out on the top window
  • analyze - throws when injecting a problematic source
  • analyze - throws when a setup fails
  • analyze - properly isolates the call to axe.finishRun
  • analyze - returns correct results metadata
  • frame tests - injects into nested iframes
  • frame tests - injects into nested frameset
  • frame tests - should work on shadow DOM iframes
  • frame tests - reports erroring frames in frame-tested
  • frame tests - returns the same results from runPartial as from legacy mode
  • for versions without axe.runPartial - can run
  • for versions without axe.runPartial - throws if the top level errors
  • for versions without axe.runPartial -reports frame-tested
  • with a custom ruleset - should find violations with customized helpUrl
  • with a custom ruleset - configures in nested frames
  • with a custom ruleset - works without runPartial

"No elements found for include in page context " error appearing when `include` having multiple web elements

Steps to reproduce:
on axe-core-maven-html: when tested for include method on multiple web elements
AxeBuilder().include(Arrays.asList("div", "h1")).analyze(webDriver);
results are blank. and giving "No elements found for include in page context"

Question: Why has this project been renamed ?

Why has this project been renamed from axe-selenium-java to axe-core-maven-html

Why is there a new project https://github.com/dequelabs/axe-selenium-java?

• It doesn't have any history. Seems to be essentially a clone of this project with few modifications
• It doesn't have a develop branch
• It doesn't have github Issues enabled

Violation does not captured for mobile simulator/devices

Setup snapshot releases (via CircleCI)

All develop commits should result in a -SNAPSHOT release being published now that we have the ability to deploy from CI.

Release v4.2

When upgrading to use [email protected] we will need to add the ability to configure frame origin

chore: upgrade to use `[email protected]`

Expectation: upgrade to use [email protected] and axe-core#runPartial()

Actual: currently uses [email protected]

Using axe in existing test cases

Im not able to use axe with existing test cases, what dependencies do i need to use AXE? i have:
com.deque
axe-selenium
3.0
test

and the axe.js file under my test. but im not able to get violations from all my pages. what do i need to do?

java.lang.RuntimeException: Error when enabling iframe communication

Getting below error while injecting axe-core script on page.

java.lang.RuntimeException: Error when enabling iframe communication
[testng] at com.deque.html.axecore.selenium.AxeBuilder.analyzeRawContext(AxeBuilder.java:457)
[testng] at com.deque.html.axecore.selenium.AxeBuilder.analyze(AxeBuilder.java:404)

writeResults should write entire results object, not violations

We my want to add a new method for this to ensure backwards compatibility.

The metadata for home page etc (urls) are wrong in maven repo

https://search.maven.org/artifact/com.deque.html.axe-core/selenium

Versions and Releases

Hello!
We are currently using axe-selenium version 3.0.
We saw that you have other versions available, including 3.1-Snapshot and 4.0 in github. Are these a replacement or we can continue using 3.0? Do you suggest we continue using 3.0 or start using newer versions? If we must switch, which maven repository actively has these dependencies?

Also, does the axe-min.js file and other js files we use in the project have to be downloaded from a matching version? Such as, The axe-min.js file taken from v. 3.0. Is it going to be compatible with newer versions or we have to get a new one?

Thank you!

Can I Run only Rules for an accessibility standard? For eg: WCAG 2.0 Level A and exclude WCAG 2.0 Level AA. This is similar to using .runonly option of axe.configure in axe-core? How can I achieve this in the test I am running in ExampleTest.java here in axe-selenium-java? Thank you!

Getting Runtime Exception when axe is analyzing for issues in selenium java

Selenium - 3.141.0

Axe Maven dependency

com.deque.html.axe-core
selenium
4.1.1

Exception - java.lang.RuntimeException: Unable to inject axe script

at com.deque.html.axecore.selenium.AxeBuilder.analyzeRawContext(AxeBuilder.java:436)
at com.deque.html.axecore.selenium.AxeBuilder.analyze(AxeBuilder.java:390)
at com.automation.pom.AxeRun.runAxeReport(AxeRun.java:50)

aXe selenium java v3.0.0 alpha.9 - TypeError: axe.run arguments are invalid

Hello,

Upgraded aXe for selenium java version to 3.0.0 (axe.min.js , test-app.js and AXE.java). receiving below error during execution.

AxeReporter is not appending nodes in axe results

https://github.com/tsharma-indeed/axe-core-maven-html/blob/develop/src/main/java/com/deque/html/axecore/selenium/AxeReporter.java#L160

The below line has an issue. This if statement will never get executed and hence, nodes willl not get appended to the axe results
if (element.getNodes() != null && element.getNodes().isEmpty())

Fix: !element.getNodes().isEmpty() Not (!) should be added to this statement so that further if statements will able to hit

Bundled Dependencies currently getting hamcrest mismatch error

Currently when using our bundled java dependencies in a project, when running test command we get the following error:

java.lang.NoSuchMethodError: org.hamcrest.Matcher.describeMismatch(Ljava/lang/Object;Lorg/hamcrest/Description;)V
at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:8)

The test cases run and the results are created via log and JSON, however, we do not get that the test case failed because isAxeClean() was asserted, its that there is a hamcrest mismatch.

Here is how I am currently calling the local jars (a setup most clients we have who use them use):

 <dependency>
  <groupId>com.deque.html.axe-devtools</groupId>
  <artifactId>selenium</artifactId>
  <version>4.1.2</version>
  <classifier>jar-with-dependencies</classifier>
  <scope>system</scope>
  <systemPath>${basedir}/src/test/resources/selenium-4.1.2-jar-with-dependencies.jar</systemPath>
</dependency>

I have currently tried matching the versions of hamcrest, excluding hamcrest matchers that are bundled in J-unit, AND have tried maven installing, all with no luck at all to get the error to stop.

If more information is needed please let me know!

Allow sandboxed iframe testing

We need to be able to test iframes with the sandbox attribute:

<iframe sandbox src="..."></iframe>

We have a solution for this in axe-webdriverjs that we can copy here.

AxeReporter: Write results as an HTML file

It would be handy for the Axe scan to be able to be configured to an HTML doc.
Having the html doc would be easier for developers to use.
The Axe C# equivalent has this functionality: https://github.com/TroyWalshProf/SeleniumAxeDotnet/blob/master/Selenium.Axe/Selenium.Axe/HtmlReport.cs

New release 4.3.2

Hi guys! Thank you for you work first of all, your tool really helps us.

Do you have any plans for new releases of axe-core-maven-html to maven central? In my company we need to use the latest version.

Thanks)

Include / exclude is incorrectly passed to axe-core

When I specify an "exclude" in the builder nothing seems to happen, the exclude is ignored.

I found this is caused by a flaw in the analyze() method. It incorrectly passes the include / exclude object as options param instead of context as stated in the axe-core docs.

Also it does not look like the case with only supplying excludes and no includes is properly handled. As per the axe-core docs I would not expect an "include" property to be set in the context, just the exclude property.

Performance issue axe selenium 3.0

I'm using axe selenium 3.0 I noticed accessibility code scan takes more time while inject. Almost I'm seeing 3-5min to complete the scan.
While debug noticed there is a significant time when complete the below line of code
AXE.inject(this.driver, this.script, this.skipFrames);

Please advise.

org.openqa.selenium.WebDriverException: syntax error

Please find trace below,

org.openqa.selenium.WebDriverException: syntax error
Command duration or timeout: 2.19 seconds
Build info: version: '2.53.1', revision: 'a36b8b1cd5757287168e54b817830adce9b0158d', time: '2016-06-30 19:26:09'
System info: host: 'UserName', ip: '192.168.0.1', os.name: 'Windows 8.1', os.arch: 'x86', os.version: '6.3', java.version: '1.8.0_25'
Driver info: org.openqa.selenium.firefox.FirefoxDriver
Capabilities [{applicationCacheEnabled=true, rotatable=false, handlesAlerts=true, databaseEnabled=true, version=32.0, platform=WINDOWS, nativeEvents=false, acceptSslCerts=true, webStorageEnabled=true, locationContextEnabled=true, browserName=firefox, takesScreenshot=true, javascriptEnabled=true, cssSelectorsEnabled=true}]
Session ID: 2839d43e-ad73-453b-818c-94f9935abdd3
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
at org.openqa.selenium.remote.ErrorHandler.createThrowable(ErrorHandler.java:206)
at org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed(ErrorHandler.java:158)
at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:678)
at org.openqa.selenium.remote.RemoteWebDriver.executeScript(RemoteWebDriver.java:577)
at com.deque.axe.AXE.inject(AXE.java:76)
at com.deque.axe.AXE$Builder.(AXE.java:235)
at com.test.test.ExampleTest.testAccessibilityWithSelector(ExampleTest.java:89)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
Caused by: org.openqa.selenium.WebDriverException: syntax error
Build info: version: '2.53.1', revision: 'a36b8b1cd5757287168e54b817830adce9b0158d', time: '2016-06-30 19:26:09'
System info: host: 'UserName', ip: '192.168.0.1', os.name: 'Windows 8.1', os.arch: 'x86', os.version: '6.3', java.version: '1.8.0_25'
Driver info: driver.version: unknown
at .handleEvaluateEvent(http://127.0.0.1/login:68)

Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test

I am not able to run the example in my environment, I get the following errors:

at org.junit.Assert.fail(Assert.java:88)
at com.deque.axe.ExampleTest.testAccessibility(ExampleTest.java:69)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test (default-test) on project axe-selenium: There are test failures.
[ERROR]
[ERROR] Please refer to C:\Users\mpravato\IdealProjects\axe-selenium-java-develop\target\surefire-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

I have updated all the plugins, added some others, after consultations and it still does not work.

Include axe-core by default

This package should be able to run with a default version of axe-core, which needs to be in sync with the axe-core version. i.e. if it is on version axe-core-selenium 4.1, that will have axe-core version 4.1 as a default. You should still be able to pass a source for different versions of axe-core.

Allow axe to run on just the page source instead of injecting into DOM

Is it possible to run axe on a given page source html string instead of having to inject itself into the DOM? If this is possible, could it enable making WebDriver as an optional dependency e.g. have alternate Builder() APIs in which we pass in the HTML source instead of the webdriver?

Upgrading to 3.1

Hello,

We want to upgrade to version 3.1 . We understand that it has been moved to [axe-core-maven-html].
We cannot find the deployed jar in maven repos. Can you please let us know where to get it from?

We searched for a dependency to look like this:

<dependency> <groupId>com.deque.html.axe-core</groupId> <artifactId>selenium</artifactId> <version>3.1</version> <scope>test</scope> </dependency>

Thank you,

Getting Error while calling analyze method java.lang.RuntimeException: Error when removing sandbox from iframes in Selenium with java

Software dependency versions used:

com.deque.html.axe-core : 4.0.0

Calling methods:

When we call the method : Results result = builder.withoutIframeSandboxes().analyze(driver)

and

Results result = builder.analyze(driver)

Error that we are getting:

"java.lang.RuntimeException: Error when removing sandbox from iframes"

Note: For few of the other applications, the Analyzer method is working as expected. But on few applications, throwing above exception message.

Here we can see the code that we have used:

File file = new File("src/main/resources/axe.min.js");

	String path = file.getCanonicalPath();
	
	AxeBuilder builder = new AxeBuilder();
	 
	String timeoutFilePath = "src/main/resources/axe.min.js";
	 
	FileAxeScriptProvider axeScriptProvider = new FileAxeScriptProvider(timeoutFilePath);
	 
	builder.setAxeScriptProvider(axeScriptProvider);
	 
	Results result = builder.analyze(driver);
	 
	List<Rule> violations = result.getViolations();
	 
	Assert.assertEquals(0, violations.size(), "No violations found");

Can any one please check share what exactly issue with AXE or application side?

axe update for java

Hey guys - i see that the core is getting updated for javascript related projects but not for java. Is there any plan to update the java project which is this one . I have been using axe for running report violations and i see difference in result when i run a plugin via chrome Vs via java code.

Tests fail when trying to run on remote browsers.

Expectation: Should be able to run the tests on remote browsers. We use a cross browser testing tool called Lambdatest(browsers on cloud server) and we want to configure the tests integrated with axe to be run on the tool.
Actual: The tests fail with below error
org.openqa.selenium.remote.UnreachableBrowserException: Error communicating with the remote browser. It may have died.
Build info: version: '3.4.0', revision: 'unknown', time: 'unknown'
System info: host: 'IS-2314s-MacBook-Pro.local', ip: 'fe80:0:0:0:479:457d:6cdf:4014%en0', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.13.6', java.version: '1.8.0_131'
Driver info: driver.version: RemoteWebDriver
Capabilities [{moz:profile=C:\Users\ltuser\AppData\Local\Temp\rust_mozprofilei96m8I, rotatable=false, moz:geckodriverVersion=0.26.0, timeouts={implicit=0.0, pageLoad=300000.0, script=30000.0}, pageLoadStrategy=normal, unhandledPromptBehavior=dismiss and notify, strictFileInteractability=false, moz:headless=false, platform=ANY, moz:accessibilityChecks=false, moz:useNonSpecCompliantPointerOrigin=false, webdriver.remote.sessionid=a4c0eb44-6f3d-42e6-81bc-a9db8dcdb869, acceptInsecureCerts=true, browserVersion=74.0, moz:shutdownTimeout=60000.0, platformVersion=10.0, moz:processID=1120.0, browserName=firefox, moz:buildID=20200210140608, javascriptEnabled=true, platformName=windows, setWindowRect=true, moz:webdriverClick=true}]
Session ID: a4c0eb44-6f3d-42e6-81bc-a9db8dcdb869
at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:658)
at org.openqa.selenium.remote.RemoteWebDriver.executeAsyncScript(RemoteWebDriver.java:591)
at com.deque.axe.AXE$Builder.execute(AXE.java:372)
at com.deque.axe.AXE$Builder.analyze(AXE.java:335)
at com.scholastic.word.utility.AxeAccessibilityUtilities.testAccessibility(AxeAccessibilityUtilities.java:34)
at com.scholastic.word.stepdefinitions.AxeAccessibilityStepDefs.i_want_to_check_the_violations_on_the_overview_page(AxeAccessibilityStepDefs.java:12)
at ✽.And I want to check the violations on the overview page(AxeAccessibilityTests.feature:8)
Caused by: javax.net.ssl.SSLException: SSL peer shut down incorrectly
at sun.security.ssl.InputRecord.readV3Record(InputRecord.java:596)
at sun.security.ssl.InputRecord.read(InputRecord.java:532)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
at org.apache.http.impl.conn.LoggingInputStream.read(LoggingInputStream.java:84)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at org.apache.http.impl.io.SessionInputBufferImpl.read(SessionInputBufferImpl.java:198)
at org.apache.http.impl.io.ContentLengthInputStream.read(ContentLengthInputStream.java:176)
at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:135)
at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:148)
at org.apache.http.util.EntityUtils.toByteArray(EntityUtils.java:136)

StackOverflow exception after upgrading from axe-core 2.1.7 to 2.2.0

Calls to AXE.Builder.analyze causes a StackOverflow exception on all but the very simplest of test scenarios.

To reproduce:

1. Update axe.min.js to the 2.2.0 release (NB: the latest axe-selenium-java release still uses 2.0.5 but 2.1.7 works fine too).
2. Modify ExampleTest.java to use www.deque.com instead of test-app.js and the ChromeDriver (Firefox not tested).
3. Run the testAccessibility() test

The error is thrown on the last line of the execute() method i.e.
JSONObject((Map) response)
This appears to be due to invalid JSON in the 'incomplete' array.
Closer inspection of the response reveals several instances of '... missingData=[{node=[org.openqa.selenium.remote.RemoteWebElement@736a305c -> unknown locator], ...' which I assume is the cause of the problem.
Note that when running the same test with axe-cli the missingData element is just 'null'

It was noted that axe-selenium-core still uses Selenium 2.53.1. Version 3.4.0 was also tried with the same results.

allow analyze() to take in a list of WebElements

axe-core context accepts "A NodeList such as returned by document.querySelectorAll."

When testing axe.run() manually in browser I found that it works with a list of HTMLElement references. But the selenium API allows analyzing only one WebElement at a time. When we want to analyze a list of WebElements, injecting axe-core into DOM using Selenium and running the analyze for each element becomes expensive. Would it be possible to have an API that accepts a list of WebElements? Is there any reason that wouldn't work? If not please let me know and if possible I can try and add the API.

@stephenmathieson

Getting error on using the ExampleTest

Hello ,

Please let me know if you can locate the issue.

I renamed ExampleTest and changed the url to google.com but the script is failing on the line-
JSONObject responseJSON = new AXE.Builder(driver, scriptUrl).analyze();

as you can see in the trace below -

java.lang.RuntimeException: java.lang.NullPointerException
at com.deque.axe.AXE.getContents(AXE.java:53)
at com.deque.axe.AXE.inject(AXE.java:70)
at com.deque.axe.AXE$Builder.(AXE.java:237)
at com.deque.axe.Test1.testAccessibility(Test1.java:67)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
Caused by: java.lang.NullPointerException
at com.deque.axe.AXE.getContents(AXE.java:44)
... 30 more

axe is not defined

We are trying to integrated axe framework with existing Selenium Test cases. It is working perfectly but sometimes test cases are getting failed due to below error,

org.openqa.selenium.WebDriverException: javascript error: axe is not defined
JavaScript stack:
ReferenceError: axe is not defined
at eval (eval at executeAsyncScript (:457:5), :3:20)
at eval (eval at executeAsyncScript (:457:5), :3:86)
at executeAsyncScript (:457:26)
at :473:29
at callFunction (:361:33)
at :371:23
at :372:3

Please help me if you already have a fix for it. Thanks in advance.

document-title and html-has-lang

Not sure if I'm doing something wrong here, but I'm seeing document-title and html-has-lang issues even though the lang attribute is set and the document has a title.

feature request: configurable timeout

Hello, I am facing issues with accessibility using aXe for one of my webpages. Apparently the DOM for the webpage is too huge for the aXe tool to perform accessibility. My script is timing out after 30seconds. Is there any input limit set for the aXe tool? Please advise. #17

You can continue to use 3.0 if you want. You should be able to upgrade to 3.1 without any difficulties. The 4.x line has breaking changes, so you would need to change your test code.

require is not defined

Trying to combine this project with webdriverjs for aXe 3.0.0. I have selenium java project which navigates through web pages. After navigating to a page, I'm calling a js file named as test.js with AxeBuilder defined in it. I'm getting an error "require" is not defined in at line number 1". Is there a way to get this worked?

Setting image-alt rule false suppreses other rules violations

When I run axe with image-alt rule to false, the check passes

public void adaCheck( final String pageName) throws Exception {
AxeBuilder builder = new AxeBuilder();
builder.setOptions("{\"rules\": {\"image-alt\" : {enabled: false } } }");
Results result = builder.analyze(driver);
}

However if I run the check as normal, (without disabling the rule) then the check fails but finds issues that seem unrelated to the image-alt rule.
These are valid issues but they don't come up when image-alt rule is false.

//abridged violations object
violations : [
{ id: "aria-allowed-role",
 description: "Ensures role attribute has an appropriate value for the element",
html: "<img id="toolbar" role="presentation">",
},
{ 
 id: "empty-heading",
description: "Ensures headings have discernable text",
nodes:  [ {html: "<h2></h2>"}]
},
{
id: "landmark-one-main",
description: "Ensures the document has a main landmark"
nodes: [{
html: "<html lang="en">"}]
}
]

I can't share any code due to corporate policy but if there is a way to enable some kind of debug log I could provide that.