pa11y / pa11y Goto Github PK
View Code? Open in Web Editor NEWPa11y is your automated accessibility testing pal
Home Page: https://pa11y.org
License: GNU Lesser General Public License v3.0
Pa11y is your automated accessibility testing pal
Home Page: https://pa11y.org
License: GNU Lesser General Public License v3.0
PhantomJS previously (<= 1.9.7
) used SSLv3 for all secure connections and since a lot of servers are disabling it due to the π© attack, the --ssl-protocol=tlsv1
flag would need to be used to change the default behaviour.
PhantomJS 1.9.8 (released 5 days ago) changed the default protocol to TLSv1, fixing the issue.
This is more of a question: should the --ssl-protocol=tlsv1
flag be passed by default in order to support older versions of PhantomJS? The alternative would be to require a newer PhantomJS binary (>= 1.9.8
) when the server doesn't support SSLv3.
If there's a redirect in the page load, PhantomJS will hang forever (it seems). This may be related to JavaScript redirects but I've experienced with server redirects too.
Re-investigate using https://github.com/Medium/phantomjs to install PhantomJS with pa11y. This would only be acceptable if it doesn't touch pre-existing installs and can run in CI.
Occasionally (rarely) PhantomJS can hang which causes the command to just sit there doing nothing.
If we can't solve the root issue, it may be worth implementing a simple timeout to prevent this from clogging up CI servers etc. If a timeout is added, make sure it can be specified with an option like this:
-t, --timeout <ms>
Possibly related to #2.
trying this
pa11y -c "http://raw.githubusercontent.com/bryanrasmussen/HTML_CodeSniffer/master/HTMLCS.js" www.sundhed.dk
or this
pa11y -c "https://raw.githubusercontent.com/bryanrasmussen/HTML_CodeSniffer/master/HTMLCS.js" www.sundhed.dk
always gives me
Loading page...
Running HTML CodeSniffer...
Error: PhantomJS timeout
another thing is I want to pass a lot more info from htmlcs, for example the html that triggered the warning, an xpath to the element etc.
If I want to the above should I just make sure that it comes out in the message.msg, and then that is adequate ( as I looks like from the code) or will I have to do some adaption of pa11y itself? (if so I would like to know how to compile)
This is the last un-(unit)tested part of the code base β mainly because it's quite large and difficult to test in its current state. Look at breaking this apart and making more testable.
This is a placeholder issue to prevent the milestone from accidentally closing.
Make a JavaScript API for pa11y which is consistent, stable and documented. This will allow for tools to be built on top of pa11y's core (e.g. #16), as well as enable pa11y to be easily integrated into other tools such as Grunt.
This may mean moving some of the more command-line-specific stuff out of lib/pa11y
. Off the top of my head, we'd need to reconsider:
process.cwd
in path resolution (for config files)console.log
/console.error
in reporters. Either that, or reporters should be exclusive to the command-line tool and pa11y should just deal in JavaScript objectsPa11y 2.0 is currently in development, and viewable on the 2.x branch for now. Issues will appear in the 2.0
milestone, and we'll be aiming for near feature parity with pa11y 1.x.
Pa11y 2.0 is currently in an alpha state, and is not ready to replace 1.x as part of your toolchain. This is mainly because not all of the accessibility rules present in 1.x have been built yet. Also, things may (and are likely to) change before we reach Beta. You can install the alpha version with:
npm install -g https://github.com/nature/pa11y/tarball/2.x
For contributors: feedback is hugely appreciated, and no change to the current 2.0 API/code is too big to consider.
The major changes in this version are as follows:
Hey @whymarrh, I've been working on splitting out the PhantomJS part of pa11y into a separate library, which is now open-sourced at https://github.com/nature/truffler. Before I start using this, can you envisage any issues? Is there anything I've missed?
There's no rush on this, but it'd be excellent if you could cast your eyes over it at some point.
Maybe tackle this by allowing inclusion of a JS file in the page as part of the program running.
Hi,
Thanks for making pa11y. We're trying to use it for our project but since we have machines with no internet connectivity we have to jump through some hoops to make HTMLCS.js available to download via HTTP.
Is there a reason you're not just including this js file in your code or is there a reason it can't be fetched from a local file system?
Happy to try and make a fix but wanted to understand if there were some reasons for this not being possible.
Thanks,
Tom
phantom stderr: 2013-12-18 17:01:57.429 phantomjs[8694:507] *** WARNING: Method userSpaceScaleFactor in class NSView is deprecated on 10.7 and later. It should not be used in new applications. Use convertRectToBacking: instead.
See #1, #2 and this blog post for more context.
Allow pa11y to use a default config file <cwd>/.pa11yrc
.
The .pa11yrc
config file should be used if present, and if a config file hasn't been specified with the --config
command-line option.
A first pass of pa11y 2.0 has been built, and is viewable on the 2.x branch for now. It's had a couple of big API changes, and the code itself is a ground-up rewrite.
I decided to stick with PhantomJS and HTML CodeSniffer for this release, if we ever move away from them it'll be much further down the line. The core library has been split into a few smaller modules and is a lot cleaner and easier to understand than 1.0; also a lot of attention has been given to the documentation.
I'd like to get feedback on the API changes which are significant (hence the 2.0 bump) before this gets released so that we can make changes if needed. @dotcode, @whymarrh, @stephenmathieson β it'd be much appreciated if you had time to review (the API more than the code itself).
The 2.x README is pretty comprehensive, and I've also written a Migration Guide to highlight the changes and help people make the switch.
Thank you
Greetings team pa11y! @adelevie introduce me to the awesomeness that is pa11y last week, and since then, we've set out to make an automated 508 testing tool for Jekyll and other static sites. We've lovingly called the Ruby gem Ra11y.
Before we go too far down the π° hole, we wanted to check in with you and make sure that we were honoring both the project's vision and the spirit of the GPL.
Pa11y is not distributed with Ra11y, instead, the Readme instructs users on how to install Pa11y via NPM. Ra11y shells out to pa11y with default arguments, and then parses the JSON returned via stdout. The executable is even configurable if the user would like to use a different command-line testing tool.
From The GPL FAQ:
By contrast, pipes, sockets and command-line arguments are communication mechanisms normally used between two separate programs. So when they are used for communication, the modules normally are separate programs.
To me, it sounds like the letter of the law is clear, but because we'll likely want to contribute improvements upstream, wanted to check in with the project maintainers personally before we were too far along.
Edit: To answer the most obvious question ("why not just use pa11y?"), Ra11y serves as an test abstraction, allowing you run to run accessibility tests against an entire directory of static files locally, aggregating the results.
The first time you run grunt from within pa11y, you get this:
$ grunt
β¦
Running "mochaTest:functional" (mochaTest) task
pa11y --help
β should be successful
β should output usage information
pa11y --version
β should be successful
β should output the current version
pa11y --config ./config.json http://localhost:4117/normal
1) "before all" hook
4 passing (2s)
1 failing
1) pa11y --config ./config.json http://localhost:4117/normal "before
all" hook:
Error: timeout of 2000ms exceeded
at null.<anonymous>
(/Users/jrobinson/repositories/git/pa11y/node_modules/grunt-mocha-test/node
_modules/mocha/lib/runnable.js:165:14)
at Timer.listOnTimeout [as ontimeout] (timers.js:110:15)
Warning: Task "mochaTest:functional" failed. Use --force to continue.
Aborted due to warnings.
Tests pass if run again.
For 2.0, it'd perhaps be good to ignore SSL errors by default. See #64. This should be much easier to do in the newer code-base.
The following list is taken roughly from HTML CodeSniffer's implemented rules. We should aim to implement tests for all of these in pa11y 2.0 as a baseline.
I've excluded notices for now to focus on actionable items.
alt
attribute on img
(error)alt
attribute on img
inside an a
with no other content (error)alt
attribute on img
but title
attribute is set (error)alt
attribute on input type="image"
(error)alt
attribute on area
(error)todo: continue adding these, also get unique codes for each
HTML Code Sniffer allows for files on file-system to be used as the source HTML.
This is very useful when you need to analyse many HTML files and want to break the process into 2 distinct phases:
Hello, I am trying to integrate Pa11y as part of Build process. However on trying to perform site validation seeing following error.
/pa11y WCAG2AA https://testsiteurl
_-::> Loading page...
_-::> URL could not be loaded
It appears my Local test environment does not have valid SSL Certificate.
for eg: Curl https://testsiteurl throws following message
curl: (60) SSL certificate problem: Invalid certificate chain
Could you please advise, if there is any way to ignore ssl certificate warning, when validating urls with Pa11y.
for eg: Curl -k testsiteurl , using -k we can ignore ssl warning with Curl
Thanks,
Raja
I feel a bit guilty about this one, since I introduced it in #25 and #32.
I still think the process should exit with an error to indicate that there were errors (as per #14), but using the actual count as the code will break if there are too many errors (more than 255). Also, using -1 wraps around, and while it is still possible to get it back, the process is a hassle (not to mention that if there are exactly 255 errors, the results count will be indistinguishable from a pa11y internal error).
Thoughts on this?
Great tool, very good implementation of HTML Code Sniffer!
For CI integration, it would be very useful to be able to specify the reporting level, eg.
0 - error & warning & notice
1 - error & warning
2 - error
With 0 being the default.
In our case, this would allow CI testing to halt a build if an error
is found in the WCAG standard but allow the build to continue if only warnings and notices are present.
I am using pa11y for accessibility audit on my web application and it generate a report which shows following details -
But i am not able to get details like where any particular accessibility error/warning exists in my html.
May be i am not able to understand.
so, please help me on this or let me know if there is any option to customize reports to get complete details to resolve issues in html.
Not exactly sure how we'd tackle yet. Maybe allow user to specify a JSON config file or pass rule names in as CLI flags.
Just noticed that pa11y doesn't follow the GPL guidelines for using the license.
Quoting from http://www.gnu.org/licenses/gpl-howto.html (emphasis is mine)...
"...the process involves adding two elements to each source file of your program: a copyright notice... and a statement of copying permission, saying that the program is distributed under the terms of the GNU General Public License..."
"The copying permission statement should come right after the copyright notices.
...
For programs that are more than one file, it is better to replace βthis programβ with the name of the program, and begin the statement with a line saying βThis file is part of NAMEβ. For instance,
This file is part of Foobar.
Foobar is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
Foobar is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with Foobar. If not, see <http://www.gnu.org/licenses/>.
This statement should go near the beginning of every source file, close to the copyright notices."
"For interactive programs, it is usually a good idea to make the program print out a brief notice about copyright and copying permission when it starts up."
"You should also include a copy of the license itself somewhere in the distribution of your program. All programs, whether they are released under the GPL or LGPL, should include the text version of the GPL. In GNU programs the license is usually in a file called COPYING."
Hello,
I cannot launch the tool because i get an error about the path of the phantomjs. However, phantomjs path is on the system environment variables, and like you can see in the image, I can run the commands of phantomjs like for example (phantomjs --version) to check if phantomjs is installed.
Do you know what could be wrong?
Thank you,
First of all, thank you for such a wonderful project. We have been using it quite extensively and have thus found some problems with it. The most problematic of which is that it spawns zombie processes that consume memory and cpu and eventually crash the server.
Some details:
If you run pa11y on a page that generates an error, p11y eventually reaches its timeout but at least two processes live on (node and phantomjs).
To reproduce the error I have been able to do the following:
pa11y -r json http://prairieschooner.unl.edu/?q=fusion%2Fwork%2Fbus-driver
and wait for it to errorps x -o "%p %r %y %x %c "
or look at the output of ps x
and you should see two zombies processes (one for node
and one for phantomjs
). Sometimes you will see one or the other.We are using SiteMaster to scan our sites with several metrics, including accessibility via pa11y. In order to get around this problem, I implemented our own custom timeout for the pa11y process, which recursively kills child processes. See: UNLSiteMaster/metric_pa11y@d478bc8
I'm not sure where exactly this problem lives. It might actually be a bug in phantomjs. I just thought I'd give you a heads up.
sieben@sieben-lincs:~|β pa11y nature.com
Welcome to Pa11y
We'll sniff your page for you now.
Loading page...
events.js:72
throw er; // Unhandled 'error' event
^
Error: spawn ENOENT
at errnoException (child_process.js:980:11)
at Process.ChildProcess._handle.onexit (child_process.js:771:34)
i know it's a huge feature request, but spidering support would be extremely helpful.
it'd be nice to also be able to specify a maximum depth of spidered pages:
$ pa11y http://website.org/ --depth 15
thoughts? i'd be happy to contribute, or create a separate module (pa11y-spider
?) for such a task.
Hello!
I got a generic "Code Sniffer Error" when I launch the following command:
pa11y scuolafalconeborsellino.it
Also when use the flag -d, I can't understand what went wrong:
Welcome to Pa11y
We'll sniff your page for you now.Debug: Starting timeout timer (30000ms)
Loading page...
Running HTML CodeSniffer...
Error: HTML CodeSniffer error
What to do to better understand the problem? I suppose this is caused by the frame included in the webpage...
Thank you very much!
dependencies such as _
are a bit large for what they're being used for and may become deeply coupled with this module. how would you feel about removing the underscore dependency and relying on smaller, cleaner modules for underscore-esque functionality (_.defaults
, _.countBy
, etc.)?
This probably involves the joyous task of trawling through HTML CodeSniffer and grabbing each of the rules (code plus message).
At the moment you have to copy rules from the console output which is quite inefficient, and doesn't allow you to ignore rules that you haven't come across yet.
Hello,
For some website, the page tested by pa11y is not the same as the one displayed in a browser (maybe cause of the user-agent ...). It would be nice to have an option to get a backup of the source page code and a screenshot (phantomjs) of the page tested by pa11y to be sure this is the good one.
Investigate setting a unique user agent to allow sites to filter out pa11y traffic from analytics etc.
This would be useful if including pa11y as part of a Git pre-commit hook or in a CI environment.
It'd only be when errors reported, not for warnings or notices.
I've been seeing errors where pa11y fails to open URLs due to failed ssl handshakes. The solution is to change the ssl protocol that phantomjs uses via this command line option:
--ssl-protocol=any
I'm sure other issues could crop up where passing user defined configuration to phantomjs would be helpful, so maybe it would be better to implement phantomjs's --config
option:
--config=/path/to/config.json
Sources:
It looks like pa11y currently renders and checks pages at a mobile width. This is useful, and goes with a mobile first attitude, however it ignore other potential breakpoints.
Especially when considering responsive design, the layout, colors, content, etc can change at different breakpoints, which can affect accessibility.
So, it would be nice if there was a command line option to configure the viewport size. Ideally, you could add several breakpoints and pa11y would run the test suite against each breakpoint, and then return the breakpoints for each error that was found.
As rules/suites are now customisable and extendable, it would be useful if the command-line tool could list all of the ones present. Thinking pa11y -l
.
Not sure of the feasibility of this but it would be great to be able to run pa11y against a fragment of HTML as well testing full pages.
The way we pass data to PhantomJS at the moment is awful. We're passing the 'standard' to use through the querystring.
There's almost definitely a nicer way.
I couldn't work out how to pass data into PhantomJS any other way using our chosen Node library. I may just be being blind.
I have installed NodeJS 0.10 but i could not install pa11y tool. I also have installed Windows SDK and VisualStudio.
However, when try to install pa11y tool "$ npm install -g pa11y", i got this error and the tool fails to install.
I see that the "weak 2.2" script and package fail to install, but i don't know why.
See discussion in #61. This could be done quite easily in pa11y 2.0, you'd need to get each element's XPath in lib/inject (which run's within PhantomJS). You could use one of the functions in these SO answers:
These were thrown together as a basic safety net before we had a decent test suite. There's now a lot of repetition and most of the JS code in test/feature
is a mess.
Idea: adding the ability to configure all the same options in the JSON config (.pa11yrc
) that are available as command line flags.
I'm not 100% sure where I would draw the line on this as not all of the command line flags are worthy (e.g. help and version), but I think the others might be useful.
Thoughts? With ideas like #50/#51 and #53, maybe the JSON config is underutilised.
Hello,
Hopefully I'm just doing something wrong but I can't get the ignore feature to work. Here is my code:
pa11y.sniff({
url: url,
standard: accessibilityStandard,
useragent: 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.104 Safari/537.36',
config: {
ignore: [
'WCAG2AA.Principle1.Guideline1_4.1_4_3.G18'
]
},
}, function(err, results) {
// Do something
});
Whenever I run this command I get nothing but WCAG2AA.Principle1.Guideline1_4.1_4_3.G18 errors (because I've eliminated everything else).
Is there something wrong in my code or is there a defect?
This is a placeholder issue to prevent the milestone from accidentally closing.
This will allow you to test local URLs with no internet β not currently possible.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.