fpapado / accessible-pipeline Goto Github PK
View Code? Open in Web Editor NEWAccessibility monitoring, made easy
License: Other
Accessibility monitoring, made easy
License: Other
The current runCore function buffers all the results in memory, until everything is done.
Only at the end, does it actually give the results to the caller.
This is fine for small workloads, but it can be prohibitive for larger ones, especially given the large size of AxE results.
Another issue with buffering is that streaming workloads (i.e. the CLI "reporter" API) need some hard-coded support for reporting progress. You can see this in the current implementation as a custom pino logger. While I like the current setup (unix-style piping is cool!), I think we could unify this use-case with a streaming API at the runCore level.
The current runCore API (that buffers) can be built on top of a streaming one, by collecting all of the results, and flushing at the end.
Something along these lines could work:
runCore
to runCoreStreaming
, that outputs a stream, or async iterator. Instead of appending to results and returning, yield each result.runCore
function, that calls runCoreStreaming
, and collects the results. Return those results once the Observable is done.runCore
to be runCoreStreaming
. Rename the return-based runCore
to runCoreBuffered
, or so. Considered; will stick with runCore
for now.This mostly concerns the CLI use case:
Some sites / environments use HTTP Basic Auth to authenticate users. This might be necessary for CI tests.
Puppeteer seems to have page.authenticate for this.
The default page limit is set to 20
, to avoid accidentally bombing a site. However, this is not surfaced adequately, and can confuse the user, who sees only a few pages crawled.
I think two things could be done:
Document the JSON format, and that it has the types from AxeResults.
Give ideas about alternative reporters, site generators etc.
At the moment, we retry on error up to --maxRetries
. Then, if the page still fails, we skip it, and remove it from the "to visit" list. This is fine for not crashing the process, but can be opaque for the users.
We should have a way to surface these errors:
streamingSendErrorProcessing
to the reporter APIreport.json
. Ideally, I'd like to keep the report as close to AxeResults
as possible. We could make it so that the type is Result = Ok AxeResult | Err Error
. It has the benefit of forcing users to account for error cases in their processors, but we might need something different.Microsoft has tsdoc, which allows extracting comments from the source. It is built with TypeScript in mind.
I'd love a future where we can pull out the documentation and put it into a markdown file.
build-docs
step in the package.json
Spotted in live use by a coworker:
A bunch of
We should write tests for this use case.
We'd start by setting up fixtures under test/fixtures/ignoring-extensions
.
We'd have the following files as fixtures:
index links to all the other files
Then, we should verify that:
runCore
only crawls index.html, about.html, image.jpgrunCore
only crawls index.html and about.htmlPerhaps we should also verify that:
This is similar to the to the option to skip fragment links
Often in websites, a bunch of "equivalent" URLs use query parameters, either for cache-busting, or for toggling functionality. Those could be skippable via --ignoreQueryParams
.
test/fixtures/skip-query-parameters
.runCore
, under Options
. Implement this check in the decision-making parts ("should visit page" etc.).cli
, similarly to Options
.This list seems long, so we could take them one at a time! If you need help setting up the test cases, I'd love to help :)
Set up CI to make documentation every time the branch is about to merge. Don't know how exactly it should be done, maybe it is possible to plug in "Docs generation" commit pre-merge to master. Up for a discussion.
This came up when talking with James.
Some of the options are good to have, like skipFragmentLinks
and ignoreExtensions
. They help cut down a lot of duplicate (or irrelevant) links, which makes runs faster, and eases the load on the site.
Ideally, I'd like to have a section called "Best practices", with a section on "Tuning the pages visited":
state.json
to see the queueThe aim of the module is to run in CI. We should consider how to document that.
Things like:
--ci
flag in the CLIrunCore
optionsAs a more contained example, we could have a guide with 1. through 4., leaving 5. and 6. as future work.
Back when I started this project, I could not find a way to query the DOM from Puppeteer. Thus, in order to run selectors (to find the anchors), I used cheerio. This needs more data to load, another parser etc. It seems brittle, and I think we can move away from it.
running "accessible-pipeline run https://example.com" results in output
Error: spawn ./dist/cli.js ENOENT
at Process.ChildProcess._handle.onexit (internal/child_process.js:264:19)
at onErrorNT (internal/child_process.js:456:16)
at processTicksAndRejections (internal/process/task_queues.js:80:21)
Emitted 'error' event on ChildProcess instance at:
at Process.ChildProcess._handle.onexit (internal/child_process.js:270:12)
at onErrorNT (internal/child_process.js:456:16)
at processTicksAndRejections (internal/process/task_queues.js:80:21) {
errno: 'ENOENT',
code: 'ENOENT',
syscall: 'spawn ./dist/cli.js',
path: './dist/cli.js',
spawnargs: [ 'run', 'https://example.com', '--ci', '--streaming' ]
}
Some flows might need the user to be logged in (via a Cookie, typically?) to get to the page we need to test.
It would be nice to have the option to do that!
I am not 100% settled on how we will achieve this.
My first thought was to add an option to read a cookie from a file.
However, that seems overly specific.
Another option would be having users to pass in a Puppeteer Page
object by themselves. They could log in before hand, using selectors.
A middle ground would be to add an onBeforeAssert
hook, that lets the user run scripts with the Page
context. Would that be better? Worse? Horrible? Let's find out! :)
BackstopJS has a setting cookies option, that allows loading cookies from a file. Maybe that would work as well?
There are some refinements to be done after #28 is merged.
Namely:
/typedocTheme/
.This is quite abstract "Make docs look less like crap" issue so I guess any improvements are welcome, the 2 points I mentioned is just the ones I've noticed myself during setting up the docs.
A few things came up during our Helsinki Hacktoberfest workshop, that I'd like to work on myself.
They are mostly about the first-time usage, as well as the development loop.
In rough order:
run-cli
to use ts-node cli.ts
, rather than the built file. Keep production: run-cli
production: run-cli
requires npm run build
, and chmod +x dist/cli.js
It might be nice to screenshot a DOM node for people to investigate issues.
However, I'm not sure how much complexity this adds!
Puppeteer offers this as an API.
The main question revolves around how the user stores these.
Since the main results are stored in JSON, it is not possible to store the image binary (unless we base64 it, which would be huge).
Thus, an alternative would be to force the user to provide us with a callback, to let them do whatever they want with the screenshot.
Something like this:
Option
like screenshotFailingNodes: (screenshot: Buffer) => void
Something that we mentioned in the Helsinki Hacktoberfest meetup.
There are different use-cases for different projects.
For example, when would you run this on CI? Would you have a hard pass/fail?
If a site has a lot of issues, perhaps it's best to ship metrics to some other service, to track progress?
I'm sure there's other scenarios that I'm missing!
AxE logs a lot of information, even for passes. It also stores a lot of HTML information, that is probably only valuable for violations (and sometimes, not even there; the streaming reporter does not use them). This can lead to a large JSON file, which can crash the reporter when running view
.
A possible solution is to:
trimReportJSON
function, taking parameters for categories (passes, violations, etc).AxeResultsReduced
--removeReportsFor passes,pending
, with defaultsA future question is whether to remove the html
parts for the violations
key as well. They are not super useful ime, and they seem the most prone to breaking!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.