accesslint/screenreaders

Screen reader automation tools by AccessLint, including VoiceOver.js and Auto-VO

screenreaders's Issues

Discussion about the API

Hello there and thank you for the amazing work you've done 🙏🏻 .

I'm passing around and wanted to ask if you had something specific in mind to improve a bit the API and eventually decouple Playwright from the module? Or is it a strong requirement that you have and that you want to stick with it?

Willing to help in any way, if needed 😊!

required system privacy and automation permissions are undocumented

Make sure that all of the System permissions that are required are documented in the README.

see #6 (comment) by @SebDuf

prompt for playwright chromium installation when running CLI

When the user runs the CLI, and they don't have playwright-chromium installed, prompt the user to add the module.

stopVoiceover.js: execution error: Error: Error: osascript is not allowed to send keystrokes. (1002)

This error occurs when launching the script without first enabling AppleScript in VoiceOver Utility. To fix this, enable AppleScript in VoiceOver Utility, and enable Terminal.app (or iTerm etc) to use macOS accessibility APIs.

A way to read inside content groups?

Is there a way to get this tool to dig a little bit deeper into content groups?

Compared to my own manual VoiceOver testing, this tool seems to give me different more "high level" output. It seems like it skips over the page content and just reads out the higher level "Content - group". Is there a way to get it to read into groups?

Here's this site I'm testing: https://kamadojoe.com

And this is the output I'm seeing:

Welcome to macOS. VoiceOver is on. Playwright 🎭 Playwright: about:blank window web content has keyboard focus
Kamado Joe® | Ceramic Grills | Ceramic Smoker | Kamado Grill
Links menu
Headings menu
Form Controls menu
Window Spots menu
Loading more items…
No items in Window Spots menu
Content - group
Kamado Joe® | Ceramic Grills | Ceramic Smoker | Kamado Grill
In Kamado Joe® | Ceramic Grills | Ceramic Smoker | Kamado Grill KamadoJoe's Instagram feed
Session ended

Maybe this is how the tool is expected to behave. Was trying to see if this cool could speed up my workflow of checking what VoiceOver says for content in the page.

script paths are incorrect for global CLI

running the CLI returns:

/bin/sh: ./osascripts/stopVoiceover.js: No such file or directory

[related to #6]

Invitation to web standards meeting about a Screen Reader Automation API on 28th March 2022

Hi @ckundo, @stephenmathieson,

I came across auto-vo on GitHub while looking for projects that automate screen readers I’m part of a web standards group called ARIA-AT that is also looking into automating screen readers, for the purpose of conformance testing.

We're trying to standardize a new API to programmatically control screen readers that we hope will be implemented by vendors to unlock new functionality and to more reliably test the behavior of screen readers.

We’d like to include the input of tooling authors like you. If you're interested in participating in this effort, you're very welcome to attend the next ARIA-AT Automation meeting on February 28 at 12pm PT / 3pm ET. See details here:

w3c/at-driver#17

Thank you!

browser opens and closes immediately with no output

When I run auto-vo --url https://smashingmagazine.com --limit 5 - a chrome page opens with smashingmagazine.
"Welcome to MacOS, Link, VoiceOver off".`
The chrome page closed automatically, and I'm back at the CLI with no output.

see #6 (comment)

Question: Can I record VoiceOver as text for comparison?

Hi, I'm trying to find a tool that I can use with react testing library to create a test for screenreader. I was wondering if VoiceOver can also return the readout as text for comparison.

"Cannot find module '../cli.js'" when following README setup for CLI

Having followed the configuration instructions (System Preferences, Voiceover Utility) and installed auto-vo with Node.js 16, I get an error trying to run the example command (--url https://example.com) for proof-of-life.

Same problem with Node.js 14...

Terminal session transcript:

sebastian@Sebastians-Onlea-Macbook-Pro / % sw_vers && npm -v && node -v                                     
ProductName:	macOS
ProductVersion:	12.1
BuildVersion:	21C52
8.3.2
v16.13.0
sebastian@Sebastians-Onlea-Macbook-Pro / % npm install -g auto-vo                                           

added 54 packages, and audited 55 packages in 947ms

4 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
sebastian@Sebastians-Onlea-Macbook-Pro / % npx auto-vo --url https://example.com --limit 5 --until 'Example'
node:internal/modules/cjs/loader:936
  throw err;
  ^

Error: Cannot find module '../cli.js'
Require stack:
- /Users/sebastian/.nvm/versions/node/v16.13.0/lib/node_modules/auto-vo/bin/auto-vo.js
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)
    at Function.Module._load (node:internal/modules/cjs/loader:778:27)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/Users/sebastian/.nvm/versions/node/v16.13.0/lib/node_modules/auto-vo/bin/auto-vo.js:3:17)
    at Module._compile (node:internal/modules/cjs/loader:1101:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/Users/sebastian/.nvm/versions/node/v16.13.0/lib/node_modules/auto-vo/bin/auto-vo.js'
  ]
}

use voiceover.js imports

i extracted the VoiceOver interface to a separate package, voiceover, that can replace the included files.

cli should only install playwright chromium

current version installs FF and webkit too...

% npx auto-vo https://example.com
Downloading Playwright build of chromium v888113 - 116.3 Mb [====================] 100% 0.0s 
Downloading Playwright build of firefox v1271 - 74.8 Mb [====================] 100% 0.0s 
Downloading Playwright build of webkit v1492 - 53.1 Mb [====================] 100% 0.0s 
Downloading Playwright build of ffmpeg v1005 - 1.3 Mb [====================] 100% 0.0s 

Feature Request: CLI option to wait a set time before processing

I currently have some Vaadin web apps which I would like to use this for.

The problem I'm facing is that the Application Server can take a few seconds to load (showing a loading spinner) before the main page is shown.

This results in output like this:

Loading app. Please wait...
Loading app. Please wait...
This content is announced automatically and does not need to be navigated into.
This content is announced automatically and does not need to be navigated into. group
tooltip

I'm certain if this is due to the load or maybe just a poorly structure page layout, but I know that I can use other tools manually to navigate and use voice over once the page is loaded.

Page closed error trying to run

Trying to set this up and get the following error.

Is there something I'm missing here?

(Path)/lib/node_modules/auto-vo/node_modules/playwright/lib/server/chromium/crPage.js:341
            this._firstNonInitialNavigationCommittedReject(new Error('Page closed'));
                                                           ^

Error: Page closed
    at CRSession.<anonymous> (Path)/lib/node_modules/auto-vo/node_modules/playwright/lib/server/chromium/crPage.js:341:60)
    at Object.onceWrapper (node:events:513:28)
    at CRSession.emit (node:events:406:35)
    at (Path)/lib/node_modules/auto-vo/node_modules/playwright/lib/server/chromium/crConnection.js:177:43

VoiceOver gets stuck on the browser content "chrome" and doesn't enter the webpage

Issue getting stuck on the page title

What happens

When I run auto-vo it never moves the VO cursor into the actual webpage. VO gets stuck reading out the page title repeatedly since it stays on the element containing the actual webpage and doesn't enter the page itself.

What should happen

VO should enter the webpage content and start navigating the page.

Reproduction

I had to fix some script paths to test this, but once those were fixed it seems to happen for me every time I run auto-vo --url https://smashingmagazine.com --limit 5 (limit has to be at least five to see the actual repetition happen).

How can this be fixed?

I don't know if this is a configuration in my VO setup that needs to be changed (I don't remember if I am running the standard configuration or if I tweaked a few things over the years). When I use VO manually I can't just keep pressing VO Right and end up in the website content. I have to press VO Shift Down to actually enter the content.

If this happens with a standard VO configuration

Is there a setting that needs to be documented aside from the "Allow VoiceOver to be controlled with AppleScript"? If so, what is it?

If this doesn't happen with a standard VO configuration

Is this something that can be addressed in auto-vo, so multiple configurations can be handled automatically, or would the documentation have to include a set of "supported" settings?

Did I just completely miss something

A very likely scenario - is this a "have you tried turning it off and on again"-thing on my end? What ever it is, I like the general idea of auto-vo, and look forward to taking it for a spin :)

Auto Vo has stopped working, and reinstalling not working

So, I had this working perfectly well on two different computers. Today I am getting this error:

npm ERR! syscall chmod
npm ERR! path /Users/username/.npm/_npx/12493/lib/node_modules/auto-vo/bin/voiceover.js
npm ERR! errno -2
npm ERR! enoent ENOENT: no such file or directory, chmod '/Users/username/.npm/_npx/12493/lib/node_modules/auto-vo/bin/voiceover.js'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent

I then tried reinstalling in terminal with:

npm install -g auto-vo

And this errors too:

/usr/local/bin/auto-vo -> /usr/local/lib/node_modules/auto-vo/bin/auto-vo.js
npm WARN rollback Rolling back [email protected] failed (this is probably harmless): EINVAL: invalid argument, rmdir '/usr/local/lib/node_modules/auto-vo/node_modules/playwright-webkit/node_modules/playwright-core/lib/third_party/highlightjs/highlightjs'
npm ERR! code ENOENT
npm ERR! syscall chmod
npm ERR! path /usr/local/lib/node_modules/auto-vo/bin/voiceover.js
npm ERR! errno -2
npm ERR! enoent ENOENT: no such file or directory, chmod '/usr/local/lib/node_modules/auto-vo/bin/voiceover.js'
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
npm ERR! A complete log of this run can be found in:
npm ERR! /Users/username/.npm/_logs/2022-01-30T21_27_20_909Z-debug.log

Any idea what is happening and how I resolve it?

Can the Auto Vo output be scoped in any way?

A feature idea - have the ability to specify a node within which to keep VO in. Not sure if there might be any way to do this. Presumably it'd involve something like setting focus on starting element (JS?) and then reading from there. Is there conceivably a way to say when to stop?

Anyway, just an idea to throw out there in case there is a way to do it. :)

Feature Request: To be able to get behind http authentication

In testing auto-vo to see if it would work for me, I found that when it goes to a URL guarded by HTTP auth it just closes the chrome window. Would it be possible to add the ability to get past http auth? Maybe if there is a way to pause the auto scan of the page or to be able to add user and password attributes to the CLI?

All of my dev and testing sites are behind HTTP auth and it would be nice to use this on those before accessibility changes are made live.

Thanks!