[Nvda-dev] NVDA and Chrome

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

[Nvda-dev] NVDA and Chrome

Dominic Mazzoni
Hello NVDA developers,

Greetings from some of the members of Google's accessibility team.  As Jamie already discovered yesterday (http://code.google.com/p/chromium/issues/detail?id=47084), we've been making great progress in making Google Chrome accessible, and we would love to work with you to make sure that it works great with NVDA.

Here's a quick summary of where we're at: as you may know, one of the big challenges with Chrome has been its multi-process architecture - the accessibility requests were all coming into Chrome's main browser process, but the WebKit DOM was stored in a separate sandboxed renderer process. We've finally come up with a solution to that issue by caching the accessible tree in the browser process and keeping them in sync, and now Chrome can respond to MSAA requests quickly and accurately.

We've been using Firefox as a model when deciding how to expose various HTML DOM elements via MSAA / IAccessible2. There are a lot of minor differences we still need to fix, but most tags are already exposed correctly now. At some point we may choose to deviate, but we thought this would be the easiest way for screenreaders to add support for Chrome without writing a new module from scratch.

Still unfinished, but our next highest priorities:
* Support dynamic content (right now only static pages work properly)
* Implement IAccessibleText for editable text boxes

We love NVDA. While our goal is to support all of the major screen readers, NVDA has been extremely helpful for development and debugging, and because of its open-source nature it's going to be one of the first screenreaders to work with Chrome. I'm quite impressed with how powerful and capable NVDA has become, and I love that it's mostly in Python, it makes the code concise and easy to experiment with.

I've attached a proposed patch to make NVDA recognize Chrome. To maximize code reuse it subclasses the mozilla/gecko virtual buffer class.

- Dominic


nvda-chrome.patch (8K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [Nvda-dev] NVDA and Chrome

Geoff Shang
Hello Dominic,

Due to the time of day in Australia, Jamie and Mick probably won't be able
to respond for several hours.  But I'm sure one of them definitely will.

In the meantime, Jamie seems also to have posted an experimental ap module
for Chrome.

http://www.nvda-project.org/test/chrome.py

See the notes at the top of the file for usage instructions.

Geoff.



Reply | Threaded
Open this post in threaded view
|

Re: [Nvda-dev] NVDA and Chrome

chris hallsworth
In reply to this post by Dominic Mazzoni
Now that is so cool! I have a sighted friend and she insists on Chrome,
as she says it's faster than even Firefox. So this is great that
eventually we NVDA users can use Chrome.


Signed by Chris Hallsworth
E-mail and Facebook: [hidden email]
MSN: [hidden email]
Skype: chrishallsworth7266
Twitter: http://twitter.com/christopherh40

On 22/06/2010 16:07, Dominic Mazzoni wrote:

> Hello NVDA developers,
>
> Greetings from some of the members of Google's accessibility team.  As Jamie
> already discovered yesterday (
> http://code.google.com/p/chromium/issues/detail?id=47084), we've been making
> great progress in making Google Chrome accessible, and we would love to work
> with you to make sure that it works great with NVDA.
>
> Here's a quick summary of where we're at: as you may know, one of the big
> challenges with Chrome has been its multi-process architecture - the
> accessibility requests were all coming into Chrome's main browser process,
> but the WebKit DOM was stored in a separate sandboxed renderer process.
> We've finally come up with a solution to that issue by caching the
> accessible tree in the browser process and keeping them in sync, and now
> Chrome can respond to MSAA requests quickly and accurately.
>
> We've been using Firefox as a model when deciding how to expose various HTML
> DOM elements via MSAA / IAccessible2. There are a lot of minor differences
> we still need to fix, but most tags are already exposed correctly now. At
> some point we may choose to deviate, but we thought this would be the
> easiest way for screenreaders to add support for Chrome without writing a
> new module from scratch.
>
> Still unfinished, but our next highest priorities:
> * Support dynamic content (right now only static pages work properly)
> * Implement IAccessibleText for editable text boxes
>
> We love NVDA. While our goal is to support all of the major screen readers,
> NVDA has been extremely helpful for development and debugging, and because
> of its open-source nature it's going to be one of the first screenreaders to
> work with Chrome. I'm quite impressed with how powerful and capable NVDA has
> become, and I love that it's mostly in Python, it makes the code concise and
> easy to experiment with.
>
> I've attached a proposed patch to make NVDA recognize Chrome. To maximize
> code reuse it subclasses the mozilla/gecko virtual buffer class.
>
> - Dominic
>
>
>
>
> _______________________________________________
> Nvda-dev mailing list
> [hidden email]
> http://lists.nvaccess.org/listinfo/nvda-dev


Reply | Threaded
Open this post in threaded view
|

Re: [Nvda-dev] NVDA and Chrome

Dominic Mazzoni
In reply to this post by Geoff Shang
On Tue, Jun 22, 2010 at 8:54 AM, Geoff Shang <[hidden email]> wrote:
Hello Dominic,

Due to the time of day in Australia, Jamie and Mick probably won't be able to respond for several hours.  But I'm sure one of them definitely will.

In the meantime, Jamie seems also to have posted an experimental ap module for Chrome.

http://www.nvda-project.org/test/chrome.py

See the notes at the top of the file for usage instructions.

Nice! It looks like we're basically on the same page, using the Gecko / Mozilla virtual buffer for now until there's a good reason to have a separate module. We're hopefully submitting the fix to the focus notification today, so that workaround won't be necessary shortly.

- Dominic
 
Geoff.


_______________________________________________
Nvda-dev mailing list
[hidden email]
http://lists.nvaccess.org/listinfo/nvda-dev

Reply | Threaded
Open this post in threaded view
|

Re: [Nvda-dev] NVDA and Chrome

Jake Joehl
In reply to this post by Dominic Mazzoni
This is a step in the right direction, both for NVDA and for Google. I
can't wait to try this out. I might be getting a laptop at some point
in the near future for one of my jobs, and I'm going to install NVDA on
it. I don't know much if anything about the development side of these
things, but I too, love NVDA and Google!
Jake

--
Email services provided by the System Access Mobile Network.  Visit
www.serotek.com to learn more about accessibility anywhere.


Reply | Threaded
Open this post in threaded view
|

Re: [Nvda-dev] NVDA and Chrome

Tamas Geczy
This is great news! Google, thank you for once again recognizing the
need for accessibility and attempting your best at implementing it.

NVDA, in my opinion, was a great choice for this, because out of all
the other screen readers it is the one which has made it the farthest
in development when looking at it from the point that 4 years ago, the
project was just starting. Both Jamie and Mick have put tremendous
effort into it and so have others in the community who helped advance
it to where it is today.

Now all we need is Google voice to be accessible too :)

All the best,
Tomi


On 6/22/10, Jake Joehl <[hidden email]> wrote:

> This is a step in the right direction, both for NVDA and for Google. I
> can't wait to try this out. I might be getting a laptop at some point
> in the near future for one of my jobs, and I'm going to install NVDA on
> it. I don't know much if anything about the development side of these
> things, but I too, love NVDA and Google!
> Jake
>
> --
> Email services provided by the System Access Mobile Network.  Visit
> www.serotek.com to learn more about accessibility anywhere.
>
> _______________________________________________
> Nvda-dev mailing list
> [hidden email]
> http://lists.nvaccess.org/listinfo/nvda-dev
>


Reply | Threaded
Open this post in threaded view
|

Re: [Nvda-dev] NVDA and Chrome

johnnybrl
In reply to this post by Geoff Shang
~Wow! this sounds awesome! I'll have to try it out. A lot of my friends love
using crome, in fact I think even my brother uses it more than firefox
nowadays. I'll be sure to give it a test run when it gets a little more
stable with NVDA.
----- Original Message -----
From: "Geoff Shang" <[hidden email]>
To: "News and discussion for NVDA \(NonVisual Desktop Access\),a free and
open source screen reader for Microsoft Windows"
<[hidden email]>
Cc: "Jonas Klink" <[hidden email]>; "David Tseng" <[hidden email]>;
"Christopher Guillory" <[hidden email]>
Sent: Tuesday, June 22, 2010 8:54 AM
Subject: Re: [Nvda-dev] NVDA and Chrome


> Hello Dominic,
>
> Due to the time of day in Australia, Jamie and Mick probably won't be able
> to respond for several hours.  But I'm sure one of them definitely will.
>
> In the meantime, Jamie seems also to have posted an experimental ap module
> for Chrome.
>
> http://www.nvda-project.org/test/chrome.py
>
> See the notes at the top of the file for usage instructions.
>
> Geoff.
>
>
> _______________________________________________
> Nvda-dev mailing list
> [hidden email]
> http://lists.nvaccess.org/listinfo/nvda-dev 



Reply | Threaded
Open this post in threaded view
|

Re: [Nvda-dev] NVDA and Chrome

James Teh
In reply to this post by Dominic Mazzoni
Hi Dominic,

Thanks for your email and your keen interest in having Chrome work with
NVDA. It's great to see Google Chrome making such significant progress
in accessibility. I'm sure it'll only get lots better from here! :)

Further replies inline.

On 23/06/2010 1:07 AM, Dominic Mazzoni wrote:
> We've been using Firefox as a model when deciding how to expose various
> HTML DOM elements via MSAA / IAccessible2. There are a lot of minor
> differences we still need to fix, but most tags are already exposed
> correctly now.At some point we may choose to deviate, but we thought
> this would be the easiest way for screenreaders to add support for
> Chrome without writing a new module from scratch.
This philosophy has certainly worked for NVDA. I wasn't sure whether
using our existing Gecko backend would work, as we primarily traverse
the tree using embedded objects in
IAccessibleText/IAccessibleHypertext/IAccessibleHyperlink for Gecko.
However, we do fall back to AccessibleChildren if these other methods
fail and this works for Chrome.

Note, however, that some other ATs use ISimpleDOM* interfaces to access
Firefox instead of just using IAccessible2, so they won't be able to use
their Firefox code for Chrome.

It'd certainly be great if we could continue to use the gecko_ia2
backend rather than having to maintain yet another backend.

> * Implement IAccessibleText for editable text boxes
It'd be great if this could be done for the omnibar as well as for the
renderer. Currently, our display hooks get used for the omnibar
automatically, but I think IAccessibleText would be better.

> I've attached a proposed patch to make NVDA recognize Chrome. To
> maximize code reuse it subclasses the mozilla/gecko virtual buffer class.
Thanks! As someone noted, I hacked up an app module to do this
yesterday. Eventually, this will probably be moved to the core as you
have done, although I do wonder: is Chrome designed to be embeddable
like Gecko and MSHTML or will the Chrome specific parts of the renderer
(Chrome_RenderWidgetHostHWND) only ever appear in the Chrome application
itself? (Native Webkit is a different issue entirely, as it natively
uses a different a11y implementation.) The reason I ask is that if it
isn't embeddable, it's probably better left as an app module.

Note that there is a little bit of unnecessary code that could be
removed from your patch. Some of the checks that we do for Gecko aren't
really necessary for Chrome.

I'm not sure that putting this into the main NVDA branch is a good idea
yet. While Chrome accessibility is coming along very nicely, there are
still some big gaps and I'm concerned that claiming support for Google
Chrome while these gaps exist will confuse users, especially as anything
in main will go into the next NVDA release. I'm wondering whether it is
better left as a separate app module for now, but need to think on this
some more.

I've already started watching and filing bugs regarding Chrome
accessibility. One issue that would be great to fix sooner rather than
later is the exposure of the display style of DOM nodes so that NVDA
doesn't merge content on to a single line that should be separated.

You and your team have done a fantastic job so far. Chrome a11y has come
in leaps and bounds in the last month or so. Keep up the great work!

Jamie

--
James Teh
Vice President
NV Access Inc, ABN 61773362390
Email: [hidden email]
Web site: http://www.nvaccess.org/


Reply | Threaded
Open this post in threaded view
|

Re: [Nvda-dev] NVDA and Chrome

Dominic Mazzoni
On Wed, Jun 23, 2010 at 12:44 AM, James Teh <[hidden email]> wrote:
Hi Dominic,

Thanks for your email and your keen interest in having Chrome work with NVDA. It's great to see Google Chrome making such significant progress in accessibility. I'm sure it'll only get lots better from here! :)

And thank you for your help! It's so much easier to make progress this way.
 
Note, however, that some other ATs use ISimpleDOM* interfaces to access Firefox instead of just using IAccessible2, so they won't be able to use their Firefox code for Chrome.

Or we should just implement ISimpleDOM also. It doesn't look that hard, but we would have to first patch WebKit to provide some of the details that currently aren't exposed in AccessibleObject.
 
* Implement IAccessibleText for editable text boxes
It'd be great if this could be done for the omnibar as well as for the renderer. Currently, our display hooks get used for the omnibar automatically, but I think IAccessibleText would be better.

Sure, that makes sense.
 
Thanks! As someone noted, I hacked up an app module to do this yesterday. Eventually, this will probably be moved to the core as you have done, although I do wonder: is Chrome designed to be embeddable like Gecko and MSHTML or will the Chrome specific parts of the renderer (Chrome_RenderWidgetHostHWND) only ever appear in the Chrome application itself? (Native Webkit is a different issue entirely, as it natively uses a different a11y implementation.) The reason I ask is that if it isn't embeddable, it's probably better left as an app module.

There's Chrome Frame (embedding Chrome in IE), which does expose the MSAA COM interfaces as well. Other than that I'm not aware of any efforts to make Chrome embeddable.

Note that there is a little bit of unnecessary code that could be removed from your patch. Some of the checks that we do for Gecko aren't really necessary for Chrome.

I'm not sure that putting this into the main NVDA branch is a good idea yet. While Chrome accessibility is coming along very nicely, there are still some big gaps and I'm concerned that claiming support for Google Chrome while these gaps exist will confuse users, especially as anything in main will go into the next NVDA release. I'm wondering whether it is better left as a separate app module for now, but need to think on this some more.

That's fine. Will the app module be included in nightly builds? Would it be okay if we either included instructions or a link to the app module from one of the dev.chromium.org pages? For example, from:


We could make it clear that this is still a work in progress - but I'd like to put the instructions somewhere so that people who are interested can try it out.
 
I've already started watching and filing bugs regarding Chrome accessibility. One issue that would be great to fix sooner rather than later is the exposure of the display style of DOM nodes so that NVDA doesn't merge content on to a single line that should be separated.

How do we expose these? Maybe can you just point us to the method in NVDA's source code that interprets these?

You and your team have done a fantastic job so far. Chrome a11y has come in leaps and bounds in the last month or so. Keep up the great work!

Thank you!

- Dominic

Reply | Threaded
Open this post in threaded view
|

Re: [Nvda-dev] NVDA and Chrome

James Teh
On 24/06/2010 12:49 PM, Dominic Mazzoni wrote:
> There's Chrome Frame (embedding Chrome in IE), which does expose the
> MSAA COM interfaces as well. Other than that I'm not aware of any
> efforts to make Chrome embeddable.
Looks like Chrome Frame still uses the chrome.exe application, so the
app module still applies.

> Will the app module be included in nightly builds?
The app module is included as of main rev 3583. Feel free to update your
docs. I've noted in the NVDA change log that this is early experimental
support.

>     I've already started watching and filing bugs regarding Chrome
>     accessibility. One issue that would be great to fix sooner rather
>     than later is the exposure of the display style of DOM nodes so that
>     NVDA doesn't merge content on to a single line that should be separated.
> How do we expose these?
See this issue report:
http://code.google.com/p/chromium/issues/detail?id=47135#makechanges
I don't seem to be able to set labels like Feature-Accessibility. Is
there any way I can file accessibility bugs so you get notified sooner?

In short, you need to expose a "display" object attribute via
IAccessible2::attributes. IAccessible2::attributes is a string property,
which might return something like:
"display:block;"

For reference, see here for the attributes exposed by Gecko:
https://developer.mozilla.org/en/Accessibility:AT-APIs:Gecko:Attrs
Note that NVDA doesn't use a lot of these attributes and I suspect the
same is true of other ATs, so I'd probably expose them on a case by case
basis as you add features, rather than using it as a guide to what to
expose.

One other attribute you may want to expose is the "level" attribute for
headings.

Jamie

--
James Teh
Vice President
NV Access Inc, ABN 61773362390
Email: [hidden email]
Web site: http://www.nvaccess.org/


Reply | Threaded
Open this post in threaded view
|

Re: [Nvda-dev] NVDA and Chrome

yahir
 how y can test the new version of chrome with  NVDA, using
accessibility appmodule ?

2010/6/24, James Teh <[hidden email]>:

> On 24/06/2010 12:49 PM, Dominic Mazzoni wrote:
>> There's Chrome Frame (embedding Chrome in IE), which does expose the
>> MSAA COM interfaces as well. Other than that I'm not aware of any
>> efforts to make Chrome embeddable.
> Looks like Chrome Frame still uses the chrome.exe application, so the
> app module still applies.
>
>> Will the app module be included in nightly builds?
> The app module is included as of main rev 3583. Feel free to update your
> docs. I've noted in the NVDA change log that this is early experimental
> support.
>
>>     I've already started watching and filing bugs regarding Chrome
>>     accessibility. One issue that would be great to fix sooner rather
>>     than later is the exposure of the display style of DOM nodes so that
>>     NVDA doesn't merge content on to a single line that should be
>> separated.
>> How do we expose these?
> See this issue report:
> http://code.google.com/p/chromium/issues/detail?id=47135#makechanges
> I don't seem to be able to set labels like Feature-Accessibility. Is
> there any way I can file accessibility bugs so you get notified sooner?
>
> In short, you need to expose a "display" object attribute via
> IAccessible2::attributes. IAccessible2::attributes is a string property,
> which might return something like:
> "display:block;"
>
> For reference, see here for the attributes exposed by Gecko:
> https://developer.mozilla.org/en/Accessibility:AT-APIs:Gecko:Attrs
> Note that NVDA doesn't use a lot of these attributes and I suspect the
> same is true of other ATs, so I'd probably expose them on a case by case
> basis as you add features, rather than using it as a guide to what to
> expose.
>
> One other attribute you may want to expose is the "level" attribute for
> headings.
>
> Jamie
>
> --
> James Teh
> Vice President
> NV Access Inc, ABN 61773362390
> Email: [hidden email]
> Web site: http://www.nvaccess.org/
>
> _______________________________________________
> Nvda-dev mailing list
> [hidden email]
> http://lists.nvaccess.org/listinfo/nvda-dev
>


--
envía un e-mail a
[hidden email]
 aprende  de tecnología y mantente informado
windows!, GNU linux! y mac os x!.


Reply | Threaded
Open this post in threaded view
|

Re: [Nvda-dev] NVDA and Chrome

Dominic Mazzoni
In reply to this post by James Teh
On Thu, Jun 24, 2010 at 6:31 PM, James Teh <[hidden email]> wrote:
Will the app module be included in nightly builds?
The app module is included as of main rev 3583. Feel free to update your docs. I've noted in the NVDA change log that this is early experimental support.

Thanks!

See this issue report:
http://code.google.com/p/chromium/issues/detail?id=47135#makechanges
I don't seem to be able to set labels like Feature-Accessibility. Is there any way I can file accessibility bugs so you get notified sooner?

Thanks, I added the label. I'll see if we can get you a chromium.org account so that you can do this yourself.

- Dominic

Reply | Threaded
Open this post in threaded view
|

Re: [Nvda-dev] NVDA and Chrome

mk360
In reply to this post by yahir
the module has been included in the revision 3583, also you can copy the
module under the appModules directory. Note that you need the lattest snap
of chrome (see comments at the top of the file).
----- Original Message -----
From: "yahir" <[hidden email]>
To: "News and discussion for NVDA (NonVisual Desktop Access),a free and open
source screen reader for Microsoft Windows" <[hidden email]>
Sent: Thursday, June 24, 2010 9:37 PM
Subject: Re: [Nvda-dev] NVDA and Chrome


how y can test the new version of chrome with  NVDA, using
accessibility appmodule ?

2010/6/24, James Teh <[hidden email]>:

> On 24/06/2010 12:49 PM, Dominic Mazzoni wrote:
>> There's Chrome Frame (embedding Chrome in IE), which does expose the
>> MSAA COM interfaces as well. Other than that I'm not aware of any
>> efforts to make Chrome embeddable.
> Looks like Chrome Frame still uses the chrome.exe application, so the
> app module still applies.
>
>> Will the app module be included in nightly builds?
> The app module is included as of main rev 3583. Feel free to update your
> docs. I've noted in the NVDA change log that this is early experimental
> support.
>
>>     I've already started watching and filing bugs regarding Chrome
>>     accessibility. One issue that would be great to fix sooner rather
>>     than later is the exposure of the display style of DOM nodes so that
>>     NVDA doesn't merge content on to a single line that should be
>> separated.
>> How do we expose these?
> See this issue report:
> http://code.google.com/p/chromium/issues/detail?id=47135#makechanges
> I don't seem to be able to set labels like Feature-Accessibility. Is
> there any way I can file accessibility bugs so you get notified sooner?
>
> In short, you need to expose a "display" object attribute via
> IAccessible2::attributes. IAccessible2::attributes is a string property,
> which might return something like:
> "display:block;"
>
> For reference, see here for the attributes exposed by Gecko:
> https://developer.mozilla.org/en/Accessibility:AT-APIs:Gecko:Attrs
> Note that NVDA doesn't use a lot of these attributes and I suspect the
> same is true of other ATs, so I'd probably expose them on a case by case
> basis as you add features, rather than using it as a guide to what to
> expose.
>
> One other attribute you may want to expose is the "level" attribute for
> headings.
>
> Jamie
>
> --
> James Teh
> Vice President
> NV Access Inc, ABN 61773362390
> Email: [hidden email]
> Web site: http://www.nvaccess.org/
>
> _______________________________________________
> Nvda-dev mailing list
> [hidden email]
> http://lists.nvaccess.org/listinfo/nvda-dev
>


--
envía un e-mail a
[hidden email]
 aprende  de tecnología y mantente informado
windows!, GNU linux! y mac os x!.

_______________________________________________
Nvda-dev mailing list
[hidden email]
http://lists.nvaccess.org/listinfo/nvda-dev