Re: [nvaccess/nvda] Speak typed characters in UWP apps on Windows 10 RS2 and above (#6631)

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [nvaccess/nvda] Speak typed characters in UWP apps on Windows 10 RS2 and above (#6631)

derek riemer

Could we cache the key buffer that we get by requesting the state of every key on the keyboard? Sorry, I'm just trying to understand this code a bit more as I'm curious.


On 12/15/2016 7:52 PM, James Teh wrote:

@jcsteh requested changes on this pull request.


In source/keyboardHandler.py:

> @@ -162,6 +167,21 @@ def internal_keyDownEvent(vkCode,scanCode,extended,injected):
 				return False
 	except:
 		log.error("internal_keyDownEvent", exc_info=True)
+	finally:

Might be worth commenting why we do this in the finally block; i.e. because we return early in several places but we still want this to be executed.


In source/keyboardHandler.py:

> @@ -162,6 +167,21 @@ def internal_keyDownEvent(vkCode,scanCode,extended,injected):
 				return False
 	except:
 		log.error("internal_keyDownEvent", exc_info=True)
+	finally:
+		# #6017: handle typed characters for UWP apps in Win10 RS2 and above where we can't detect typed characters in-process 
+		focus=api.getFocusObject()
+		if sys.getwindowsversion().build>=14986 and not gestureExecuted and not isNVDAModifierKey(vkCode,extended) and not vkCode in KeyboardInputGesture.NORMAL_MODIFIER_KEYS and focus.windowClassName.startswith('Windows.UI.Core'):

Calling sys.getwindowsversion probably isn't expensive, but we already have it cached in winVersion.winVersion anyway, so I'd use that instead.


In source/keyboardHandler.py:

> @@ -162,6 +167,21 @@ def internal_keyDownEvent(vkCode,scanCode,extended,injected):
 				return False
 	except:
 		log.error("internal_keyDownEvent", exc_info=True)
+	finally:
+		# #6017: handle typed characters for UWP apps in Win10 RS2 and above where we can't detect typed characters in-process 
+		focus=api.getFocusObject()
+		if sys.getwindowsversion().build>=14986 and not gestureExecuted and not isNVDAModifierKey(vkCode,extended) and not vkCode in KeyboardInputGesture.NORMAL_MODIFIER_KEYS and focus.windowClassName.startswith('Windows.UI.Core'):
+			keyStates=(ctypes.c_byte*256)()
+			for k in xrange(256):
+				keyStates[k]=ctypes.windll.user32.GetKeyState(k)
+			charBuf=ctypes.create_unicode_buffer(5)
+			# Normally calling ToUnicodeEx would destroy keyboard buffer state and therefore cause the app to not produce the right WM_CHAR message.
+			# However in UWP apps in Windows 10 RS2 and above, wm_keydown / wm_char is never processed, thus NVDA will be the only one calling it. 

Is this comment still true? I thought the point of the flag was to stop this state destroying behaviour, rather than it being something specific to UWP.


In source/keyboardHandler.py:

> @@ -162,6 +167,21 @@ def internal_keyDownEvent(vkCode,scanCode,extended,injected):
 				return False
 	except:
 		log.error("internal_keyDownEvent", exc_info=True)
+	finally:
+		# #6017: handle typed characters for UWP apps in Win10 RS2 and above where we can't detect typed characters in-process 
+		focus=api.getFocusObject()
+		if sys.getwindowsversion().build>=14986 and not gestureExecuted and not isNVDAModifierKey(vkCode,extended) and not vkCode in KeyboardInputGesture.NORMAL_MODIFIER_KEYS and focus.windowClassName.startswith('Windows.UI.Core'):
+			keyStates=(ctypes.c_byte*256)()
+			for k in xrange(256):
+				keyStates[k]=ctypes.windll.user32.GetKeyState(k)
+			charBuf=ctypes.create_unicode_buffer(5)
+			# Normally calling ToUnicodeEx would destroy keyboard buffer state and therefore cause the app to not produce the right WM_CHAR message.
+			# However in UWP apps in Windows 10 RS2 and above, wm_keydown / wm_char is never processed, thus NVDA will be the only one calling it. 
+			hkl=ctypes.windll.user32.GetKeyboardLayout(focus.windowThreadID)
+			res=ctypes.windll.user32.ToUnicodeEx(vkCode,scanCode,keyStates,charBuf,5,4,hkl)
  1. I assume 4 is the flag? Please comment as to what the 4 does.
  2. I assume 5 is the size? Perhaps use len(charBuf) to be clearer and so we don't break things if we change the size in future.

In source/keyboardHandler.py:

> @@ -162,6 +167,21 @@ def internal_keyDownEvent(vkCode,scanCode,extended,injected):
 				return False
 	except:
 		log.error("internal_keyDownEvent", exc_info=True)
+	finally:
+		# #6017: handle typed characters for UWP apps in Win10 RS2 and above where we can't detect typed characters in-process 
+		focus=api.getFocusObject()
+		if sys.getwindowsversion().build>=14986 and not gestureExecuted and not isNVDAModifierKey(vkCode,extended) and not vkCode in KeyboardInputGesture.NORMAL_MODIFIER_KEYS and focus.windowClassName.startswith('Windows.UI.Core'):
+			keyStates=(ctypes.c_byte*256)()
+			for k in xrange(256):
+				keyStates[k]=ctypes.windll.user32.GetKeyState(k)
+			charBuf=ctypes.create_unicode_buffer(5)
+			# Normally calling ToUnicodeEx would destroy keyboard buffer state and therefore cause the app to not produce the right WM_CHAR message.
+			# However in UWP apps in Windows 10 RS2 and above, wm_keydown / wm_char is never processed, thus NVDA will be the only one calling it. 
+			hkl=ctypes.windll.user32.GetKeyboardLayout(focus.windowThreadID)
+			res=ctypes.windll.user32.ToUnicodeEx(vkCode,scanCode,keyStates,charBuf,5,4,hkl)
+			if res>0:
+				for ch in charBuf[:res]: 
+					eventHandler.queueEvent("typedCharacter",focus,ch=ch)

Is this saying there can be multiple separate characters? I know there are languages where a single key press produces a conjunct Unicode character sequence (e.g. Tamil), but we probably actually want that to be handled as one character. We can't do that with WM_CHAR right now, but is there a reason we shouldn't do this here since we can?


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.


-- 
Once upon a time, I had a signature. One day, my signature disappeared. The end.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Nvda-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/nvda-devel
Loading...