rainforestapp/flashlight-vnc

Rainforest's fork of Flashlight-vnc

Flashlight-vnc

Rainforest's fork of Flashlight-vnc. We use this for our crowdsourced testing platform to connect our testers to our virtual machines.

flashlight-vnc's Issues

Flashlight sends spurious key events

Flashlight is sending spurious keypress/key release events. Analysing the protocol using Wireshark we see that typing a word like "dog", which should be six commands:

key press "d" / key release "d" / key press "o" / "key release "o" / key press "g" / key release "g"

is instead rendered as:

key press NULL / key release NULL / key press "d" / key release "d" / key press NULL / key release NULL / key press "o" / key release "o" ... and so on

The NULL is literally a 0x00 byte.

Our main suspicion is that the onTextinput is generating writeKeyEvents for the character 0x00 for some reason.

If fixed, the "Massive ugly hack" a few lines may also (probably) be removed.

| character is not being sent to VM

One of the users used | (symbol from above \ not capital i) in their password and it's not getting sent to VM properly.

Example: http://sharing.rainforestqa.com/3w351J3q3M2X/o

I don't have r/w access here so it's on you @jbarber

Support for sound

Improve the SecurityErrorEvent message

The error message when failing to connect to the security policy is not very useful:

An security error occurred (Error #2048). Check your policy-policy server configuration or disable security for this domain

We should change this to tell the user what they should do.

Ctrl key does not work

The ctrl key does not work on the client machine. While fixing, this be very careful to not break to copy/pasting feature as this is related to this.

Ctrl-V should still paste the content of the server's clipboard, but other use of the ctrl-key should be sent directly to the client.

Support for sound

Client sends too many Ctrl's

Using the following program:
http://delphiforfun.org/programs/utilities/KeyCodes.htm

we can see that a single press of the 'a' key results in the following key events:

OnKeyDown, Key code=255, Control keys=, Key name
OnKeyup, Key code=17, Control keys=, Key name CTRL
OnKeyup, Key code=17, Control keys=, Key name CTRL
OnKeyDown, Key code=65, Control keys=, Key name a
OnkeyPress a
OnKeyup, Key code=17, Control keys=, Key name CTRL
OnKeyup, Key code=64, Control keys=, Key name CTRL
OnKeyup, Key code=255, Control keys=, Key name

'A' results in:

OnKeyDown, Key code=16, Control keys=Shift{Left}, Key name SHIFT
OnKeyDown, Key code=255, Control keys=Shift{Left}, Key name
OnKeyup, Key code=17, Control keys=Shift{Left}, Key name CTRL
OnKeyup, Key code=17, Control keys=Shift{Left}, Key name CTRL
OnKeyDown, Key code=65, Control keys=Shift{Left}, Key name A
OnKeyPress A
OnKeyup, Key code=17, Control keys=Shift{Left}, Key name CTRL
OnKeyup, Key code=65, Control keys=Shift{Left}, Key name A
OnKeyup, Key code=255, Control keys=Shift{Left}, Key name
OnKeyUp, Key code=16, Control keys=, Key name SHIFT

'3' results in:

OnKeyDown, Key code=255, Control keys=, Key name
OnKeyup, Key code=17, Control keys=, Key name CTRL
OnKeyup, Key code=17, Control keys=, Key name CTRL
OnKeyDown, Key code=51, Control keys=, Key name 3
OnkeyPress 3
OnKeyup, Key code=51, Control keys=, Key name 3
OnKeyup, Key code=17, Control keys=, Key name CTRL
OnKeyup, Key code=255, Control keys=, Key name

'#' results in:

OnKeyDown, Key code=16, Control keys=Shift{Left}, Key name SHIFT
OnKeyDown, Key code=255, Control keys=Shift{Left}, Key name
OnKeyup, Key code=17, Control keys=Shift{Left}, Key name CTRL
OnKeyup, Key code=17, Control keys=Shift{Left}, Key name CTRL
OnKeyDown, Key code=51, Control keys=Shift{Left}, Key name 3
OnKeyPress #
OnKeyup, Key code=17, Control keys=Shift{Left}, Key name CTRL
OnKeyup, Key code=51, Control keys=Shift{Left}, Key name A
OnKeyup, Key code=255, Control keys=Shift{Left}, Key name
OnKeyUp, Key code=16, Control keys=, Key name SHIFT

With a VM-inside-a-VM, 'a' gives:

OnKeyDown, Key code=255, Control keys=Shift{Left}, Key name
OnKeyUp, Key code=17, Control keys=Shift{Left}, Key name CTRL
OnKeyDown, Key code=255, Control keys=Shift{Left}, Key name
OnKeyUp, Key code=17, Control keys=Shift{Left}, Key name CTRL
OnKeyup, Key code=255, Control keys=Shift{Left}, Key name
OnKeyup, Key code=17, Control keys=Shift{Left}, Key name CTRL
OnKeyDown, Key code=64, Control keys=Shift{Left}, Key name A
OnKeyPress a
OnKeyup, Key code=17, Control keys=Shift{Left}, Key name CTRL
OnKeyup, Key code=65, Control keys=Shift{Left}, Key name A

'3' gives:

OnKeyDown, Key code=255, Control keys=Shift{Left}, Key name
OnKeyUp, Key code=17, Control keys=Shift{Left}, Key name CTRL
OnKeyDown, Key code=255, Control keys=Shift{Left}, Key name
OnKeyUp, Key code=17, Control keys=Shift{Left}, Key name CTRL
OnKeyup, Key code=17, Control keys=Shift{Left}, Key name CTRL
OnKeyup, Key code=17, Control keys=Shift{Left}, Key name CTRL
OnKeyDown, Key code=51, Control keys=Shift{Left}, Key name 3
OnKeyPress 3
OnKeyup, Key code=17, Control keys=Shift{Left}, Key name CTRL
OnKeyup, Key code=65, Control keys=Shift{Left}, Key name 3

Reconnecting message should be visible in the middle of the screen

When using the terminal in full screen, we don't see the error message when being disconnected from the terminal. The message should be clearly visible on the middle on the screen. The countdown before a reconnection attempt should also be visible.

Here's a mock of what this should look like.

Copy/Paste in IE9HD failing

Customer: MindBodyGreen
Customer ID: 4939

Expected Behavior: Copy/Pasting values generated by social variables captures values completely when copying and enters value correctly when pasting into VM.

Actual Behavior: Copy/pasting failing to properly capture/enter values in IE9HD; doesn't seem to occur in other IE VMs.

Job Group:
https://admin.rainforestqa.com/job_groups/2542104

Keyboard stops working temporarily after switching tab

I've seen this bug in chrome on OS X. Might affect other browsers as well.

Steps to repro:

1. Have flashlight vnc session running
2. Make sure you can type in a text box in flashlight
3. Open a new tab
4. Switch to the tab (I've only been able to reproduce using the cmd-shift-[ shortcut to switch tab)
5. Switch back to the tab where flashlight is running
   -> Keyboard does not work for a few seconds then it starts working again

Pasting iOS Safari

from asana

So the problem is isolated to HTML text input in safari, pasting to native text widgets works fine.

Pasting works by sending the keypresses required to type the string, and the solution appears to be to send each character in the string with a greater gap inbetween them. Ghetto.

Pasting text sometimes missed a character

Do pasting properly?

The RFB protocol has a ClientCutText verb, which looks like it's a way of sending pasted data.

Unfortunately, it looks like our version of qemu doesn't support ClientCutText(), so it's no use implementing it in the client.

Further wrinkle, the client uses a TextField as the basis of the VNC display, which according to TFM for the PASTE event :

TextField objects do not dispatch clear, copy, cut, paste, or selectAll events. TextField objects always include Cut, Copy, Paste, Clear, and Select All commands in the context menu. You cannot remove these commands from the context menu for TextField objects. For TextField objects, selecting these commands (or their keyboard equivalents) does not generate clear, copy, cut, paste, or selectAll events. However, other classes that extend the InteractiveObject class, including components built using the Flash Text Engine (FTE), will dispatch these events in response to user actions such as keyboard shortcuts and context menus

You can only access the system clipboard via the Paste event:

In Flash Player 10, a paste operation from the clipboard first requires a user event (such as a keyboard shortcut for the Paste command or a mouse click on the Paste command in a context menu). Clipboard.getData() will return the contents of the clipboard only if the InteractiveObject has received and is acting on a paste event. Calling Clipboard.getData() under any other circumstances will be unsuccessful. The same restriction applies in AIR for content outside the application sandbox.

I did try to remove the TextField, and move the event handlers for keyup/down to work, but couldn't then get the app to send key presses or the clipboard.