thanhlev/keyboard_mouse_emulate_on_raspberry

Bluetooth Keyboard Mouse Emulator on Raspberry Pi

Make things work first

Step 1: Setup

 sudo ./setup.sh

Step 2: Run the Server

sudo ./boot.sh

Step 3.1: Run Keyboard Client (using physical keyboard)

• Need a physical keyboard connected to raspberry PI board

./keyboard/kb_client.py

Step 3.2: Run Keyboard Client (no need physical keyboard, send string through dbus)

• Dont need a physical keyboard connected to raspberry PI board

./keyboard/send_string.py "hello client, I'm a keyboard"

Step 3.3: Run mouse client (using physical mouse)

• Need a physical mouse connected to raspberry PI board

./mouse/mouse_client.py

Step 3.4: Run Mouse client (no need physical mouse, string mouse data through dbus)

• Dont need a physical mouse connected to raspberry PI board

./mouse/mouse_emulate.py 0 10 0 0

To understand what I'm doing in the background

Make Raspberry Pi3 as an emulator bluetooth keyboard

Keyboard setup demo (old version)

Mouse setup demo (ongoing)

Emulate Bluetooth mouse with Raspberry Pi

keyboard_mouse_emulate_on_raspberry's Issues

Would your tool also work on other devices like Intel NUC?

Hi, before I try to get the tool running just wanted to ask for a general opinion how likely it would work out of the box or if I should not expect great results. Thanks! Helge

iOS not auto connect to mouse (RPI), how solve it?

iOS not auto connect to mouse (RPI), how solve it?

Can't run client

Can't run any of the clients, but this is specifically what I get when trying to run ./keyboard/kb_client.py

``noah@raspberrypi:~/keyboard_mouse_emulate_on_raspberry $ ./keyboard/kb_client.py
Setting up keyboard
setting up DBus Client
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/dbus/bus.py", line 177, in activate_name_owner
return self.get_name_owner(bus_name)
File "/usr/lib/python3/dist-packages/dbus/bus.py", line 361, in get_name_owner
return self.call_blocking(BUS_DAEMON_NAME, BUS_DAEMON_PATH,
File "/usr/lib/python3/dist-packages/dbus/connection.py", line 652, in call_blocking
reply_message = self.send_message_with_reply_and_block(
dbus.exceptions.DBusException: org.freedesktop.DBus.Error.NameHasNoOwner: Could not get owner of name 'org.thanhle.btkbservice': no such name

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/noah/keyboard_mouse_emulate_on_raspberry/./keyboard/kb_client.py", line 109, in
kb = Keyboard()
File "/home/noah/keyboard_mouse_emulate_on_raspberry/./keyboard/kb_client.py", line 47, in init
self.btkservice = self.bus.get_object(
File "/usr/lib/python3/dist-packages/dbus/bus.py", line 241, in get_object
return self.ProxyObjectClass(self, bus_name, object_path,
File "/usr/lib/python3/dist-packages/dbus/proxies.py", line 250, in init
self._named_service = conn.activate_name_owner(bus_name)
File "/usr/lib/python3/dist-packages/dbus/bus.py", line 182, in activate_name_owner
self.start_service_by_name(bus_name)
File "/usr/lib/python3/dist-packages/dbus/bus.py", line 277, in start_service_by_name
return (True, self.call_blocking(BUS_DAEMON_NAME, BUS_DAEMON_PATH,
File "/usr/lib/python3/dist-packages/dbus/connection.py", line 652, in call_blocking
reply_message = self.send_message_with_reply_and_block(
dbus.exceptions.DBusException: org.freedesktop.DBus.Error.ServiceUnknown: The name org.thanhle.btkbservice was not provided by any .service files

3b or 3b+，and Kernel version

What I want to know if you used 3b or 3b+, and its Kernel version

Integer to signed 8-bit f

min(127, max(-127, int(self.x * speed))) & 255

the lower boundary should be -128.

min(127, max(-128, int(self.x * speed))) & 255

I got my own code working thanks to your last update - good inspiration, thanks!

Refuses to connect to android devices

I find this project somewhat amazing – there are at least three sets of instructions, all different:

1. https://thanhle.me/make-raspberry-pi3-as-an-emulator-bluetooth-keyboard/, this contains a link to this project as part of the code.
2. The README for this project.
3. https://thanhle.me/emulate-bluetooth-mouse-with-raspberry-pi/

The name of the project has also been changed, fortunately Github works around that. After entering tmux a, I find there are only three panes, so there's nothing to use to run bluetoothctl. It's possible that there's another session that I haven't worked out how to connect to.

So after splitting one of the panes to get a bluetoothctl session, I can get all the parts working. However, both android devices I have tested can see the ThanhLe_Keyboard_Mouse device, and to manage to get paired with it, but they won't connect. The bluetoothctl output shows them connect and then immediately disconnect. I have no idea why and no idea how to find out.

I only investigated using a Raspberry Pi 3 as a bluetooth keyboard because I've got one, and a spare keyboard and a user raised the idea of keyboard shortcuts for my android Editor app. I found that most of the implementations online can't possibly work because the BlueZ project made fundamental changes to the API that, AFAICS, make certain that any previous projects won't work.

I found that it does work with a Windows desktop

So, having wasted about a day attempting to get this to work on android, I have given up.

Keyboard reconnect function?

Hello! Really enjoying using this service to connect my Pi as a keyboard to iOS devices. Does the DBUS library the service uses allow the server to reconnect (or attempt to reconnect) with clients on a periodic basis? I'd like to try to automate re-connection so that the connection is restored each morning.

Thank you!

Play/Pause with ipad

Any idea why the playpause key does not work on a ipad?

usb instead of bluetooth?

this project is nice and I was wondering if the same thing can be done to control a computer remotely. I always wanted to control a server and be able to install things, modify BIOS settings, etc. Mouse and keyboard seem to be working. The next step would be getting the screen sent to me, there are cheap USB dongles for HDMI capture on aliexpress. But for all that it shouldn't be bluetooth but instead raspberry pi as a usb device emulating mouse/keyboard.

unable to get the keyboard data to remote pc

Hello there,

         First of all thanks for the working code. I was playing with your code from yesterday and i'm getting the keyboard data from RPI to remote pc via Bluetooth. Now the problem is today I'm not getting any communication  from RPI! and more-over Bluetooth connections seems good but still i have't any packet from rpi

can you please help me to fix the issue
Thanks

Cannot pair using an iPad

I'm using a freshly installed raspberry pi and using the instructions. The bluetooth device appears in the bluetooth settings and after pressing it, a dialog shows a confirmation code, after pressing continue it takes a while before an error message appears. Looking at the log below it seems to time out during pairing.

I've googled in search of solutions but unfortunately without luck. Is there anyone with a relatively new iOS version that is willing to share their configurations or workarounds?

Following is the output of sudo btmon

...
HCI Event: IO Capability Response (0x32) plen 9 #443 [hci0] 497.153151
Address: 50:DE:06:25:45:A1 (Apple, Inc.)
IO capability: DisplayYesNo (0x01)
OOB data: Authentication data not present (0x00)
Authentication: General Bonding - MITM not required (0x04)
HCI Event: IO Capability Request (0x31) plen 6 #444 [hci0] 497.153241
Address: 50:DE:06:25:45:A1 (Apple, Inc.)
< HCI Command: IO Capability Request Reply (0x01|0x002b) plen 9 #445 [hci0] 497.153288
Address: 50:DE:06:25:45:A1 (Apple, Inc.)
IO capability: DisplayYesNo (0x01)
OOB data: Authentication data not present (0x00)
Authentication: General Bonding - MITM required (0x05)
HCI Event: Command Complete (0x0e) plen 10 #446 [hci0] 497.153995
IO Capability Request Reply (0x01|0x002b) ncmd 1
Status: Success (0x00)
Address: 50:DE:06:25:45:A1 (Apple, Inc.)
HCI Event: Number of Completed Packets (0x13) plen 5 #447 [hci0] 497.327546
Num handles: 1
Handle: 11
Count: 1
HCI Event: User Confirmation Request (0x33) plen 10 #448 [hci0] 497.358840
Address: 50:DE:06:25:45:A1 (Apple, Inc.)
Passkey: 906238
@ MGMT Event: User Confirmation Request (0x000f) plen 12 {0x0001} [hci0] 497.358876
BR/EDR Address: 50:DE:06:25:45:A1 (Apple, Inc.)
Confirm hint: 0x00
Value: 0x000dd3fe
HCI Event: Simple Pairing Complete (0x36) plen 7 #449 [hci0] 530.117117
Status: Authentication Failure (0x05)
Address: 50:DE:06:25:45:A1 (Apple, Inc.)
@ MGMT Event: Authentication Failed (0x0011) plen 8 {0x0001} [hci0] 530.117172
BR/EDR Address: 50:DE:06:25:45:A1 (Apple, Inc.)
Status: Authentication Failed (0x05)
HCI Event: Disconnect Complete (0x05) plen 4 #450 [hci0] 530.118063
Status: Success (0x00)
Handle: 11
Reason: LMP Response Timeout / LL Response Timeout (0x22)
@ MGMT Event: Device Disconnected (0x000c) plen 8 {0x0001} [hci0] 530.118125
BR/EDR Address: 50:DE:06:25:45:A1 (Apple, Inc.)
Reason: Unspecified (0x00)

device_confirm_passkey: Operation not permitted

I'm attempting to connect my Raspberry Pi to an Nvidia Shield to act as a keyboard. I can get it connected, but if either side disconnects (e.g. reboots) any reconnect attempts fail. Below are the systemctl status logs with the errors. Thoughts?

May 09 21:19:33 raspberrypi systemd[1]: Started Bluetooth service.
May 09 21:19:33 raspberrypi bluetoothd[1514]: Starting SDP server
May 09 21:19:33 raspberrypi bluetoothd[1514]: Excluding (cli) input
May 09 21:19:33 raspberrypi bluetoothd[1514]: Bluetooth management interface 1.18 initialized
May 09 21:19:33 raspberrypi bluetoothd[1514]: Sap driver initialization failed.
May 09 21:19:33 raspberrypi bluetoothd[1514]: sap-server: Operation not permitted (1)
May 09 21:20:02 raspberrypi bluetoothd[1514]: No agent available for request type 2
May 09 21:20:02 raspberrypi bluetoothd[1514]: device_confirm_passkey: Operation not permitted
May 09 21:20:45 raspberrypi bluetoothd[1514]: No agent available for request type 2
May 09 21:20:45 raspberrypi bluetoothd[1514]: device_confirm_passkey: Operation not permitted

SOLUTION: Does not work on RPI 4

I was tweaking another implementation when I found this one. At first I thought my changes did nothing, but I had to restart as I believe my old tweaking had some cross over into this great project.

I changed this line in bluetooth.service as the path has chnaged for bluetoothd:

-ExecStart=/usr/lib/bluetooth/bluetoothd --noplugin=input
+ExecStart=/usr/libexec/bluetooth/bluetoothd --noplugin=input

I also changed this in dbus/org.thanhle.btkbservice.conf, which I do not know if this made any difference and I did not seek to understand it (just looking at another implementation and trying it out first...probably bad practice):

-                <deny own="org.thanhle.btkbservice"/>
+                <allow own="org.thanhle.btkbservice"/>

how to use it?

fantastic work! The project can run successful and I can see a keyboard from my phone and the connection is ok. But I can not use it to input on the phone? I wonder if there is a description on how to use it?

Address already in use

Hi, I'd like to use the project but i keep getting this behaviour :

root@raspberrypi:~/src/BL_keyboard_RPI/server# python btk_server.py 
/usr/lib/python2.7/dist-packages/gtk-2.0/gtk/__init__.py:57: GtkWarning: could not open display
  warnings.warn(str(e), _gtk.Warning)
Setting up service
Setting up BT device
Configuring for name yBO_Keyboard
Configuring Bluez Profile
Reading service record
Profile registered 
Waiting for connections
Traceback (most recent call last):
  File "btk_server.py", line 226, in <module>
    myservice = BTKbService();
  File "btk_server.py", line 198, in __init__
    self.device.listen();
  File "btk_server.py", line 159, in listen
    self.scontrol.bind((self.MY_ADDRESS,self.P_CTRL))
  File "/usr/lib/python2.7/dist-packages/bluetooth/bluez.py", line 145, in bind
    return self._sock.bind (addrport)
_bluetooth.error: (98, 'Address already in use')

strace show that:
bind(5, {sa_family=AF_BLUETOOTH, sa_data="\21\0\"\350q:\n\0\0\0\0\0"}, 14) = -1 EADDRINUSE (Address already in use)

i have tried to change the value of P_CTRL to something else than 17 but it does not help

if i understood correctly, the port 17 (0x0011) is also the attibute 0x0004 of the sdp service definition

root@raspberrypi:~/src/BL_keyboard_RPI/server# cat sdp_record.xml | grep -A 4 0x0004
        <attribute id="0x0004">
                <sequence>
                        <sequence>
                                <uuid value="0x0100" />
                                <uint16 value="0x0011" />

but i have currently no service using this port:

root@raspberrypi:~/src/BL_keyboard_RPI/server# sdptool browse --xml local | grep -A 4 0x0004
        <attribute id="0x0004">
                <sequence>
                        <sequence>
                                <uuid value="0x0100" />
                                <uint16 value="0x001f" />
--
        <attribute id="0x0004">
                <sequence>
                        <sequence>
                                <uuid value="0x0100" />
                                <uint16 value="0x001f" />
--
        <attribute id="0x0004">
                <sequence>
                        <sequence>
                                <uuid value="0x0100" />
                                <uint16 value="0x0017" />
--
        <attribute id="0x0004">
                <sequence>
                        <sequence>
                                <uuid value="0x0100" />
                                <uint16 value="0x0017" />

thx for your help.

Can't locate module

pi@raspi:~/Desktop/Raspberry/Keyboard mouse emulate on raspberry/keyboard_mouse_emulate_on_raspberry-master/keyboard $ python send_string.py "Test string" Traceback (most recent call last): File "send_string.py", line 4, in <module> import dbus ImportError: No module named dbus
I've already tried to import dbus-next (dbus_next) instead of dbus, I've added export PYTHONPATH="${PAYTHONPATH}:usr/local/lib/python3.7/site-packages:/usr/lib/python3.7/site-packages"
I've reinstalled pip, python, dbus library, dbus-next and updated system, I'm running on latest Raspbian.
Everything ends same, ImportError line 4.

error connecting to /tmp/tmux-0/default (no such file or directory)

Hey, great work on this. I have been searching for a well built bluetooth HID emulator for the Pi for a while now. When I try to run the boot.sh script, i get this error:

error connecting to /tmp/tmux-0/default (No such file or directory)

I am running this on a Pi 0W, but will test on my Pi 3 later tonight. Another question, can I send keystrokes over SSH? Say I SSH into my Pi, run the script, and share the keyboard from my machine SSHing into the Pi to wahtever device I have picking up the bluetooth signal.

Mice support

Hi,
Many thanks for your work.
I was wondering how could I add mice support to your project.
Did you already try ?

Bluetooth Device is Seen As Audio

This is a great out-of-the-box script. I only had to change one thing for RPI 4. But the device keeps being recognized as both an Input device and a Audio device.

In lower right sound button of Windows:

In Bluetooth Settings in Windows:

Absolute mouse position

First I wanna say this project is great and has been very useful to me. I've been using the mouse and keyboard emulate functions a lot and it would be awesome if I could send absolute mouse coordinates to mouse_emulate.py rather than just relative movements. Maybe there is a simple alteration I can make to the send_mouse byte array to make this happen?

Slow mouse polling?

Hi there, just had a test run of your project.
I'm trying to use a Pi Zero as a keyboard+mouse hub for a couple servers using your tool.
I've tried pairing to a computer to test and it looks like the mouse movement is laggy (slow polling maybe?).

Any idea on fixes / where to improve? (I can't see in the mouse and btk_server scripts where improvements can be made as I am new to HID devices...).

some problems when connected to ios or pc

on ios , with accessibility opening , mouse works well , but sometimes, 'WARNING:root:[Errno 11] Resource temporarily unavailable' occurs

but keybord never works , neither './keyboard/kb_client.py' nor './keyboard/send_string.py "hello client, I'm a keyboard" demo'
the same on PC

.

Wrong keyboard path, keyboard not found

Hi! I really enjoy this project, thank you!
When I tried to use it the first time, I had a problem: the mouse was working, but the keyboard wasn't (keyboard not found). Then I found the problem: the keyboard path (in kb_client.py, line 57). The original was:

/dev/input/event0

I tried with

/dev/input/by-path/platform-...-event-kbd

and it worked. I use a Raspberry Pi 4 with the last version of Raspberry OS.

hello,where is the reporter descriptor?

i use this, and i want to know the reporter descriptor? thank you for a lot.

setup error

bluetoothd needs to be started with --compat flag

After several hours experimenting it works perfectly when you start bluetoothd with the compatibility (--compat) flag
sudo /usr/sbin/bluetoothd --compat --nodetach --debug -p time

When started without --compat this error appears:

pi@raspberrypi:~/BL_keyboard_RPI/server $ sudo python btk_server.py
PuTTY X11 proxy: Unsupported authorisation protocol
/usr/lib/python2.7/dist-packages/gtk-2.0/gtk/__init__.py:57: GtkWarning: could not open display
  warnings.warn(str(e), _gtk.Warning)
Setting up service
Setting up BT device
Configuring for name ThanhLe_Keyboard
Configuring Bluez Profile
Reading service record
Profile registered
Waiting for connections
Traceback (most recent call last):
  File "btk_server.py", line 214, in <module>
    myservice = BTKbService();
  File "btk_server.py", line 188, in __init__
    self.device.listen();
  File "btk_server.py", line 149, in listen
    self.scontrol.bind((self.MY_ADDRESS,self.P_CTRL))
  File "/usr/lib/python2.7/dist-packages/bluetooth/bluez.py", line 140, in bind
    return self._sock.bind (addrport)
_bluetooth.error: (98, 'Address already in use')

How to send ENTER, UP ARROW etc

For example
./send_string.py "UP ARROW" or ENTER for example, how can I do this please?

Remove GTK Dependency

Hello,

Love the app - have it running and working it through on an RPi Zero W. One thing I'd like to do is to remove the GTK dependency however I'm not too with it on GTK but I have worked out that the gtk.main() call starts some kind of event loop that the rest of the code depends on. I think I can use glib to start this which has no gtk dependency. If you have any pointers I'd be very grateful for any help!

Thanks.

can't work on RPi 4

Thank you very much for the explanation. I have been trying it for many days.
Everything is working fine, but I can’t send keystrokes to any devices.
I am using RPi 4.

Server doesn't get a connection on a previously paired host

I use Windows 10 as a host system that I pair with my Pi.
I've added hosts MAC as trusted on my Pi etc.

The first time I find my Pi and pair it with Windows 10 (no pin code used) the server in Pi sees the socket connection and works fine.
If I turn Bluetooth off on Windows 10 and turn it back on - the server doesn't see the connection. Even though device in Win 10 displayed as "Connected" after turning Bluetooth back on. Restarting the server in Pi doesn't help.
No combination of restarting Bluetooth/Pi/Windows/your server works - I have to manually remove the Bluetooth device in Win 10 and find it and pair it again.

I don't know if this project is supported still, would appreciate any help debugging this. Not sure where to start.

Can't connect to keyboard on Mac

The bluetooth keyboard appears in the bluetooth settings menu and when I click connect it prompts me to connect but after I click it nothing happens it fails to connect to emulated keyboard on the raspberry pi. Anyone have this problem or know a solution?

Windows 10 compatibility

Hi,

I tried your project to make a raspberry as a custom bluetooth peripheral with a windows 10 computer. When I want to pair the device everything goes right, then window 10 store the connection but doesn't offer a way to connect the raspberry a second time. So, if I want to use again the raspberry as keyboard and mouse I have to unpair the devices.

To resolve this issue I've modified the code to make the raspberry a client of Windows using the connect function. Alas WIndows 10 accept a short connection and then disconnect. Have you any clue about how to solve this problem?

With Android phones and linux system I don't have any problem.

Thanks
Emanuele

Connect & Disconnect

The first connection works fine however when a disconnection happens the dbus methods for request disconnection and release are not called. You can see the disconnection happening from the sysout on the from the bluetoothd call. Note that I'm running headless on an RPi0W so I've removed the tmux stuff from the startup. I'm investigating but if anyone else knows how to resolve this I'd be grateful

Error on startup

Hi,

I have encountered this error upon startup.

pi@raspberrypi:~/projects/keyboard_mouse_emulate_on_raspberry $ sudo ./boot.sh                                           
[ ok ] Starting bluetooth (via systemctl): bluetooth.service.
DC:A6:32:D4:1F:82
pi@raspberrypi:~/projects/keyboard_mouse_emulate_on_raspberry $ ./mouse/mouse_client.py                                  
detected mouse: /dev/input/event0
INFO:root:Connected MouseInput@/dev/input/event0 (Logitech USB Optical Mouse)                                            
ERROR:dbus.proxies:Introspect error on :1.26:/org/thanhle/btkbservice: dbus.exceptions.DBusException: org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.                 
DEBUG:dbus.proxies:Executing introspect queue due to error
ERROR:dbus.connection:Unable to set arguments (0, b'\x00\xfe\xff\x00') according to signature None: <class 'ValueError'>:
embedded null byte
Traceback (most recent call last):
  File "./mouse/mouse_client.py", line 149, in <module>
    i.change_state(event)
  File "./mouse/mouse_client.py", line 116, in change_state
    self.send_current(self.state)
  File "./mouse/mouse_client.py", line 97, in send_current
    self.iface.send_mouse(0, bytes(ir))
  File "/usr/lib/python3/dist-packages/dbus/proxies.py", line 70, in __call__                                            
    return self._proxy_method(*args, **keywords)
  File "/usr/lib/python3/dist-packages/dbus/proxies.py", line 145, in __call__                                           
    **keywords)
  File "/usr/lib/python3/dist-packages/dbus/connection.py", line 641, in call_blocking                                   
    message.append(signature=signature, *args)
ValueError: embedded null byte
pi@raspberrypi:~/projects/keyboard_mouse_emulate_on_raspberry $ 

The same happens when I start it with sudo. It's a Raspberry PI 4.

python3-rpi.gpio is also needed

Thank you for this project!
I've been looking for this for a while!

In my case the package python3-rpi.gpio was also needed 😉

Transport endpoint is not connected error

Hi, I got the Transport endpoint is not connected error message, but the rpi pi 3 a+ is connected to my android phone:

What I did:
I did run ./boot.sh after installing the project based on this article. And pair my device. I also see that the rpi show a connection window, and connected succesfully message on its display (I run a raspbian desktop on it)

key msg:  dbus.Array([dbus.Byte(0), dbus.Byte(0), dbus.Byte(0), dbus.Byte(0), dbus.Byte(0), dbus.Byte(0)], signature=dbus.Signature('y'))          │161 1 [0, 0, 0, 0, 0, 0, 0, 0] 0 23 0 0 0 0 0
ERROR:root:[Errno 107] Transport endpoint is not connected

What did I do wrong? What should I do?