tiny-pilot/tinypilot

Use your Raspberry Pi as a browser-based KVM.

TinyPilot

Overview

Turn your Raspberry Pi into a browser-based KVM.

https://tinypilotkvm.com

Features

• Video capture (HDMI/DVI/VGA)
• Keyboard forwarding
• Mouse forwarding
• Fullscreen mode
• Paste text from clipboard

Official builds

TinyPilot official hardware packages give you everything you need to run TinyPilot and allows you to fund TinyPilot's development for future improvements.

TinyPilot Voyager 2a

Voyager 2a is TinyPilot's professional-grade KVM over IP device. Its quiet, compact design makes it a great fit for professional environments such as offices, data centers, and server rooms.

Build your own

• Raspberry Pi 4 (all variants work)
• HDMI to USB dongle
  • They have no brand name, and there are several variants, but they're all built on the same MacroSilicon 2109 chip.
  • They're available for $10-15 on eBay and AliExpress.
• 3 Amp power supply
• USB-C to USB-A cable (Male/Male)
• microSD card (Class 10, 8 GB or larger)
• HDMI to HDMI cable
  • Or [other] to HDMI, depending on how your target machine displays output.
• (Optional) A USB-C OTG split connector: Supports continuous power when the target computer turns off.
  • Requires two additional USB-A to microUSB cables and a 3 Amp power adapter.
  • If you're using this split connector, choose a USB-C to USB-A cable that's 12" or shorter to minimize voltage drop along the cable.
• (Optional) VGA to HDMI Adapter - If your target system has VGA output.
  • Alternatively, you can replace the VGA to HDMI adapter and HDMI to USB dongle with a VGA to USB adapter.

See "TinyPilot: Build a KVM Over IP for Under $100" for a more detailed tutorial on how to assemble these parts to create a TinyPilot.

Pre-requisites

• Raspberry Pi OS Bullseye (32-bit)
• python3-venv

Simple installation

You can install TinyPilot on a compatible Raspberry Pi in just two commands.

curl \
  --silent \
  --show-error \
  https://raw.githubusercontent.com/tiny-pilot/tinypilot/master/get-tinypilot.sh | \
    bash - && \
  sudo reboot

When your Pi reboots, you should be able to access TinyPilot by visiting your Pi hostname in the browser. For example, if your device is named raspberrypi:

• http://raspberrypi/

If you're using an HDMI to CSI capture chip (such as with a TinyPilot Voyager series device), see the additional configuration steps required for video capture.

Developer installation

See the CONTRIBUTING file.

Other installation options

• Advanced installation options

Diagnostics

If you're having trouble with TinyPilot, you can retrive logs from the web dashboard by clicking "Logs" in the bottom of the main dashboard.

If you can't access the web dashboard, you can retrieve the logs by SSHing into the device and running the following command:

sudo /opt/tinypilot-privileged/scripts/collect-debug-logs

This log is useful if you file a bug report.

You can read more details about the logs in the wiki.

Security considerations

TinyPilot does not support authentication or transport-level encryption. You should only use TinyPilot on networks that you trust. Anyone who accesses the TinyPilot URL can shutdown or restart your Pi and type arbitrary commands into the device to which your Pi is connected.

To use TinyPilot on untrusted networks, you can upgrade to TinyPilot Pro, which adds password-based authentication and TLS for end-to-end encryption.

As a free alternative, you can adjust your Nginx configuration (included by default with the installation) to require HTTP Basic Authentication and add a self-signed TLS certificate.

Detailed project updates

If you're interested in seeing what's happening with the project at a granular level, weekly updates appear every Friday on What Got Done:

• What Got Done: TinyPilot

See also

• TinyPilot Wiki: Guides for tasks related to TinyPilot.

Acknowledgments

TinyPilot would not be possible without the excellent and generous work from many open source projects, the most notable of which are listed below:

• uStreamer
• Flask and Flask-SocketIO
• vdesktop
• litestream
• Raspberry Pi
• nginx

Mailing list

For news about major TinyPilot releases and other updates about the project, sign up for the TinyPilot mailing list:

• TinyPilot Mailing List

Upgrade to Pro

TinyPilot Pro includes additional features for professional users, including:

• Virtual drive mounting and booting
• Password-based authentication

Support the project and upgrade to Pro at https://tinypilotkvm.com/product/tinypilot-pro.

tinypilot's Issues

Add support for multiple target systems

The pi might be a little underpowered for multiple simultaneous KVM sessions, but it would be nice to be able to attach more than one HDMI capture device and then connect to whichever one was desired at a particular moment by using multiple instances of tinypilot running on the same host (whether PI or other).

VGA capture

These KVM devices are very helpful in loud server rooms when a system has no LOM.

Has anyone tried using a VGA adapter? will it work?
I ask because many rackmount servers only have VGA.

perhaps add a small blurb regarding VGA to the README.

Well done. Work for an university and we love opensource <3

Video stream hangs after inactivity

A customer reported that the video stream freezes if the TinyPilot web interface is open for a long time.

Current workaround

Refreshing the page fixes the issue.

Repro steps

TBD

Add support for shutting off / restarting KVM Pi device

Add a button in the browser that allows the user to shut down or reboot the KVM Pi device (not the target device KVM Pi is connected to).

Reduce video stream lag

Currently, the lag is between 0.8s and 1.20s using ustreamer.

I recommend swapping it out with mjpg-streamer. As the HDMI-to-USB dongle is providing video in mjpeg format, mjpg streamer can stream it with notably lower lag. From my tests, i would say it is bellow 0.2s. Can you please check?

Installation steps for standalone server:

sudo apt update
sudo apt upgrade

sudo apt-get install build-essential libjpeg8-dev imagemagick libv4l-dev cmake git

mkdir ~/mjpg-streamer
cd ~/mjpg-streamer
 
git clone https://github.com/jacksonliam/mjpg-streamer.git
cd mjpg-streamer/mjpg-streamer-experimental
 
make
sudo make install

Start the video stream server by running:
/usr/local/bin/mjpg_streamer -i "input_uvc.so -f 30 -r 1920x1080" -o "output_http.so -w /usr/local/share/mjpg-streamer/www"

Yes, I was running it at 1080p at 30 fps. Didn't see any lag between my local and browser stream.

The stream runs on http://raspberrypi:8080/?action=stream

I played around with it for a bit and it seems to perform MUCH faster then ustream with nginx proxy.

Integrate keyboard disconnect reason with shutdown errors

Add support for connector to power switch interface on MOBO.

This will be a combination of hardware and software. My plan is to add software to drive a (configured) GPIO pin for each attached host low for a selected time (250ms, 1s, 5s, 10s, 20s) to effectively simulate the user pressing the power button.

This will require tying the R.Pi GPIO pin to the active side of the power switch connection on the MOBO (ideally with the chassis switch still attached via stackable headers) and the R.Pi GND to the GND side of the MOBO power switch connection. (by the same stackable header).

My plan is to create a tiny PCB that holds a stackable header and an additional right-angle header wired in parallel. The cable from the R.PI would go to the right angle header and the stackable would be pass-thru to the existing power switch in the chassis. At the R.Pi end, I plan to implement a 40-pin female header on a PCB with 4 right-angle 2-pin header connectors. Each 2-pin header connector would connect to 1 GPIO and GND. The PCB would be labeled with both the system number (corresponding to the software changes made as part of this issue) and the GPIO number (in case someone wanted to customize).

A connector plugged in backwards would not work, but will not harm the PC when tested. The connector can be reversed at either the right-angle connector inside the PC or at the PI to resolve the situation.

Nginx "Bad gateway" after restarting TinyPilot

There seems to be an issue where nginx will report "

Repro steps

1. Start a TinyPilot device normally
2. sudo service tinypilot restart
3. Visit http://tinypilot/

Expected

TinyPilot web interface loads normally.

Actual

Nginx reports "Bad Gateway"

Workaround

Restart nginx:

sudo service nginx restart

Gracefully handle non-response on /shutdown REST call

The server sometimes won't respond to the /shutdown call because it's shutting down...

The client side should assume that non-response from the /shutdown call means it was successful.

Video is a broken image

I got my hobbyist bundle in the mail. I fired it up and accessing the site results in a broken image. Looking at the inspector, the error is:

GET http://localhost:8000/stream?advance_headers=1 404 (NOT FOUND)

However, I'm able to change the port in the URL above to 8001 in another browser window and the video works fine.

I ran /opt/tinypilot/quick-install and after git log in /opt/tinypilot on the Pi returns the following as the top commit:

commit 7e67d9092d7f6e9e49b6520c85b448470ba8b82b (HEAD -> master, origin/master, origin/HEAD)
Author: Michael Lynch <[email protected]>
Date:   Mon Aug 3 17:02:48 2020 -0400

    Prevent history area from expanding when history is disabled (#110)

    This fixes an issue where the key history area would expand as the user types keys even if they had disabled key history. The keys wouldn't display, but the empty space expanding was distracting/confusing.

    Fixes #103

Also tried rebooting after running quick-install; same issue.

Thanks!

Add support for browser/OS-captured key combinations

KVM Pi needs to allow the user to manually send certain key combinations that the OS or browser capture before JavaScript on the page receives them.

• Ctrl+T
• Ctrl+W
• Ctrl+Alt+Del

Add ability to paste

It'd be handy to be able to paste in the contents of the clipboard into the remote machine (this is particularly useful for long, strong passwords which are a nuisance to type in).

Raspberry Pi 4 requirement

Did anybody try to run this on a Pi 3 instead ? Does it makes such a difference ?

Add instructions for setting up WiFi

Support typing into TinyPilot from a mobile device

TinyPilot's web interface mostly works on mobile but it was designed for desktop.

The big missing component is that there's nothing to prompt the mobile keyboard to open, so the user can't type. It shouldn't be too hard to add an <input> tag that's only visible on small viewports.

Add support for rebooting

Keyboard Connection Error: server is not an accepted origin

Hi again :)

My Rpi4 came in today and i followed the instructions on the blog post and can see the screen, however i can't use the keyboard.. It keeps coming up with a red box telling me there is a keyboard connection error (Transport Error)..

When i run "journalctl -f " i'm seeing :

INFO in server: http://192.168.2.11 is not an accepted origin.

I've set my hosts file to point the name "tinypilot" to my pi's IP address and that seems to allow it to work.. however it's not particularly useful if i have to change hosts files on all my machines...

I thought i had read in another post that this CORS stuff was fixed.. is there a way to fix it?

Update install instructions in README

Add confirmation before restarting / powering down

Add hardware requirements to README

History box expands even if unticked

Tested in Firefox and Chrome (both latest) on Windows 10.

Navigate to the page, untick the history, start typing.. the box expands anyway.

Gracefully time out when keystroke forwarding events fail

KVM Pi seems to hang if it fails to forward a keystroke event (e.g., when the USB cable is disconnected). This causes the entire web app to become unresponsive.

Restrict CORS origins for socket.io connections

In #98, I removed strict checking for CORS requests. My reasoning is that it's not providing meaningful protection at the moment, but it's creating significant usability issues (see #97).

CORS origin checking currently does not much protection because:

1. The user base of TinyPilot is so small that CSRF attacks are highly unlikely.
2. Even with CORS origin checking, third-party origins can still send requests to the /socket.io route from the user's browser. CORS enforcement prevents them from reading responses, but there's nothing useful in the responses anyway.

We should revisit this decision in the future to ensure that we're not sacrificing real protection by dropping origin checking, even if (1) is no longer true.

Perform software upgrade through the web interface

Currently the user can upgrade to the latest stable version by simply running SSH'ing in and running:

/opt/tinypilot/scripts/upgrade

But it would be better if the user could do this right from the web interface.

Notes

• We can retrieve the release tag in the installed version from git tag | tail -n 1
• We can retrieve the releases from Github using curl -H "Accept: application/vnd.github.v3+json" https://api.github.com/repos/mtlynch/tinypilot/releases

Design

Ansible

The first thing we need to do is move scripts/upgrade to /opt/tinypilot-privileged/update.

(note the change from update -> upgrade). Now that we have two version, I want to use "update" to refer to version-to-version updates and "upgrade" to refer to free -> Pro upgrade)

We can follow a similar pattern that we applied in:

tiny-pilot/ansible-role-tinypilot#68

Except for one difference: the tinypilot user should have sudoers access to /opt/tinypilot-privileged/update. The tinypilot user currently has sudoers access to the /sbin/shutdown binary, via this Ansible play.

We should keep scripts/upgrade instead of deleting it entirely, but change it to just print out an error saying, "This script has moved to /opt/tinypilot-privileged/update" and return -1 in case users have scripts that call this path.

Backend

Add three new routes:

• /api/version (GET)
  • Return the installed version according to the current git commit ID.
• /api/latestRelease (GET)
  • Return the latest version available on the Github repo according to the HEAD commit ID on master in the mtlynch/tinypilot repo.
• /api/update (POST)
  • Perform an update by running /opt/tinypilot-privileged/update under sudoer context.

Frontend

Add a menu item under "Options" "Commands" called "Update". When the user presses this option, a dialog should pop up. In the background, the frontend calls /api/version and /api/latestRelease.

If version != latestRelease the dialog should say, "An update is available. Would you like to install the latest version?" Giving options "Update" or "Cancel." If the user selects "Update," the frontend should call /api/update, and if that succeeds, /api/restart. During the update, the dialog should show a countdown timer from five minutes and show the message, "Please wait while TinyPilot updates."

If version == latestRelease, the dialog should say, "You are running the latest version of TinyPilot" and an "OK" button that closes the dialog.

This UI is overly simple for the first pass, and we can try adding more details later.

Wi-Fi hotspot support

It could be nice to add an hotspot option.
TinyPilot could then become standalone and portable.

• Connect the USB and HDMI to server
• Connect the computer via WIFI on TinyPilot's hotspot
• Access server remotely

Add support for "full screen" mode

This is more of an enhancement then a "issue".

It would be great to have an option to full-screen the HDMI stream. Some of the installers makes the display really small and not really visible (alpine linux being one of them...)

[Errno 2] No such file or directory when running "pip list --format=freeze" in quick-install

Hardware: Raspberry Pi 4 Model B Rev 1.1, Revision a03111
OS: Raspberry Pi OS (32-bit) Lite, up to date

Linux raspberrypi 5.4.51-v7l+ #1327 SMP Thu Jul 23 11:04:39 BST 2020 armv7l GNU/Linux

Somehow the quick-install fails with
fatal: [localhost]: FAILED! => {"changed": false, "cmd": "/opt/tinypilot/venv/bin/pip list --format=freeze", "msg": "[Errno 2] No such file or directory", "rc": 2}

I suspect ansible failing already:

Building wheels for collected packages: ansible
  Running setup.py bdist_wheel for ansible ... error
  Complete output from command /tmp/tmp.3iDxsDrTiL/venv/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-syk1vi37/ansible/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-a3rj78f2 --python-tag cp37:
  usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: -c --help [cmd1 cmd2 ...]
     or: -c --help-commands
     or: -c cmd --help

  error: invalid command 'bdist_wheel'

  ----------------------------------------
  Failed building wheel for ansible
  Running setup.py clean for ansible
Failed to build ansible

Full gist at
https://gist.github.com/Savasci/7f7de61a1ff684c16d19dadaaa980abb

Reduce logging

The TinyPilot flask server spams the logs frequently. We should make this configurable to reduce writes.

Add navbar

Update modifier key UI

The current modifier key UI is pretty ugly and unintuitive

Update the web UI to make the functionality obvious and visually pleasing.

Issue: Keyboard connection error

Hardware: RaspberrryPI 4B 8GB
OS: RaspberryOS 64bit

Display work flawlessly. installation script never installs the files to setup the hid, if installed manually get keyboard connection error message. Generates a Error -71 on a Linux host the tinypilot is connected to.

Support mouse input

Mouse input capturing is doable. We can use a really similar mechanism to what we're already doing to emulate a keyboard:

https://www.raspberrypi.org/forums/viewtopic.php?t=234495

• Possibly useful: https://github.com/Blub/netevent
  • Not sure if it supports the scenario we want where we're translating JavaScript events to HID mouse events.

Pi Zero W

The lower power requirements, smaller size, and lower heat of the Pi0 allow for an interesting possibility. The 0 could be totally powered through the OTG cable and with a otg adapter, the hdmi capture card could still be attached.

I'm picking up a P and a OTG adapter. I will report back.

Limit keystroke history to one row of keys

Are there any html files in the sytem i can customise?

I'd like to work on a theme for TinyPilot but i can't find the html files. are there any or are they all baked in ?

Suggestion: VNC server

It would be great if TinyPilot could expose a VNC server. I know it already exposes a web interface, but I use Apache Guacamole to manage my servers and it would be nice to expose TinyPilot using a standard protocol.

ARM64 is not supported.

Thanks for this great project

When installing on an arm64-based Raspian installation, I received the following error on installation (via curl):

Command "/tmp/tmp.0izQM6JOtJ/venv/bin/python3 -m pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-xlsfga9z --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple --extra-index-url https://www.piwheels.org/simple -- setuptools>=40.6.0 wheel "cffi>=1.8,!=1.11.3; platform_python_implementation != 'PyPy'"" failed with error code 1 in None

By disabling arm64 (arm_64bit=0), the installation succeeds as intended.

Possibility to use g_mass_storage module?

After digging into this a little bit (I don't have the hardware to test anything at the moment), it should be possible to use the g_mass_storage kernel module to also expose a block device or an ISO to the connected host.

Some limitations:

• It seems like that at the moment, without patches, it only supports acting as a CDROM, not DVD. Filesize limits are a problem.
• You might have to unload and reload the module to change the iso file path.

Could be interesting however.

Suggestion: KVM switch support

Maintainer edit from @mtlynch: Full list of compatible/incompatible KVMs is on the wiki.

---

Hi,

I use a commercial KVM-IPdevice connected to a regular (not cheap) KVM switch for controlling my home servers.
So, I'm interested if this project can support the connection to one HDMI based KVM switch.

Anyone has tested it?

Futhermore, please can you add support for an ICON in the WEB UI to send a custom user key press? The reason is to map the "switch" keystroke to a button, as actually the "Scroll Lock" is used in several KVM switches. For example the idea is to send this with a button: < SCROLL > + delay 10ms + < SCROLL > + delay 10ms + < F1 >.

I hope you want to consider it.
And congratulations for this great project! 👍
Regards.

Don't send repeated modifier keys

TinyPilot should apply special treatment when the user holds down a modifier key (Ctrl, Alt, Shift, OS). For a key like 'a', a user holding it down indicates they want to keep sending the 'a' keystroke, but holding down 'Shift' shouldn't send repeated 'Shift' keystrokes.

Repro steps

1. Open TinyPilot's web interface
2. Hold the Shift key (or any other modifier key)

Expected Behavior

TinyPilot generates a single keystroke for the Shift key.

Current Behavior

TinyPilot rapidly sends 'Shift' keystrokes as long as the user holds the key down.

Add support for ODroid / Raspberry Pi OS 64-bit

I believe the Micro USB port on the Odroid C2 is OTG... so do you think it would work?

Windows BitLocker Support

It seems that bitlocker (ms drive encryption) boots into some video mode which is not captured by the recommended (non-branded) HDMI capture device (or lost in the encoding).

Additionally the bitlocker screen times out and powers down about the time that the pi actually boots through to UI available. Just wanted to document somewhere that the bitlocker timeout can be disabled with the following command:

bcdedit /set {bootmgr} bootshutdowndisabled 1

Windows Details:

Windows Edition: Win 10 Enterprise 
Version: 1809
OS Build: 17763.1282

Add instructions for read-only filesystem

Add support for power/reset from DIY-IPMI

Leverage the reboot/power button functionality from https://github.com/Fmstrat/diy-ipmi

Relay board: https://www.amazon.com/dp/B0057OC5WK
GPIO cables: https://www.amazon.com/gp/product/B01BV2A54G
Wiring diagram: https://raw.githubusercontent.com/Fmstrat/diy-ipmi/master/Resources/Diagram.png

rebooting host, lose screen after bios screen

tiny pilot shows just a black screen.. no amount of rebooting the pi seems to bring it back.

Update screenshot in README

HDMI Bridge based on chip TC358743XBG

I assume your project might also work together with an HDMI Bridge to CSI-2 (Raspberry Pi camera port) based on the popular TC358743XBG chip as HDMI IN? https://auvidea.eu/b101-hdmi-to-csi-2-bridge-15-pin-fpc/
Or the B102 in my case (Raspberry Pi zero 22 pin port)

Ordered one of these and will be able to test it in a few days. Only problem might be the increased resolution which might slow it down even more when combined with a raspberry pi zero. But maybe there is a way to lower the resolution or lower FPS.

Add instructions for hardware

Add CSRF protection for device shutdown / restart

https://flask-wtf.readthedocs.io/en/stable/csrf.html

Add support for OS Key doubleclick

The way the modifier keys currently work is:

1. User pushes a modifier key button in the web UI
2. User presses a button on their physical keyboard

This causes TinyPilot to send the next key event with the modifier forced on (e.g., "Ctrl+T"), when it otherwise be captured by the browser.

It would be good to support a mechanism where the user can send just the modifier key. We should listen to the modifier keys dblclick events and send a key event of just the modifier key when that happens.