ekzhang/rustpad

Efficient and minimal collaborative code editor, self-hosted, no database required

Rustpad

Rustpad is an efficient and minimal open-source collaborative text editor based on the operational transformation algorithm. It lets users collaborate in real time while writing code in their browser. Rustpad is completely self-hosted and fits in a tiny Docker image, no database required.

rustpad.io

The server is written in Rust using the warp web server framework and the operational-transform library. We use wasm-bindgen to compile text operation logic to WebAssembly code, which runs in the browser. The frontend is written in TypeScript using React and interfaces with Monaco, the text editor that powers VS Code.

Architecturally, client-side code communicates via WebSocket with a central server that stores in-memory data structures. This makes the editor very fast, allows us to avoid provisioning a database, and makes testing much easier. The tradeoff is that documents are transient and lost between server restarts, or after 24 hours of inactivity.

Development setup

To run this application, you need to install Rust, wasm-pack, and Node.js. Then, build the WebAssembly portion of the app:

wasm-pack build --target web rustpad-wasm

When that is complete, you can install dependencies for the frontend React application:

npm install

Next, compile and run the backend web server:

cargo run

While the backend is running, open another shell and run the following command to start the frontend portion.

npm run dev

This command will open a browser window to http://localhost:3000, with hot reloading on changes.

Testing

To run integration tests for the server, use the standard cargo test command. For the WebAssembly component, you can run tests in a headless browser with

wasm-pack test --chrome --headless rustpad-wasm

Configuration

Although the default behavior of Rustpad is to store documents solely in memory and collect garbage after 24 hours of inactivity, this can be configured by setting the appropriate variables. The application server looks for the following environment variables on startup:

• EXPIRY_DAYS: An integer corresponding to the number of days that inactive documents are kept in memory before being garbage collected by the server (default 1 day).
• SQLITE_URI: A SQLite connection string used for persistence. If provided, Rustpad will snapshot document contents to a local file, which enables them to be retained between server restarts and after their in-memory data structures expire. (When deploying a Docker container, this should point to the path of a mounted volume.)
• PORT: Which local port to listen for HTTP connections on (defaults to 3030).
• RUST_LOG: Directives that control application logging, see the env_logger docs for more information.

Deployment

Rustpad is distributed as a single 6 MB Docker image, which is built automatically from the Dockerfile in this repository. You can pull the latest version of this image from Docker Hub. It has multi-platform support for linux/amd64 and linux/arm64.

docker pull ekzhang/rustpad

(You can also manually build this image with docker build -t rustpad . in the project root directory.) To run locally, execute the following command, then open http://localhost:3030 in your browser.

docker run --rm -dp 3030:3030 ekzhang/rustpad

We deploy a public instance of this image using Fly.io.

In the media

• July 11, 2021: Featured in Console 61 - The open-source newsletter.
• June 5, 2021: Front-page Hacker News post. Reddit discussions in r/rust and r/programming.

^{All code is licensed under the MIT license.}

rustpad's Issues

Dark mode

A dark mode, possibly based on the prefers-color-scheme media query, would be nice! Given that this uses Monaco, leveraging some existing theme should be fairly straightforward.

Zero-knowledge server

This is a feature request.

From a privacy point of view, it would be nice if the server could have no knowledge of the content.
Does the server need to know the content for real-time collaboration?
If zero-knowledge during edition is not possible, could it be possible when sharing a read-only version of the document?

PrivateBin (a zero-knowledge pastebin) is using the URL #hash after a GET parameter to store the key then some JavaScript to encode/decode it.

Thank you,

Third-party websocket client

I would love to use rustpad for my hardware live coding project.
Is it possible to connect a third-party client via WebSocket to rustpad?
If you can point me to any examples or resources it would be great.

Thank you.

Seeing two cursors when more than one user is connected

Hi,

I am seeing two cursors when more than one user is connected while typing

Screen.Recording.2022-02-11.at.12.58.17.AM.mov

Save code files

Is there anyway to make rustpad save code files even after 24 hours?

There is a way to make code files still available after server restart (using docker volumes), but in order to do that, may I know the directory where the code files are saved?

Record a code editing

For review interviewing require to record the interview. Maybe add this functional to this project?

filesystem persistence & urls

very cool project.

having rustpad pre-load from the filesystem would be neat, and occasionally to write that data back out. If you could run rustpad from inside an existing project, and have all the files you need already there that would be neat.

Output logs to Docker

Apologies if I'm barking up the wrong tree - but for some reason when I'm running this in a Docker container, I can't see any log messages in /var/log/docker.log relating to rustpad. The Rustpad web app itself works fine.

Context: I'm running the dockerized image straight from https://hub.docker.com/r/ekzhang/rustpad. The environment is Unraid v6.9.2 with Docker v20.10.5

Any way to integrate with oauth2?

I am looking at self hosting this backed by oauth2 login primarily google.
Is the capability supported currently?

Non docker install?

Do we have non docker install docs?

Bug/Discussion: Cursor-Position in collaborative coding

When using this I've stumbled upon an issue. I'm not sure if my suggested fix is actually always a good idea, or how easy it would be to solve, so this should probably be discussed.

If multiple persons work in different areas of a file, this can lead to conflicts, when the person that is closer to the beginning of the file inserts/deletes newlines, or with both in a single line just when it enters anything.

Because this will not change the position of the cursor of the other person in the file, meaning its cursor will move in relation to the context the person was working on, since that context is indirectly moved by the other person.

This can be fairly disrupting, if one was currently writing something, it might be spread over multiple lines or be... interlaced with the past characters.

The fix would be to move the cursor accordingly, depending on its current position. If its on the same line and past the other cursor, it has to be moved forward, if its simply below, it has to be moved down, if its above or generally before the other cursor, it doesn't get influenced.

Docker build fails

Hi,

I'm trying to build the image locally with the command docker build -t rustpad .. However it exits with the following error:

=> ERROR [wasm 6/6] RUN wasm-pack build --target web rustpad-wasm                                                                                                                                                                                                                                                                                                                                                   187.6s
------
 > [wasm 6/6] RUN wasm-pack build --target web rustpad-wasm:
#15 187.5 Error: Error during execution of `cargo metadata`:     Updating crates.io index
#15 187.5 
------
executor failed running [/bin/sh -c wasm-pack build --target web rustpad-wasm]: exit code: 1

Any clues how this could be solved?
Thanks!

Spell Checking

Hi there,

I've been searching for a way to enable (browser built-in) spell checking - to be able to use the LanguageTool extension in Rustpad, but I can't seem to find a way.

Couldn't find anything under F1 or shift-F10. Forcing spell check in about:config (Firefox) didn't do anything either.

Is there some way to enable browser spell checking – or could this be added as a feature?

Connecting to server issue when using reverse proxy

I'm using a reverse proxy for SSL and when I open rustpad, it always tell me that it's connecting to server, which it never does. Any help?

Using NGINX for reverse proxy
Using Cloudflare for SSL Provider

Trouble connecting with Nginx to Docker instance

I have my docker instance of rustpad running on port 3030, and it connects just fine when specifying the port number. However, when I attempt to use Nginx to reverse proxy, it's failing to sync to the server.

I see nothing indicating an error in the web socket connection. It does several retries and then ultimately fails. I also see no errors in the JS console or the docker log console.

Is it merely an issue with nginx not passing certain headers?

Feature request: save session in hard disk as a backup

since all the sessions are live in RAM and they're expire after 24 hour or end of it, there's some problems when exit that session accidentally, so i believe save sessions in hard disk as a backup is useful feature at least there's a history of these sessions listed there.

Ease-of-use: Upload and download of Files

Hey, I've used this Application a bit now, and I've got a few ideas.

As my Exams are coming up I will not be able to contribute much to these, but I wanted to record them here nonetheless.

First there were 2 things that I wondered about, that were already addressed, or likely out of scope, and that are persistent files/sessions (so it can work as a kind of pastebin) (it seems #6 addresses that), and multi-file workspaces are probably widely out of scope.

But I had another thought that should not be too hard to implement, and IMO would significantly increase ergonomics.

"File"-Upload and Download does not seem to be implemented, unless I missed it. The way I imagine it, is that one would be able to "Upload" a text-file via drag and drop, this would just paste in the content, and ideally also determine the Programming-language via file-extension. To download, I've seen websites that intercept the Ctrl-S command to save files (eg drawio). I'm not actually sure how they do it, but it appears there is a way, and that would be ideal IMAO.

Thanks, Sam.

State Persistence for volatile deployments

It appears that the current implementation keeps the state table in volatile memory (RAM) rather than committing it to disk. As such, deploying this in a scenario where the service's lifetime is not guaranteed (container orchestration, e.g. kubernetes) the state would be lost in an eviction scenario.

It would be nice if there was a way to offload the state table into a somewhat more persistent datastore or use something like sqlite with a pvc in kubernetes to help maintain state.

Sidebar should hide on a narrow display

It seems like there’s currently no way to collapse the left-hand sidebar (dark mode, language etc.). It takes up a constant amount of width of the display, so on a narrow window (mobile phone or split screen) the remaining space for actually writing text gets very limited if existent at all… The possibility to hide it would also benefit people with wide displays as it removes visual clutter.

Render markdown option

Would it be possible to enable an option when editing with Markdown, to render the markdown on the side? There are javascript libraries to do this, like md-block.

This would make this project a viable alternative to Etherpad which would be awesome because this uses orders of magnitude less RAM than Etherpad (being Javascript-based).

Read-only mode?

Not a bug, but a feature request. I LOVE the editor abilities of this, but would like the option to share it as read-only. I suppose an approach to having this work so that the author can still make changes, is perhaps a new link that references the original "tag", but is one that, when de-referenced, renders the code in a read-only console.

Or, if it can generate a new link "tag", of which it's read only? Not sure how it's implemented internally but either way, I would love this option if considered. Thanks.

Lapce integration

Hi.
I'm learning Rust by trying to extend the Lapce project.
I would love to see a function to start a Rustpad session by sharing a file inside Lapce.

Collaborative editing of local file?

Greetings

This project looks amazing! I was wondering: can Rustpad be used for 2 people to collaboratively edit a file on the server, the people using only they web browser?
My goal is to allow CSS tweeking on the server.

Thanks in advance!

Regards

URL constructor: rustpad_wasm_bg.wasm is not a valid URL.

Did a fresh clone, then:

wasm-pack build --target web rustpad-wasm
npm install
cargo run
npm run dev

At localhost:3000, I see the following error on the web console:

Uncaught (in promise) TypeError: URL constructor: rustpad_wasm_bg.wasm is not a valid URL.
    init rustpad_wasm.js:406
    <anonymous> index.tsx:9
[rustpad_wasm.js:406:16](http://localhost:3000/node_modules/rustpad-wasm/rustpad_wasm.js)

Nothing is displayed in the browser tab.

Docker container SSL certificates

Hey!

I'm using the given example command from README.md to run a rustpad instance on my server:

docker run --rm -dp 3030:3030 ekzhang/rustpad

But unfortunately I couldn't figure out how to enable SSL for the running instance. I have certificates on my server and I tried to give net access to the container but no luck. What am I supposed to do here?

I think It'd be nice to mention such case in the documentation.

Font size setting

Hi, thanks for this great tool ! )

Please consider adding font size setting (per-document, as for theme).
Current defaults looks too small.

Set <Editor/> fontSize as configurable or docker environment parameter

At present, for HiDPI displays, the editor fontsize is a little too small. After inspecting the source I see that the fontSize is hardcoded.

<Editor
    theme={darkMode ? "vs-dark" : "vs"}
    language={language}
    options={{
        automaticLayout: true,
        fontSize: 13,
    }}
    onMount={(editor) => setEditor(editor)}
/>

Is it possible to expose this fontSize setting as a Docker environment variable?

Latex syntax

Thank you for the amazing project

is there a way to add more syntax like Latex?

Support more editors with ace

https://github.com/ajaxorg/ace

Which is what rust-playground uses, using that can probably allow more keybindings like vim or emacs keys.

Rustpad breaks when adding Emoji

Hi, I have been using Rustpad for a little while now using a self-deployed instance. While collaborating with a few friends on a document today, we found out that inserting unicode emoji (e.g. by using the "Windows-Key"+"." shortcut, or pasting from somewhere else) breaks the document. Rustpad then tries reconnecting, fails and throws the "Desynchronized" error message.

The document cannot be edited afterwards anymore, so the existing content needs to be migrated to a new rustpad (minus the emoji). I noticed the same behaviour on the rustpad.io instance.

Would be great if you could take a look at this. Unfortunately I'm not of much help regarding Rust or JS. Thanks :)

How do I compile and build a production docker instance?

Given my very limited and sketchy knowledge of docker and rust, I'm not 100% sure where and which flags I can set to compile and build a docker image that is tagged production instead of a dev instance (its showing as "development" in the bottom left hand footer).

A little documentation on which parameters or flags that I can use as, say, a Docker environment variable would be lovely and much appreciated. Cheers.

UI: Minimize sidebar/toggle

Requesting a sidebar toggle or toggle for fullscreen. Love the app, but the sidebar makes navigation on mobile a bit tricky. Thanks!

Configuration option for Sqlite retention period?

It seems like when using Sqlite to retain pads beyond the memory storage limit they are stored indefinitely?

Would it be possible to add some configuration option to discard old pads after 30 days or so from the Sqlite storage?

Thanks!

Really nice project by the way!

Mobile Responsive support?

Hi, I really love your application because it's simplicity.
The application works great on large screen devices like computers and tablets. However, when I open it using my phone (iPhone 7), the left panel seem to cover all the screen that users like me can not see the text content.
Do you have any plan to add Mobile Responsive in the future?

Thank you and stay safe!

Problems running on Azure VM

Hi,

I'm trying to run this on an Azure VM via Docker. Docker is installed and when running the command:

docker run --name rustpad -p 80:80 -d ekzhang/rustpad

The container itself starts, but it's not available by the IP of the VM in the browser. Anything I'm missing?

Thanks.

Is it possible to run the code in the browser?

Love that this is written in Rust. I'm wondering if it's possible to run the code in the shared editor?. If so, how do you do that?

Save file

So, how to save the edited file? Where it save the file?

arm64 (aarch64) Support

Hey!

Given the technologies used in this project, and that I already run code-server just fine on it, this should be running just fine on an Raspberry-Pi (at least versions >=3).

With that in mind, and seeing that the official Docker-Image just works on amd64, I cloned the repository and tried around a bit.

Btw, from what I've seen its a nice code-basis. The tsx-Code (the JS part of the frontend) could use a few folders but otherwise real nice, insofar as I'm a good judge of that.

Well, I saw some good and some problematic stuff, but the problematic stuff should be possible to work around. The good stuff is that rustpad-server builds just fine.

The problem is that wasm-pack seems to REALLY not like arm64. I was able to fix some issues by installing wasm-pack via cargo install instead of the install script, and switching from rust:alpine with musl to the debian based rust:latest. After this (and loong build-times, even on my raspi 4 with an SSD, but thats Rust) it failed because of another binary wasm-pack looking for AFTER most of the compilation of the project - not wasm-pack but your rustpad-server-, wasm-opt. Now, I could fix that too, and I think it should be fine after this - I guess this does some kind of link-time-optimization? -, but I'm unsure what influence that might have on performance, and given how long it took to get here to start, and that this should be avoidable (see later), I think it would be great to solve this on an project-level.

Because given that the purpose of that step is to generate some architecture-independent wasm-"binary", and Rusts excellent cross-compilation support, I'm sure there is a way to just create

1. the architecture-independent wasm, js etc artifacts
2. and the architecture-dependent binary

In your pipeline and either at least distribute these artifacts, or better yet find a way to make the pipeline build an arm64 docker-image that also gets distributed via docker-hub.

After all once you have these artifacts all thats left to do is copy these into the image, switch the user and declare the entrypoint, all that in a simple "scratch" container.

Now, the reason I post this here, and don't just do it and make a PR, is that

1. I have no experience with docker-multiarch etc, so if someone has some comments on this/wants to help I'd love that
2. This would influence the pipeline quite a bit, and I wanted to check with you that such changes would even be accepted/ generally hear what your feeling is about all this.

Sam.

Client-Side UI Configuration Options

Hello team, excellent work on this, it fits my needs almost perfectly. There are three configuration options I'd like to see at client side that would complete this dream app.

1. As a user, l'd like to be able to change the fontSize.
2. As a user, l'd like to be able to change the fontFamily (in particular, JetBrains Mono),
3. As a user, I'd like to be able to select other color themes (in particular, Dracula).

Bonus brownie points:

1. If you could write some documentation on hub.docker.com so that noobs like me can follow. These folks, for e.g., write great documentation: linuxserver.io

Is it possible to run rustpad on redox-os?

Hi all! I like the redox-os operating system and the rustpad project, and I would like to know if it is possible to run rustpad on redox-os.

1. A question does rustpad work on redox-os?
2. Is it possible to run rustpad on redox-os?

Technical issue and not feature request.

rust playground integration

this would be so cool with rust playground integration

thank you very much

Support for authentication?

Hello, thanks for your great work!

Would it be difficult to support some basic form of authentication? I am a bit wary of deploying Rustpad on a production server in case it could be abused and affect the rest of the services running on that server.

I guess one could implement authentication by running Rustpad behind a proxy, but I thought that perhaps something simple (eg Basic authentication) could be implemented directly inside Rustpad to avoid the extra complexity.

Changing hour and day values for cleanup

Hey!

I saw that rustpad has fixed values for the cleanup intervals:

rustpad/rustpad-server/src/lib.rs

Lines 82 to 83 in 64d0b63

	const HOUR: Duration = Duration::from_secs(3600);
	const DAY: Duration = Duration::from_secs(24 * 3600);

I think it would be nice to make them configurable via dotenv or any other solution. Do you think it's applicable? If so, I'd like to submit a PR.

Feature request: Add an option to bypass loading monaco's resources from cdn

Awesome app.
However, the default cdn of monaco-editor is 'cdn.jsdelivr.net', which takes endless time to load in some areas (china specifically). It would be really nice to have an option to load monaco-editor's resources from other location.

I've found that monaco-editor/react did offer a loader option , which we can provide other path to load the editor's resources. reference 1, reference2. With this config, it's possible to load resources fom other faster cdn, or even load directly from the server where rustpad was deployed (would need to npm install monaco at first though)

I'd like to accomplish this request myself, but I'm not fimiliar with frontend frameworks. Also I deploy rustpad by docker, it's a nightmare to keep track your update and build my patched image.
Please help me.

Announcement: engineer-man/piston integration for code compilation/execution

Hello (using the issue tracker as a mailing list),

I've been looking for a replacement of the free trials of "online code interview tooling" to get both collaborative editing and the ability to compile/execute the common code. Piston provides the ability to self host a multi-language code execution service with a unified API so I thought it was a nice fit.

Therefore I've been working on expanding the Client-Server protocol and adding an extra React component to integrate Piston code execution API to rustpad.

This picture has been done testing with the public instance of piston (PISTON_URL=https://emkc.org/api/v2/piston cargo run)

For the time being it's on a branch that I happily force-push https://git.sr.ht/~gagbo/rustpad/log/gagbo/piston-integration, but if there's interest I can take some design and coding review to clean everything up and try to get it merged here. Whatever gets decided it's a nice project for me to get into front-end development so I'll keep making dubious changes to the UI/UX.

Editor: word wrap option

A word wrap option would be a nice add for the editor.

FreeBASIC Syntax highlighting

Hi, please add FreeBASIC Syntax highlighting.
https://freebasic.net/

Maybe this help?
https://github.com/KDE/syntax-highlighting/blob/master/data/syntax/freebasic.xml

Btw thanks for making Rustpad, this is really awesome :D

how long files in a link are available

how long files in a link are available? I mean life time.

Any plans of implementing text and voice chats?

Support for Notion-like comments?

While collaborating on an idea that we are sketching out, it is often nice to be able to highlight a chunk of code on it and leave a comment (not a code comment) that can begin a discussion thread.

This would be a pretty major and potentially orthogonal feature, but might be pretty amazing for use-cases like design docs, demos, and potentially even coding interviews, where tools like Rustpad may be most useful.

Of course, this would mean a bunch of parallel metadata for each document, and could get quite tricky in general (what happens when someone edits the borders of a commented piece of code while someone else is commenting it?) so it might be too hard / too much work.

What do you think? Achievable? Something you'd be excited to do?