vakabus/gimmedat

HTTP server for safely receiving files from friends (written in Rust)

gimmedat's Issues

implement browser based upload

Vulnerability: empty files can still fill the disk

At the moment, nothing prevents the anyone with a valid link from uploading unlimited number of empty files, which could lead to filesystem metadata space exhaustion. Could be fixed by limiting the number of allowed files.

implement file size checks

Workaround HTTP upload size limit

It is a bit annoying that HTTP proxies pretty much always have a request body size limit, which makes it impossible to upload large files. By default, nginx buffers the entire request (and Caddy probably too), so you want to have a reasonable limit. For nginx, it's probably a good idea to set proxy_buffering off and remove the size limit (for gimmedat location only).

However, splitting the large files in JS would simplify deployment (and avoid arguments with admins). Also, it would allow us to retry only the failed block, not entire file if the network fails. I can think of two ways of implementing it:

1. Splitting the file to predefined blocks if it's too large. Then uploading them as individual files, suffixed with -part001, ...

2. Alternatively, issue an "edit token" after a successful file upload with very limited validity (few minutes). The token would allow the user to append another block to the file.

• Option 1 is obviously easier to implement and has zero additional security risks.
• Option 1 requires the admin to manually merge the files.
• For a combination of low block size and large file, option 1 could produce directory with ~100k files (nginx default limit is 1M, 100G file isn't extreme IMO)
• Option 2 would allow us to add a delete button for accidentally uploaded files.
• Both options only solve the problem for JS uploaded, I didn't find a way to do it with curl.

I was thinking of implementing it, but I couldn't decide which way. What do you think?

Localization

Adding support for different web UI languages would be nice... Especially Czech.

Vulnerability: data limit checks can be bypassed due to a race condition

The upload size limit is checked before uploading any data. Therefore, it is possible to open lot of concurrent connections, let them pass the size check and then upload data over the given limit.

Can be fixed by continuously checking data allowance while uploading...

Mark partial or failed uploads

I think we should give the file .part extension during upload, and only rename it to it's real name after it's fully uploaded.

Having 4GB mp4 file might suggest I have got the video, but the upload could have failed at some point. I'll probably notice later that the video file is corrupted, but some other awesome formats are valid even when trimmed.

We could also delete files when the connection breaks, but there is certainly some failure mode which will leave the partial file on disk.

Link instead of list of uploaded files

Optionally providing a link (e.g. to a web gallery) instead of a list of uploaded files would be useful for integration with other services

Feature consideration: using arbitrary directories as upload destination

I am thinking about integration with other services running on the same host server...

Imagine, we have a web gallery for photos running on the server. It uses a simple directory as its datastore and anything you copy there will be available in the web interface when you load the webpage. (e.g. PiGallery). There is also gimmedat running on that server.

Is there a reasonably general and meaningful way, how to make gimmedat and the gallery work together? I want to be able to tell my friends: "if you upload the files, they will all be visible in the gallery on this link". And ideally, I would like to be able to set it up on my phone.

In other words, the goal would be:

• somehow make sure that the uploaded files are accessible from specified data directories elsewhere (not necessarily in cwd)
• configure the destination of files from the web UI

Questions:

• Isn't this out of scope?
• How to do it?

implement upload link expiration