chriskuehl/fluffy

A file-sharing web app that doesn't suck.

fluffy: a file sharing web app that doesn't suck.

Overall build status
fluffy-server
fluffy

What is fluffy?

fluffy is a Flask-based web application that allows you to upload arbitrary files to the web. Once you upload the files, you get a link to the file which you can share.

The reference instance of fluffy is online at fluffy.cc. You can also run your own!

What isn't fluffy?

• fluffy isn't social. Files are given a long, random (unguessable) name. There's no upload feed or list of files.
• fluffy isn't intrusive. Your files aren't resized, compressed, stripped, or modified in any way.
• fluffy isn't annoying. A simple, modern page for uploading your files. No ads, no memes, and no comments.

Philosophy and motivation

fluffy was created out of frustration from seeing hundreds of files (mostly images) be lost or deleted over the years from popular image hosts such as imageshack which either deleted files or closed their doors entirely. Fluffy is designed so that it is easy to stop accepting uploads while still serving existing files, with the hope being that a "shut down" would involve no longer accepting uploads, but still continuing to serve existing uploads.

fluffy only handles uploading and storing your files. There's no database, and it's up to you to figure out how you serve the uploaded files. Once fluffy stores a file, it forgets about it.

This not only makes the code simple, but also makes maintenance easy. If you wish to stop accepting uploads, you can easily throw the existing uploads on S3 or any web server to ensure their continued availability.

This does make some features hard or impossible to implement, however, so if you want to do anything post-upload at the application level, fluffy probably isn't for you.

Storing files

fluffy hands off uploaded files to a storage backend, which is responsible for saving the file. The following backends are currently available:

• File. Storage on the local filesystem. You can easily serve these with any web server.
• Amazon S3. Storage on Amazon S3. You can serve these with S3 static websites or with CloudFront (if you want a CDN).

Writing a storage backend is dead simple and requires you to implement only a single method. The current backends are both about ten lines of code.

Serving files

fluffy won't serve your files, period. It's up to you to figure this part out. Depending on which backend you use, you may get it easily. For example, Amazon S3 makes it easy to serve uploaded files via the web.

Run your own fluffy

There's a public "reference implementation" of fluffy at fluffy.cc.

To host your own copy of fluffy, just adjust settings.py to your needs, being sure to uncomment whichever storage backend you wish to use. There's no database, so setup is very simple.

Once you've adjusted the configuration, you can deploy fluffy the way you deploy any Flask app. fluffy is tested with Python versions 3.5 and 3.6.

Command-line uploading tools

Two tools, fput and fpb, are provided. They can be installed with pip install fluffy and used from the command line. Use --help with either tool for more information.

Additionally, Debian packages for the command-line tools are available in the GitHub releases tab. These packages contain no binary components and should be compatible with most releases of Debian and Ubuntu.

Contributing, license, and credits

Contributions to fluffy are welcome! Send your pull requests or file an issue. Thanks for the help!

fluffy is released under the MIT license; see LICENSE for full details.

Running locally for development

To run fluffy during development, run make dev. You should now have fluffy running at http://localhost:5000.

FAQ

Why are there only certain languages in the dropdown? Can I add more?

Since it's just a normal <option> dropdown now, I didn't want to have all hundreds of languages that Pygments supports, as I thought that would make the UI worse for little benefit. Instead, currently there's a hand-picked list of languages that I thought were most popular (but it's definitely biased toward what I use!).

In the long term, I'd love to replace the dropdown with something smarter (maybe a JS dropdown with all the possible languages, featuring the most popular at the top, but with all available below, or with autocomplete or something).

In the medium term, definitely feel free to open an issue or send a PR to add another language. I'll happily merge it.

As a workaround, note that the "automatically detect" can detect languages not in the dropdown (but it's not very accurate much of the time, unfortunately). Additionally, if you use the CLI, you can pass -l <language> and use any language supported by Pygments.

Why are there only a few themes to choose from in the pastebin?

Mostly it's just lack of time to add more. If you have a Pygments theme you like, please open an issue or PR, I'll definitely help get it added.

Primarily the reasons are:

• There are a lot of Pygments themes and many of them are (imo) low-quality or extremely similar. I thought it was better to hand-curate them and have a small-ish number, rather than a large number where it's hard to find the really good themes.

• It's not quite as easy as just adding the style name to a list. Fluffy also needs to know colors to use for borders, line numbers, highlights, diff highlights, hover versions of all of these, ANSI color codes that work, etc. There's something like 30 constants to set per theme.

• With the current implementation, every new theme bloats the CSS a little more. (There's probably a technical solution to this.)

fluffy uses awesome icon sets developed by FatCow.

fluffy's Issues

Default to monokai for ANSI color pastes?

I think it might be better to default to a dark theme (like monokai) for ANSI color pastes, they really look significantly better. Personally I'm always switching back-and-forth between default for text pastes and monokai for ansi color.

This may mean introducing a separate option for ansi color? I'm not really sure how to do this in a way that isn't confusing.

diffs highlight lines with a "-" as the first non-whitespace characters as a deletion

example:
https://i.fluffy.cc/95CBBSKm0FTfTdkJZpRkQpWG502w6VCS.html

the list items are mistakenly highlighted as deletions

Main link on http://fluffy.cc/ points to localhost:5000

Clicking big 'fluffy' on http://fluffy.cc/ links to localhost:5000, not prod url

Add some kind of rotating "Pro-Tip!" message on some pages

fluffy has a some neat features that aren't necessarily obvious, it'd be cool to be able to expose those better

examples:

• You can hit Ctrl-V to paste text or images on the home page that are on your clipboard
• There's a cli you can install
• You can click the line numbers on pastebin to highlight
• You can paste rendered Markdown
• You can add ?text to the end of the URL if you want to start at pb view

Pasted text gets CR LF (windows) newlines

1. Paste some text (multiple lines)
2. Click "Raw Text" and copy the URL
3. curl the URL into a file

The file now has CR LF newlines -- gross!

Probably because of http://stackoverflow.com/a/14217315/612279

I think it's reasonable to always normalize to unix newlines coming from the pastebin flow (we would never modify uploaded files, of course). A workaround for now is to upload the text as a plaintext file.

Add ability to set default server

I think the best way is reading from a JSON file. Probably check these two locations:

• /etc/fluffy.json
• ~/.config/fluffy.json

Option to configure background color?

This is a pretty dumb config option request (maybe there's a custom css option instead?)

but it would be nice to change the background color of fluffy - the specific use case I have in mind is to be able to quickly distinguish between the public and an internally hosted version of fluffy, to make it harder to paste things into the wrong fluffy :P

(side question: is there a docs page for settings.py? I poked around before making this ticket but couldn't see anything)

Thanks!!

Horizontal scrollbar covers text when scrolling on OS X

On OS X, the scrollbar is hidden except when scrolling (contrast with Linux/Windows where scrollbars are always visible, even when not scrolling). Unfortunately when scrolling horizontally, this means that the last line of text is covered on OS X, which can make it hard to read if you're scrolling while reading.

Maybe we should add some padding or something to the bottom on OS X? Ideally we'd want it to look something like it does on Linux, where the text is never covered:

Add customizable footer text

Deployments might want to link to internal runbooks or other docs.

Factor out jQuery

We probably don't need this dependency.

Public instance is very slow

On the public instance:

$ echo asdf | time fpb
[...]
fpb  0.30s user 0.04s system 11% cpu 2.848 total

On a certain company's internal instance:

$ echo asdf | time fpb
[...]
fpb  0.18s user 0.02s system 39% cpu 0.485 total

Something must be misconfigured on the public instance?

(This only applies for new uploads, not for viewing files -- on the public instance, uploaded files are served directly by S3 with CloudFront in front of it, so it's very fast.)

Transform tty color codes into HTML color markup

Often people pipe commands with color into fpb. The result isn't too good; the tty color sequences end up in the output and you see junk like:

'�[0;32m'
Got all modules, running make all
'�[1;37m'

It'd be cool to detect when these are present and remove them, replacing them with appropriate HTML color markup.

Make pastebin the default view?

Or at least provide an option, not sure which is better for fluffy.cc (probably still files?)

Buttons are not quite aligned on paste view (Firefox)

one of these is an input type="submit", the other is a link

they look okay on chrome, probably need more css on firefox

Is there a delete uploaded file option?

I uploaded file. I copy the URL. I can share said URL and anyone has access to the uploaded file. What do I do if I want to delete this file?

Thanks,

Dragging and dropping only works in Chrome.

Currently dragging/dropping works by covering the "drop files here" box with a hidden file input. This doesn't work reliably across browsers (although it does work perfectly in Chrome).

The original reason for wanting to use inputs instead of JavaScript events for capturing the files was so the submission could take place using normal forms. However, we use XHR anyway now so there's no longer any benefit to this.

There's no way to enforce an upload size limit

Support pasting rich text into web UI

It would be neat if you could paste rich text into the web UI and have the resulting paste keep the formatting.

Would need to somehow balance this against the "annoying" behavior that rich text paste can sometimes have (e.g. you copy from somewhere, but you don't necessarily want to keep the weird font it's using).

Pages scale poorly on small devices

Maybe simply adjusting the viewport would fix this, maybe we should add some responsive CSS.

Icons (small and large) are being served by the app, not the CDN

Use pygments-ansi-color package

I extracted the ansi color pygments highlighting code from fluffy into its own package: https://github.com/chriskuehl/pygments-ansi-color

Let's make fluffy use this package.

Add pastebin functionality

It would be quite nifty to provide pastebin-like functionality via fluffy. It would be fairly easy to create an HTML template for holding pastes, and generate this page as static HTML (with syntax highlighting and other bonus features handled either via JS or at page generation time) to be stored in the storage backend.

The hardest part would probably be adding the functionality to the home page without cluttering it.

Add support for "trusted networks" to identify special users

We should have the ability to have "trusted networks" which are treated as special users. The app can then be configured in one of two modes:

1. Anybody can upload
2. Only special users can upload

And we should add an option to allow (or not allow) special users to break the max file size limit.

contenteditable pastebin area allows modifications

some ways it is possible:

• middle-click pasting (linux)
• right-click and "paste"
• moving text (select, drag and drop)

hopefully can be resolved, since contenteditable makes selections a lot friendlier

Diff highlights don't work with horizontal scrolling

e.g. https://i.fluffy.cc/v1JrxknC65TGMQB4hFcTQhBtsmDm1t1W.html

cc @jvperrin

Add an option to line wrap text to fit in the viewport

For example, this is hard to read as-is, but would be much easier to read if it was line wrapped.

Attempting to upload a directory produces an "unknown error".

We should either allow directory uploads (if feasible) or produce a useful error message.

Folders can be queued for upload via drag-and-drop in Chrome and possibly in other ways as well.

Pygments style isn't great

Some lexers, such as diff, don't even appear with the current style.

Improve diff highlighting to understand multiple files

Here's an example diff with several different file types: https://i.fluffy.cc/4GH3s8gk5kvk6LZNLjgHSsHdnc7PQDsR.html

The autodetection often fails completely (like in the case above) or detects the wrong or a random file type, making the rest of the diff look bad.

It'd be cool to split the diff into chunks and highlight them individually. I'm thinking for the example above, these chunks would make sense:

diff --git a/ocfweb/middleware/errors.py b/ocfweb/middleware/errors.py
index e39319e..26dd093 100644
--- a/ocfweb/middleware/errors.py
+++ b/ocfweb/middleware/errors.py
@@ -15,10 +15,6 @@ class OcflibErrorMiddleware:

         if isinstance(exception, ResponseException):
             return exception.response
 
-        # maybe it's a real exception?
-        if settings.DEBUG or settings.TESTING:
-            return
-
         if isinstance(exception, Http404):
             # we don't care about reporting 404 errors
             return

@@ -54,6 +50,10 @@ class OcflibErrorMiddleware:

             ))
         except Exception as ex:
             print(ex)  # just in case it errors again here
+            if blah:
+                do_foo(
+                    x, y, z,
+                )
             send_problem_report(dedent(
                 """\
                 An exception occured in ocfweb, but we errored trying to report it:

diff --git a/ocfweb/static/js/site.js b/ocfweb/static/js/site.js
index 4e48734..112e029 100644
--- a/ocfweb/static/js/site.js
+++ b/ocfweb/static/js/site.js
@@ -3,16 +3,19 @@ $(document).ready(function() {

     var resizeTimeout;
     var updateFooterHeight = function() {
         var height = $('.ocf-footer').outerHeight();
-        $('body').css('margin-bottom', height);
+            $('body').css('margin-bottom', height);
     };
 
     $(window).resize(function() {
         clearTimeout(resizeTimeout);
-        setTimeout(updateFooterHeight, 10);
     });
 
     updateFooterHeight();
 
+    $(document).thing(function() {
+        return 'lol nope';
+    });
+
     // Close dropdown menu if clicked/tapped outside of
     $(document).click(function(event) {
         var menuOpen = $(".navbar-collapse").hasClass("in");

diff --git a/tox.ini b/tox.ini
index 9553550..4244304 100644
--- a/tox.ini
+++ b/tox.ini
@@ -3,6 +3,10 @@ max-line-length = 119

 # flake8 complains about 'redefinition' of imported pytest fixtures
 ignore = F811
 
+[some-new-section]
+# lol
+foo = blah
+
 [pep8]
 # autopep8 will rewrite lines to be shorter, even though we raised the length
 ignore = E501

Understanding diffs is a bit hard, but supporting the git diff format and maybe diff -u would be a good start.

Update docker image to stretch

Attempt to parse shebangs using identify

This would greatly improve on pygments' ability to guess lexers when a shebang is present.
https://github.com/chriskuehl/identify

Anchors could be encoded better

Currently if you select lots of sequential lines, it looks like:

#L12,L13,L14,L15,L16,L17,L18,L19,L20,L21,L22,L23,L24,L25,L26,L27,L28,L30,L31,L32,L33,L34,L35,L36,L37,L38,L39,L40

maybe we could simplify this?

#L12-L28,L30-L40

Uploading is impossible without JavaScript or on old browsers.

There should be a "dumb" upload form which simply provides a standard form and file input for uploading single files when JS is disabled or an old browser is in use.

allow assets to be installed in places other than s3

e.g. on the filesystem

Add ability to limit uploading to trusted users

Not obvious what files to delete

In some cases the administrator needs to delete a file (e.g. if PII was accidentally posted or something). Each upload corresponds to 1 or more S3 objects, and it's not obvious what all of them are.

For example, when uploading a text file, there are actually three objects created:

• The file details view
• The file itself (plaintext)
• A pastebin of the file ("view text")

Let's expose these somehow. One easy option might be to add headers to the S3 objects during upload linking to the other objects. For the filesystem backend it's harder, but we could probably add links in the HTML pages at least (probably nothing we can do about the actual object).

Use Content-Disposition header to force downloads to use the original filename

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Disposition

I think we could send this on direct file downloads (probably just for non-text things, since for "raw" you expect to get the text, not a download of a text file) with the original filename. I think this is a much better UX than downloading a file with a random name and the original extension.

plain diffs are highlighted poorly

they should get the special treatment

files with multiple extensions lose an extension (.tar.gz -> .gz)

files with multiple extensions are not served properly -- they lose an extension

example:

https://i.fluffy.cc/L6j2RJm6QR5LRLThdSQ4Ddd9X7Zv76JM.html

fpb/fput should have an option to go directly to the raw file

I'd like to use something like fput --raw <somefile> and have it give me a link to the file itself. Right now, it can only produce a link to the file details page.

Similarly, fpb --raw would produce a link to the plaintext version. Annoyingly, fpb -r already exists, maybe we can find a better name than raw?

Ideally we'd probably add support to the server for this, since there's no point in uploading multiple views (e.g. file details, pastebin view) when only providing the raw link, as nobody will ever be able to find them.

Improve public instance deploy process

Currently it's very manual: build a Docker image, restart it. There's also the step of uploading prod assets to the bucket, which is easy to miss and breaks the site in subtle ways; I just forgot to do this again, and so JS was broken for a few hours :(

Keyboard shortcuts (Cmd-C, ...) don't work on OS X on paste view

@bwbaugh reports that he can't use shortcuts like Cmd-C in the paste view page on Chrome OS X (it works for me on both FF + Chrome on Linux).

curl a fluffly link should give you the raw content (no html)

That would be neat!

Uploads on Firefox never reach 100% progress.

Firefox doesn't seem to ever receive the 100% progress notification, so the page waits at a lower progress until the file gets stored. Uploading files works fine, but it's confusing to wait there are ~90% for a few seconds.

This isn't a problem on Chrome.

pasting source code with filesystem file store causes 500 error

i forget if there is a workaround for this.

if pasting source code using the filesystem backend (instead of s3) and "Guess the language for me" is selected, the server throws a 500.

Replace awscli call with boto

Now that boto supports Python 3, it looks like we could replace the call to external s3cli with boto.

Store group upload detail pages in storage backend

Currently uploading a group of files results takes the user to a very long URL, like this:

http://fluffy.cc/details/zeJw1zVtuozAAQNGtVHxXVSAP019ADQKMBowwIorQYB6GEAdsFyirrzLjLODce7lonkUcyCYHnLZ-TXbQP94tK_oaEURLs9D0Y_zR3t-08i-51awS2vX97aKZrssZ5JEbejhq14ovA_LjNqlwmvu8pAqRB5P1KouRP0gtxIMrPkSNuT_6e0BOFCTpahD3TEF2sD4pt-jXH8WLomOdLIr_yOhF5QVZWO1InrC898IJhXujNBPbNfw5Uej-qOpBfXjagGpsfRTnUTBYN7OXn9vo4A3YCE29_JD3cXgiUUvZsVY8G__k0UZZi3VIZxQFEKYHTjijG_BhkNByN6vZN3-tyDDC8tzgQPBVbNQFN1PcdIRiPSbT6I4vILuXAIf7ucU2MKZoxrF10m3JBguDrN1cGYaeEnNXL0rozpBHMg8cHdhLaCAYyFw2IMO7NTORA5RYRNtp1-sv4aCZBg%3D%3D

The list of file names is gzipped and encoded in the query string. This allows the page to act like a permalink without keeping an actual record of which files were uploaded together. However, it results in an obscenely long URL which is impractical for posting almost anywhere.

Instead we should generate the details page ahead of time (like we do with info pages) for each upload with more than one file, and store it in the storage backend. This will require adding additional responsibilities to the storage backend, but I believe it's a worthy compromise.

Detail pages can be easily generated the same way as info pages and stored similarly (as simple HTML files). It will be up to the backend how it stores these, but in general it should probably just store them the same way it stores info pages.

Uploading a file greater than the max file size is broken

Tried to upload a file greater than max file size on my phone, fluffy popped an error up and then got stuck on this screen:

http://imgur.com/a/sBR2C

The app shouldn't be able to serve static assets in prod

Document how the cli can be integrated in a friendly workflow

Probably how to set it up as a hotkey to take a screenshot + upload in OS X and maybe Linux.