revoltchat/autumn

Pluggable file server micro-service.

autumn's Issues

bug: Incorrect image dimensions for JPEG.

What happened?

Image: https://autumn.revolt.chat/attachments/hJbF5z9JZXthhuVgXa6SUYK7RBrvVtxfn5xbA1k0EJ
Attachment:

{
  "_id": "hJbF5z9JZXthhuVgXa6SUYK7RBrvVtxfn5xbA1k0EJ",
  "tag": "attachments",
  "filename": "IMG_2567.jpeg",
  "metadata": {
    "type": "Image",
    "width": 4032,
    "height": 3024
  },
  "content_type": "image/jpeg",
  "size": 1532809
}

The actual image is 3024x4032, but the attachment metadata on the message shows it being 4032x3024. (is this related to #21?) The image was taken on an iPhone 12.

bug: "thumbnail-ificiation" skipped on decode error

What happened?

Low priority issue tbh, but here we go.

It was noticed when adding an animated .webp for a channel icon, sometimes it would be animated and other times not. Even though the same method of conversion (some ffmpeg commands) from .gif to .webp was used between the different uploads.

I spent some time looking into what was happening and noticed that the assets that would be received as an animated .webp would not be correctly resized by Autumn. For example, look at these as a comparison.

Working as expected:

• https://autumn.revolt.chat/icons/UwEqmLzBLkMUs1kjb5iR4gFRmM8o9JjFtFtvEK3Mlu
• https://autumn.revolt.chat/icons/UwEqmLzBLkMUs1kjb5iR4gFRmM8o9JjFtFtvEK3Mlu?max_side=48

No working as expected:

• https://autumn.revolt.chat/icons/kVXj2K9pTpERDnZlwIsH30WAz5DTb_kB9Qyyeeu99F
• https://autumn.revolt.chat/icons/kVXj2K9pTpERDnZlwIsH30WAz5DTb_kB9Qyyeeu99F?max_side=48

This is literally caused by a decode error in the try_resize function, Error decoding Err(Decoding(DecodingError { format: Exact(WebP), underlying: Some(ChunkHeaderInvalid([82, 73, 70, 70])) })) and with the current implementation, if this resizing fails it sends the unaltered file.

This is literally the definition of a bug that is a feature 😆 This is an upstream issue, so might be fixed in the future but this does raise a question on error handling for clients and intended behaviour. I would recommend failing harder (500) which is more expressive to the user that something has gone wrong and would reduce the chances of somebody pulling their hair out as some work and some don't seemingly randomly. I've been playing around with an implementation, but it really is dependent on what behaviour you'd expect in this scenario, 500 or make exceptions for animated content allowing for animated channel icons etc.

Thanks,
GDWR - Griff#1126

Ability to mark attachment as a spoiler.

feat: introduce active threat protection

Configure some sort of anti-malware service that autumn can call out to for non-media files.

feat: don't strip orientation EXIF data

Images rotate upon being posted so that they are sideways, or upside down when the image is from the phone's camera roll. If the image is a screenshot however rotation does not happen.

support .mov as videos

Revolt currently doesn't embed videos in the .mov format.

AUTUMN_LOCAL_STORAGE_PATH crash when file is accessed

Hey,

I tried to setup S3 but I get always S3Error without any helpful message. So I saw in code support for local files using AUTUMN_LOCAL_STORAGE_PATH. It seems like writing files working but not reading.

thread 'actix-rt:worker:2' panicked at 'there is no reactor running, must be called from the context of a Tokio 1.x runtime', /home/rust/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.5.0/src/runtime/blocking/pool.rs:85:33
stack backtrace:
   0:          0x1032140 - std::backtrace_rs::backtrace::libunwind::trace::h6292875aed2739db
                               at /rustc/3f5aee2d5241139d808f4fdece0026603489afd1/library/std/src/../../backtrace/src/backtrace/libunwind.rs:90:5
   1:          0x1032140 - std::backtrace_rs::backtrace::trace_unsynchronized::h8e4cae471de489bb
                               at /rustc/3f5aee2d5241139d808f4fdece0026603489afd1/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:          0x1032140 - std::sys_common::backtrace::_print_fmt::h9b5c8993cc054166
                               at /rustc/3f5aee2d5241139d808f4fdece0026603489afd1/library/std/src/sys_common/backtrace.rs:67:5
   3:          0x1032140 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h025a584127ec484d
                               at /rustc/3f5aee2d5241139d808f4fdece0026603489afd1/library/std/src/sys_common/backtrace.rs:46:22
   4:          0x107825c - core::fmt::write::hc12d0803d7cd91f9
                               at /rustc/3f5aee2d5241139d808f4fdece0026603489afd1/library/core/src/fmt/mod.rs:1096:17
   5:          0x1029402 - std::io::Write::write_fmt::hf551e7bfd8a97193
                               at /rustc/3f5aee2d5241139d808f4fdece0026603489afd1/library/std/src/io/mod.rs:1568:15
   6:          0x1034465 - std::sys_common::backtrace::_print::h06e7e2e14f5705f1
                               at /rustc/3f5aee2d5241139d808f4fdece0026603489afd1/library/std/src/sys_common/backtrace.rs:49:5
   7:          0x1034465 - std::sys_common::backtrace::print::h0bc916dc7550b9e3
                               at /rustc/3f5aee2d5241139d808f4fdece0026603489afd1/library/std/src/sys_common/backtrace.rs:36:9
   8:          0x1034465 - std::panicking::default_hook::{{closure}}::h5ad01a85289feef4
                               at /rustc/3f5aee2d5241139d808f4fdece0026603489afd1/library/std/src/panicking.rs:208:50
   9:          0x1033fc3 - std::panicking::default_hook::h08254923b362a124
                               at /rustc/3f5aee2d5241139d808f4fdece0026603489afd1/library/std/src/panicking.rs:225:9
  10:          0x1034c01 - std::panicking::rust_panic_with_hook::hf455788adcc6037d
                               at /rustc/3f5aee2d5241139d808f4fdece0026603489afd1/library/std/src/panicking.rs:591:17
  11:          0x1034747 - std::panicking::begin_panic_handler::{{closure}}::h77f62bd790d73507
                               at /rustc/3f5aee2d5241139d808f4fdece0026603489afd1/library/std/src/panicking.rs:497:13
  12:          0x10325dc - std::sys_common::backtrace::__rust_end_short_backtrace::h425b40ec298ee3a1
                               at /rustc/3f5aee2d5241139d808f4fdece0026603489afd1/library/std/src/sys_common/backtrace.rs:141:18
  13:          0x10346a9 - rust_begin_unwind
                               at /rustc/3f5aee2d5241139d808f4fdece0026603489afd1/library/std/src/panicking.rs:493:5
  14:          0x1075961 - core::panicking::panic_fmt::h1a635ccd39b86574
                               at /rustc/3f5aee2d5241139d808f4fdece0026603489afd1/library/core/src/panicking.rs:92:14
  15:          0x10756f3 - core::option::expect_failed::ha87475f95863321c
                               at /rustc/3f5aee2d5241139d808f4fdece0026603489afd1/library/core/src/option.rs:1292:5
  16:           0x54e65f - tokio::runtime::blocking::pool::spawn_blocking::h948597f852164303
  17:           0x43b7aa - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h28a4e146992a07ba
  18:           0x44bacc - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h9086e5a8b0a1ea32
  19:           0x55885b - <actix_web::handler::HandlerServiceResponse<T,R> as core::future::future::Future>::poll::hf99f402f5e2ac2ad
  20:           0x582ee7 - <actix_web::handler::ExtractResponse<T,S> as core::future::future::Future>::poll::h00221b1dce674f00
  21:           0x4d5515 - <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll::h28fc1776b1efbcca
  22:           0x4d4cb9 - <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll::h6bd969c979888f5a
  23:           0xdc46c1 - <core::pin::Pin<P> as core::future::future::Future>::poll::he41b5249af7b482a
  24:           0x61442f - <futures_util::future::either::Either<A,B> as core::future::future::Future>::poll::h5be6bc862ca0dac2
  25:           0x43e2ad - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h42cc8f570f27bf98
  26:           0x58909d - <actix_web::middleware::logger::LoggerResponse<S,B> as core::future::future::Future>::poll::hd05829a343667aba
  27:           0x5b9269 - actix_http::h1::dispatcher::InnerDispatcher<T,S,B,X,U>::poll_response::hb39b55966b679030
  28:           0x52b384 - <actix_http::h1::dispatcher::Dispatcher<T,S,B,X,U> as core::future::future::Future>::poll::hd30b7a9c0dd75c84
  29:           0x4f67e1 - <actix_service::and_then::AndThenServiceResponse<A,B> as core::future::future::Future>::poll::h52b597d7835251cc
  30:           0x44c46f - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h984a60181ba545e6
  31:           0x46137a - <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h0badf3bccd13ff6f
  32:           0x4ecfdc - tokio::runtime::task::raw::poll::ha440b3bbfc95d746
  33:           0xfc95c3 - std::thread::local::LocalKey<T>::with::hb9b119d47c737a68
  34:           0xfe2a07 - tokio::task::local::LocalSet::tick::h50d53f76f886c843
  35:           0xf5731a - tokio::macros::scoped_tls::ScopedKey<T>::set::h2b8525658211642e
  36:           0xf5d74b - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h5b091028af80f405
  37:           0xf59f72 - std::thread::local::LocalKey<T>::with::hfa0971c26e7ac9e5
  38:           0xf574e5 - tokio::macros::scoped_tls::ScopedKey<T>::set::h82ceec5494a6f0fb
  39:           0xf5d648 - tokio::runtime::basic_scheduler::BasicScheduler<P>::block_on::hc3cac2796d9982f2
  40:           0xf445bf - tokio::runtime::context::enter::hed4c7f17f1bcdcb8
  41:           0xf59acf - std::sys_common::backtrace::__rust_begin_short_backtrace::hcfe22e4e5d1b5f66
  42:           0xf529c3 - core::ops::function::FnOnce::call_once{{vtable.shim}}::hfbc3c3a33de52672
  43:          0x103ca7a - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h04fa9632ed2971ec
                               at /rustc/3f5aee2d5241139d808f4fdece0026603489afd1/library/alloc/src/boxed.rs:1548:9
  44:          0x103ca7a - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h26f394c18ccce17c
                               at /rustc/3f5aee2d5241139d808f4fdece0026603489afd1/library/alloc/src/boxed.rs:1548:9
  45:          0x103ca7a - std::sys::unix::thread::Thread::new::thread_start::h3dc2fd766800d863
                               at /rustc/3f5aee2d5241139d808f4fdece0026603489afd1/library/std/src/sys/unix/thread.rs:71:17

Update README to be consistent across all projects.

Auto-create S3 buckets

feat: JPEG-XL, AVIF support

fix: support other database names

related issue: https://github.com/lightspeed-tv/backend/issues/3

issue: "cannot assign requested address and namespace" with MongoDB and credentials

What do you want to see?

My current understand is that Autumn does not currently support using a password or username in production.

I used the following redacted URI for Mongo, I believe this should work but recieved an error
AUTUMN_MONGO_URI="mongodb://username:password@localhost:27017/database"
cannot assign requested address and namespace (error 99)

While database authentication is certainly not needed, I'd appreciate its implementation

feature request: Embed player support for more audio types

What do you want to see?

I was testing Revolt and although i love it so far, one of the things i would like to be added is for the embed player to have support for more audio files and codecs like .wav, .ogg, .opus, etc. Apparently the embed player only supports .mp3 files so far

PWA

• Yes, this feature request is specific to the PWA.

feat: upload multiples files in attachments tag

As far as I see to upload files you have to make an individual request for each file, I think it would be better to allow uploading several files at once to avoid that.

Certain videos don't embed

insert told me to report this here

Link to video in question: https://fuckthefren.ch/sc/1629062006-video0_4-2.mp4

html execution

Allow configuration of S3 bucket names

At the moment, installation assumes that you are using Minio, and requires you to use hardcoded bucket names. This is a problem for anyone wanting to use a cloud storage service such as Amazon S3, since bucket names need to be unique.

Allowing bucket names to be configured will allow use with cloud hosts such as Amazon S3, and make storage much cheaper for many instances as a result.

feature request: flac audio embed

What do you want to see?

I would like to be able to embed flacs because most of my music is flacs

Block or warn clients when large content is about to be displayed.

Either prevent large media from being played back by recognizing it as a file or selectively block this on clients.
Consider adding to revolt.js instead of file server.

bug: Needlessly modifying images' ICC profiles

What happened?

Sending images via Revolt appears to strip ICC color profile information, which causes some images to display incorrectly. Upon sending an image like this to Revolt, the color information is stripped from the image & it ends up looking wrong. Discord only strips the ICC color information in previews, when pressing "view full image" it is actually intact. Additionally, most modern browsers support ICC color management.

I understand stripping metadata to preserve user privacy, but it is never harmful to keep the ICC color profile around to make sure the image's color information is displayed properly. Additionally, every iPhone takes images tagged with the P3 color profile. ICC color information can be preserved with exiftool by running exiftool -all= --icc_profile:all input.jpg.

Cron delete old files

Cannot seek in videos on Chrome.

Chrome expects the ability to receive partial content from a remote server.

https://stackoverflow.com/questions/10583931/cant-seek-html5-video-or-audio-in-chrome

Bug: Specially Crafted EXE Files May Bypass Virus Scanner

What happened?

Note: This is filed as a 'bug' rather than a security vulnerability because it's not in scope of the criteria set in SECURITY.md, as it requires client interaction to be dangerous.

---

Context

Currently the file server uses the mime type obtained from the tree_magic library to determine file type.

autumn/src/routes/upload.rs

Lines 62 to 68 in d4f4f72

	// ? Find the content-type of the data.
	let mut content_type = tree_magic::from_u8(&buf);
	// Intercept known file extensions with certain content types
	if content_type == "application/zip" && filename.to_lowercase().ends_with(".apk") {
	content_type = "application/vnd.android.package-archive".to_string();
	}

And the audio mime types are blindly accepted.

autumn/src/routes/upload.rs

Lines 215 to 220 in d4f4f72

	/* mp3 */ "audio/mpeg" |
	/* wav */ "audio/wav" |
	/* ogg */ "audio/x-vorbis+ogg" |
	/* opus */ "audio/x-opus+ogg" => {
	Metadata::Audio
	}

The tree_magic library used here uses the signatures from usr/share/mime/magic.

Problem

It's possible to manually craft an EXE that gets detected as a WAV file, and therefore be treated as an Audio file during upload.

Code:

// Load a file
let input: &[u8] = include_bytes!("hello.exe");

// Find the MIME type of the file
let result = tree_magic::from_u8(input);

// returns WAVE instead of EXE

Sample file: hello-world.zip

Just replace the file path in the code.

Cause

It's possible to alter the EXE header such that it's instead detected as a WAV file.

The modification is done at offset 0x8 of the file, by adding a WAVE text in ASCII, so the file is detected as audio/vnd.wave (audio/wav) rather than application/x-ms-ne-executable.

0:0000  4D 5A 90 00 03 00 00 00 57 41 56 45 FF FF 00 00  MZ........WAVE........

Due to the priorities either in tree_magic or the source magic file, the WAVE magic can take priority over the EXE Header. Thus the application sees it as a music track, and does not scan the file.

The modified EXE will still run, because we modified the DOS header, which is mostly ignored by Windows.
(I tested in Wine, and then on Win10)

feature request: Support for webp files

What do you want to see?

Support for webp files such as images would be nice for previewing

Scale animated avatars / banners on upload.

feat: sha256 hash of files and processed files

feat: blurhash of image files