mpult/rexit

Rexit - Liberate your Reddit Chats. This tool will export your reddit chats into a plethora of formats

Hi there! I'm Maximilian Pult

All rounder IT guy I currently work for a medium-sized business, im the System Administrator in charge of buissnes automation, infrastructure and Cybersecurity.

• 🔭 Employed as System Administrator
• 🌱 I’m currently learning resilitant/distributed applications.
• 🌱 I’m currently learning more about Kubernettes
• ⚙️ Mastering: .html,.css,.js,.ts, and more
• 💬 I'm mostly active within the Cyber Security, Vue.JS, Node.JS communities

📬 Get in Touch

• GitHub: Github
• E-mail: [email protected]

📚 Looking for my Resume? E-Mail me!

rexit's Issues

Exporting Saved posts

Implement User Options

Users should be able to configure the behaviour in a interactive UI style Crate

MFA/2FA Support

Motivations

Please add support for accounts with MFA/2FA enabled.

• Would you like to implement this feature? [y/n]
  I'm not sure I'd be able to (N).

Solution

Support for accounts with MFA/2FA enabled. Ideally, the CLI should ask for the 2FA code.

Alternatives

Using token based auth, but I was unable to figure out how this works.

libssl.so.3: cannot open shared object file

I downloaded the latest version of the linux build and ran it. I get

./rexit: error while loading shared libraries: libssl.so.3: cannot open shared object file: No such file or directory

I haven't done a lot of Rust stuff so I'm not sure where to continue.

Here's my system information

Add Exporting

This is a essential part of this project, thus it has a high priority.

• Implement method of internal data transfer to export function
• Implement exporting
  • TXT
  • JSON
  • CSV

Logging

Users should be let known what is happending using info messages. EG login suceeded etc.

Add next batch feature

Currently only the first batch of sync data is analysied, it needs to be determined if relavent data is present in the next batches. if yes then that timeline is appended to the initial sync data (maybe save to temp file), same goes for the state > events attribute

Polish main functionality

There are a few tidbits that need to be polished before it can be accepted as a release.

• #5
• Replace user ID with display name
• Set default log level as INFO
• Replace Image placeholder with link to image
• Refactor
• Add username & password login option

Add GitHub actions for CICD

"No such file or directory" on image download attempt

Bug description

Hello again, @MPult ! Looks like we got ourselves another bug here 🐞
Running with image downloads, getting fatal error like this:

Rexit git:(main) /Users/***/.cargo/bin/rexit messages --images --redact --formats csv,json,txt

2024-02-07 17:49 - INFO: Getting image: [REDACTED]
thread 'main' panicked at src/ReAPI/images.rs:66:68:
called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: "No such file or directory" }
stack backtrace:
   0:        0x1005b5a60 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hb1669d376cd16fb9
   1:        0x1005c3f30 - core::fmt::write::hfb70cbdb2260ac51
   2:        0x1005aaf30 - std::io::Write::write_fmt::h70ee144642a582c5
   3:        0x1005b589c - std::sys_common::backtrace::print::h404168d6289bebbe
   4:        0x1005a1c90 - std::panicking::default_hook::{{closure}}::hcad9841f2663dad9
   5:        0x1005a19e8 - std::panicking::default_hook::he7ca7c46dc216a28
   6:        0x1005a2154 - std::panicking::rust_panic_with_hook::h501076dcf33dacb0
   7:        0x1005b5f28 - std::panicking::begin_panic_handler::{{closure}}::h0665b0808da4d87c
   8:        0x1005b5c68 - std::sys_common::backtrace::__rust_end_short_backtrace::h6d7b13146df55e47
   9:        0x1005a1eec - _rust_begin_unwind
  10:        0x1005e8408 - core::panicking::panic_fmt::h5de4b603c189570c
  11:        0x1005e8750 - core::result::unwrap_failed::h6c2edae44e6d47ca
  12:        0x100289810 - rexit::ReAPI::images::get_image::{{closure}}::h71ed81371e6bd8e5
  13:        0x10028d634 - rexit::ReAPI::rooms::download_rooms::{{closure}}::hc29a397cde86e3d3
  14:        0x100290164 - rexit::main::{{closure}}::h5ce55bb2c365ebb1
  15:        0x100287928 - tokio::runtime::park::CachedParkThread::block_on::hf8573b3dd26c461a
  16:        0x1002ecfa8 - tokio::runtime::context::runtime::enter_runtime::he2113a63a4b0b715
  17:        0x1002d6ac8 - tokio::runtime::runtime::Runtime::block_on::h51c10d5419defe39
  18:        0x1002cbd94 - rexit::main::h22180b121616fe83
  19:        0x1002e8068 - std::sys_common::backtrace::__rust_begin_short_backtrace::h38bfbb5c4b2d170f
  20:        0x1002e56a4 - std::rt::lang_start::{{closure}}::h675f2af4ce139a66
  21:        0x1005a1ddc - std::panicking::try::h8fbeda15452e3045
  22:        0x100598700 - std::rt::lang_start_internal::h74e1b60c5b4153f8
  23:        0x1002cbe94 - _main

• Would you like to work on a fix? [y/n]

To Reproduce

Steps to reproduce the behavior:

1. ...
2. ...
3. ...
4. ...

this is on the main branch, with RUST_BACKTRACE=full

Expected behavior

Should finish normally

Screenshots

Environment

• OS: MacOS Sonoma 14.3
• {{project-name}} version: [e.g. 0.1.0]

Additional context

Would greatly appreciate a fix!

Add a --debug flag to accept https proxies that don't have valid certs

Allow setting a --debug flag to allow https proxies that don't have valid certs. This is very useful for debugging and it will be made very clear to the user that this flag is a security risk

Debug flag should export logs into a file

Improvement Suggestions

Sorry, @MPult , the template of feature suggestions did not make sense to me, so starting from a blank.

As someone with A LOT of messages and images, may I suggest the following:

1. It would be great to organize messages and images into subdirectories under /out with names corresponding to user names, such as /out/user_1 , /out/user_2 etc.
2. The sorting of messages is not intuitive as you'd expect the oldest to be on top. Yes, they can be resorted by timestamp, but not the txt format
3. Please add HTML format with links to images in the local folder (same folder as html file is suggestion No. 1 from this list is implemented

Thank you so much for all your hard work, @MPult , @oneElectron and whoever else has contributed!

Login Failed

Login Failed

This might be user error, but I only get "Login Failed". I have changed my password to something short and easy to enter since I can't see what I am typing into cmd prompt. I'm sure I'm typing it correctly, but it still fails. Any advice?

• Would you like to work on a fix? Not able.

To Reproduce

Steps to reproduce the behavior:

1. ...
2. ...
3. ...
4. ...

Expected behavior

Screenshots

Environment

Windows 10

Additional context

Error parsing array in rooms

Bug description

I'm new to the tool and just started using it. I successfully logged in using a bearer token because the username and password seemed to fail.

I do see the out folder after running but it is empty.

Maybe I'm doing something wrong that's obvious.

To Reproduce

Steps to reproduce the behavior:

1. RUST_BACKTRACE="full" rexit messages -t
2. entered the bearer token
3. confirmed the bearer token

Next I tried the saved posts and that seems to fail too.

Backtrace

2024-04-21 04:30 - INFO: Login Successful
thread 'main' panicked at src/ReAPI/rooms.rs:51:10:
Error parsing array
stack backtrace:
0: 0x10538b496 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h39dedd9998475f7a
1: 0x10539a7a3 - core::fmt::write::h996a70950ef02a8b
2: 0x10538d928 - std::io::Write::write_fmt::hd6709b73e8721808
3: 0x10538b27f - std::sys_common::backtrace::print::h0c71c33392329482
4: 0x10537ba25 - std::panicking::default_hook::{{closure}}::h7c2dc6602e4afdfe
5: 0x10537b7ef - std::panicking::default_hook::hb8432d7d4b595a9c
6: 0x10537be68 - std::panicking::rust_panic_with_hook::h55cf7f98a5172bf4
7: 0x10538bad6 - std::panicking::begin_panic_handler::{{closure}}::hfb4dd0b5a9ef6466
8: 0x10538b6f9 - std::sys_common::backtrace::__rust_end_short_backtrace::hcbbcf4c6d355f082
9: 0x10537bbc6 - _rust_begin_unwind
10: 0x1053c1325 - core::panicking::panic_fmt::ha2d9edabf12b6300
11: 0x1053c19ba - core::option::expect_failed::ha1c277500abbfd3f
12: 0x1050072a7 - rexit::main::{{closure}}::hf89d91bb37411eec
13: 0x104fff366 - tokio::runtime::park::CachedParkThread::block_on::h446c42231b05c1eb
14: 0x10504418d - tokio::runtime::context::runtime::enter_runtime::h9d9d4d925d5f4bd5
15: 0x105068c50 - tokio::runtime::runtime::Runtime::block_on::h80285ab1ffaa2f2f
16: 0x105051ded - rexit::main::h191a9d02c00e6598
17: 0x1050641f6 - std::sys_common::backtrace::__rust_begin_short_backtrace::h712d1604905064ce
18: 0x10506a2bc - std::rt::lang_start::{{closure}}::h07c8b02b45aa39de
19: 0x10537bac4 - std::panicking::try::h3e54f477477c1d0e
20: 0x1053656ce - std::rt::lang_start_internal::h5ac0ebe39c2c25d0
21: 0x105051ecc - _main

Screenshots

Environment

• OS: macOS 10.15.3
• rexit version: 1.2.0

Additional context

Please let me know if you need me to provide any further context

Add imgur download support for saved posts

Automated test

tests should use password flow and compare export to a known good

Correlate messages and images

As far as I can tell, there currently is no way to correlate messages and downloaded images.

The message often just says "Image" or includes a filename that is different from the download image.

Would be nice to be able to view the downloaded images "in context".

This seems to be true for all export formats.

Download Images

Add a CLI flag that will download the images.

Currently the mxc:// url is displayed as a message, but since this is a cumbersome thing to do Rexit should provide a function for this.

GET /_matrix/media/v3/download/{serverName}/{mediaId} is the URL SPEC

CI and Refactoring

Error - Missing field mimetype

C:\> rexit messages
> Your Reddit Username MyUserName
> Your Reddit Password ********
2023-09-16 12:03 - INFO: Login Successful
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error("missing field `mimetype`", line: 1, column: 9469)', C:\Users\user.op\.cargo\registry\src\index.crates.io-6f17d22bba15001f\rexit-1.2.0\src\ReAPI\messages.rs:75:33
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Version: 1.2.0

does not capture all of message chats

unfortunately it does not capture all of the messages.

it stopped in 11,001 lines

Improve docs

• Readme should only include information relevant for USER
• Secondary doc should explain the functioning of rexit with many links to the matrix spec. also info on developing etc.
• readme should show output formats