dfirscience/efficient-nsrl

A filtered and split version of the National Software Reference Library (NSRL) for more efficient known-good DFIR filtering.

efficient-nsrl's Issues

UnicodeEncodeError

Receiving the following error when attempting to parse RDS 2.78 Modern Minimal from https://s3.amazonaws.com/rds.nsrl.nist.gov/RDS/current/rds_modernm.zip:

PS C:\Users\%USERNAME%\Downloads\Efficient-NSRL-master> python .\ensrl.py .\NSRLFile.txt ENSRL v0.1.0 - Use ctrl+c to exit [INFO] 2022-34-02T18:34:27 Creating ENSRL output directory. [INFO] 2022-34-02T18:34:27 Sorting NSRL... this will take a while. |████████████████████████████████████████| (!) 84961 in 16.1s (5285.44/s) Traceback (most recent call last): File "C:\Users\%USERNAME%\Downloads\Efficient-NSRL-master\ensrl.py", line 124, in <module> main() File "C:\Users\%USERNAME%\Downloads\Efficient-NSRL-master\ensrl.py", line 120, in main parseNSRL(NSRL,OUTD) File "C:\Users\%USERNAME%\Downloads\Efficient-NSRL-master\ensrl.py", line 82, in parseNSRL for l in alive_it(f): File "C:\Users\%USERNAME%\AppData\Local\Programs\Python\Python39\lib\site-packages\alive_progress\core\progress.py", line 466, in __iter__ for item in self._it: File "C:\Users\%USERNAME%\AppData\Local\Programs\Python\Python39\lib\encodings\cp1252.py", line 23, in decode return codecs.charmap_decode(input,self.errors,decoding_table)[0] UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 3998: character maps to <undefined>>

Checked python version:
PS C:\Users\%USERNAME%\Downloads\Efficient-NSRL-master> python -V Python 3.9.4

Not Compatible with RDS3 to RDS2 Convert?

I downloaded RDS_2023.03.1_modern_minimal.db from the NSRL and converted back to RDS2 via the pdf "RDSv3 to RDS 2.XX Format Text Files" provided by NIST. I created NSRLOS.txt and NSRLFile.txt in one directory.
Sadly, after running for a while, the ensrl.py script exits without error message. This is the debug output:

root@debian:/data/RDS_2023.03.1_modern_minimal/Efficient-NSRL# python3 ensrl.py ../NSRLFile.txt --debug
ENSRL v0.1.0 - Use ctrl+c to exit
[DEBUG] 2023-35-14T15:35:16 NSRL set to ../NSRLFile.txt
[INFO] 2023-35-14T15:35:16 Creating ENSRL output directory.
[DEBUG] 2023-35-14T15:35:16 Creating file ENSRL-Windows.txt on /data/RDS_2023.03.1_modern_minimal/Efficient-NSRL/ENSRL
[DEBUG] 2023-35-14T15:35:16 Creating file ENSRL-Mac.txt on /data/RDS_2023.03.1_modern_minimal/Efficient-NSRL/ENSRL
[DEBUG] 2023-35-14T15:35:16 Creating file ENSRL-Linux.txt on /data/RDS_2023.03.1_modern_minimal/Efficient-NSRL/ENSRL
[DEBUG] 2023-35-14T15:35:16 Creating file ENSRL-Android.txt on /data/RDS_2023.03.1_modern_minimal/Efficient-NSRL/ENSRL
[DEBUG] 2023-35-14T15:35:16 Creating file ENSRL-iOS.txt on /data/RDS_2023.03.1_modern_minimal/Efficient-NSRL/ENSRL
[DEBUG] 2023-35-14T15:35:16 Creating file ENSRL-Other.txt on /data/RDS_2023.03.1_modern_minimal/Efficient-NSRL/ENSRL
[INFO] 2023-35-14T15:35:16 Sorting NSRL... this will take a while.
|████████████████████████████████████████| 293889315 in 1:01:49.7 (79224.04/s)
[DEBUG] 2023-37-14T16:37:06 File <_io.TextIOWrapper name='/data/RDS_2023.03.1_modern_minimal/Efficient-NSRL/ENSRL/ENSRL-Windows.txt' mode='r' encoding='UTF-8'> is empty. Removing...
[DEBUG] 2023-37-14T16:37:06 File <_io.TextIOWrapper name='/data/RDS_2023.03.1_modern_minimal/Efficient-NSRL/ENSRL/ENSRL-Mac.txt' mode='r' encoding='UTF-8'> is empty. Removing...
[DEBUG] 2023-37-14T16:37:06 File <_io.TextIOWrapper name='/data/RDS_2023.03.1_modern_minimal/Efficient-NSRL/ENSRL/ENSRL-Linux.txt' mode='r' encoding='UTF-8'> is empty. Removing...
[DEBUG] 2023-37-14T16:37:06 File <_io.TextIOWrapper name='/data/RDS_2023.03.1_modern_minimal/Efficient-NSRL/ENSRL/ENSRL-Android.txt' mode='r' encoding='UTF-8'> is empty. Removing...
[DEBUG] 2023-37-14T16:37:06 File <_io.TextIOWrapper name='/data/RDS_2023.03.1_modern_minimal/Efficient-NSRL/ENSRL/ENSRL-iOS.txt' mode='r' encoding='UTF-8'> is empty. Removing...
[DEBUG] 2023-37-14T16:37:06 File <_io.TextIOWrapper name='/data/RDS_2023.03.1_modern_minimal/Efficient-NSRL/ENSRL/ENSRL-Other.txt' mode='r' encoding='UTF-8'> is empty. Removing...
root@debian: