weee-open/pesto

Drive SMART data checker, cleaner and system image loader.

pesto's Issues

Add functionality to choose iso for cannolo

• Send list_iso /some/path or similar with the configured path
• Get a list of iso files
• In queued_cannolo send the disk and the iso path (enclose it in " (and escape any quote inside the string))

Display tarallo code in left table

Document env vars

Add shutdown_on_end

Shut down the server when nothing else is queued

Add ability to select local or remote mode

Add ability to select local or remote mode to include different use-related scenarios.

Automate tests

It's possible to test the server. Somehow. Automate the tests via GitHub CI. This requires writing some tests before.

Stop server threads/processes gracefully when terminating

Right now other processes block the main thread, or other threads are terminated randomly

Add entries for remote mode

Add IP and user entries in settings tab to establish ssh connection to the desired device.

Add ssh support

Use paramiko to make ssh sessions and get remote smartctl results

Update tarallo after badblocks

Create a requirements file and implement automatic dependency installation

Write tests for Tarallo implementation

• Write some tests to check the Tarallo implementation is working correctly in every possible case, i.e: the disk is already in the DB, the disk is not in the DB but is corrupted, the disk is not in the DB and is in working conditions, etc...

• Bonus:
  Make a "tests" directory which contains some small virtual disks (they need to be uploaded to GitHub, so <25MB!), e.g. made with VirtualBox, for testing.

Add even more buttons

There are some buttons, both directly mapped to a command and to a process, but not for all the needed ones, currently.

Always available:

• Refresh
• queued_smartctl
• queued_sleep

Available only when has_critical_mounts is False (gray out or something if True, see #38):

• queued_badblocks
• queued_cannolo
• "Procedura standard"
• "Procedura standard senza cannolo"

Refuse to do destructive operations if disk has a mountpoint

Instead of letting the user choose which disks to ignore formatting, which is an approach prone to errors, we can use e.g. lsblk to detect the disk where the root / partition is mounted, and some other reserved paths (e.g. /media, /mnt, /run), and automatically ignore them.
The only disks we want to format are /dev/sd*, excluding the system disk, which may not always be /dev/sda.

For every other disk containing a swap partition, we should turn the swapoff before formatting.

Quick launch of standard procedure

queued_badblocks, queued_smartctl, queued_sleep and optionally queued_cannolo. Add a single button or a similar quick way to do that.

Add WEEE Open theme

Low priority

Handle network errors when accessing tarallo

If tarallo is unreachable, a million exceptions with random messages are thrown from every corner of requests and pytarallo and all disks are skipped. We need more try-excepts.

Smart check + overwrite = crash

1. Do a smart check
2. Do another one on the same disk
3. Click yes to overwrite
4. Client crashes

Traceback (most recent call last):
  File "/S/Software/pesto/./pinolo.py", line 863, in gui_update
    self.smartTabs.text_boxes[tab].append(line)
AttributeError: 'SmartTabs' object has no attribute 'text_boxes'

And it also dumps core.

Pretty print disk and uploaded status in smartctl tab

Aside from raw smartctl output, pretty print status and uploaded status.
Make it more visible, or color code it, or something. Maybe make a table, or place the 2 bits of information on the same line in some way.
Related to #50 (pretty print that, too)

SSD trimming

To erase a SSD use TRIM with:
# blkdiscard -f /dev/sdX

Auto-terminate server if in local mode

Will be handy if the server could know that is running in a local configuration and, when the connection is closed, it must shutdown itself to avoid that the socket in use remain occupied in case of a client crash.

To make it, the client can run the process with the command:

python pesto_server.py --local

when in local mode, and in remote mode:

python pesto_server.py

Implement close_at_end

But how?

Bug in get_disks procedure on windows machine

In Windows machines there are no mountpoints and the server raise an exception:

Exception while scanning for disks, skipping
Traceback (most recent call last):
  File "C:\Users\Lol\Desktop\WEEE-Open\pesto\basilico.py", line 868, in scan_for_disks
    disks[path] = Disk(disk_lsblk, TARALLO)
  File "C:\Users\Lol\Desktop\WEEE-Open\pesto\basilico.py", line 31, in __init__
    self._mountpoint_map = self._lsblk["mountpoint_map"]
KeyError: 'mountpoint_map'
Exception while scanning for disks, skipping
Traceback (most recent call last):
  File "C:\Users\Lol\Desktop\WEEE-Open\pesto\basilico.py", line 868, in scan_for_disks
    disks[path] = Disk(disk_lsblk, TARALLO)
  File "C:\Users\Lol\Desktop\WEEE-Open\pesto\basilico.py", line 31, in __init__
    self._mountpoint_map = self._lsblk["mountpoint_map"]
KeyError: 'mountpoint_map'

On windows machines it will be sufficient to send a mountpoint flag only in C: and asking for every drive if the user is sure to erase the device.

Server does not detect new connected drives when it's running

Add GUI

Queue is not queued

Threads wait on a lock. But they don't wait in order! If 2 threads are queued, when the lock is released, which one will go is random and depends on the scheduler!

Save queue to disk

And reload on server restart.

Use the "stale" field to mark these entries if they were in any state other than "queued" and ignore them, don't do anything, client needs to show they're stale and the last state, if they were running they probably stopped halfway. Queued entries don't need to be stale and they can be started, if the disk still exists.

Add sleep commands

Maybe also put HDDs to sleep automatically

Server has to find the disks by itself based on the system

Now the client can get all the drives thanks to the server but it cannot know what os is running in the server to choose the function to get drives list. Instead the server can get the os in which it is running on and decide if use get_disks() or get_disks_win().

Add dark and vaporwave themes

Print date and time in smartctl output tab

Since multiple ones for the same disk can be shown, add a date and time.

They are ordered but doing it more than twice gets confusing otherwise.

Un-update tarallo before operations?

Should we set erased = no before erasing and similar?
It would be easier to detect if the piallaggio has completed or not or if it has failed or if the final update of tarallo succeeded or not.
Discuss.

Unswap disks

Call swapoff, see #15

Save queue to disk periodically

and reload it when starting

Add drop down menu for queue management

Right click to show drop down menu with:

• stop
• pause
• resume
• remove

Turbofresa integration, extreme edition

1. Connect via SSH as usual
2. Send commands to a unix socket or named pipe or something, instead of bash
3. The other end is connected to turbofresa.py
4. Turbofresa does the multithreaded stuff to handle multiple badblocks processes and sends notifications back and forth and calls smartctl

Implement a log

The log file must be saved somewhere in /var and configurable via the config file.
Make use of the logging library.

Add profile list for different SSH configurations

Display error_that_can_be_manually_fixed on client

When received, display it in a huge message box.

Review the piallaggio flow

See WEEE-Open/turbofresa#35 (comment)

Make the script configurable via config file

Stop badblocks if it's too slow

It means the disk is failing, see commented-out code

Split requirements.txt for client and server

Also write the right commands to install one or the other in the readme

Implement cannolo command

Add progress bar to badblocks

It would be a very nice, if not needed, improvement.
Suggestions:

• use tqdm in Python
• pass the badblocks progress bar through Python, if it has one

Add command to upload disk to tarallo

Only if it doesn't have a code. A button to do that is needed.

Add ficticious tarallo code for test mode

Client crashes on startup when there are no saved settings

Traceback (most recent call last):
  File "/S/Software/pesto/./pinolo.py", line 825, in main
    window = Ui(app)
  File "/S/Software/pesto/./pinolo.py", line 119, in __init__
    self.set_items_functions()
  File "/S/Software/pesto/./pinolo.py", line 147, in set_items_functions
    self.latest_conf()
  File "/S/Software/pesto/./pinolo.py", line 281, in latest_conf
    self.port = int(self.settings.value("remotePort"))
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

Ask confirm for piallaggio only if there are mountpoints (except [SWAP])

Should we do that?
The current method of "do not bother me again" answer is good for testing, but conditions users into immediately ignoring the message box, this is just a disaster waiting to happen when someone erases /dev/sda with no warning...

Make GUI resizable

Make some widgets fill available space instead of hardcoding every size (hardcode just some sizes)