aarons / mystic-cryptic Goto Github PK
View Code? Open in Web Editor NEWThis shell utility schedules backups and encrypts them
This shell utility schedules backups and encrypts them
A good name for this repo would be fun.
I'm torn between boring and descriptive or fun and whimsical. Since this is a hobby project, I'm leaning toward whimsy.
Other ideas?
This utility depends on cron
and find
having Full Disk Access in the users OSX settings panel.
This isn't a default setting, so they need to add it manually.
It would be good if setup.sh can detect this issue, then explain how to setup the OSX environment properly.
Passing in the full paths to directories and files is a bit of a pain.
A user interface would be pretty nice to make it clearer and easier to use.
Currently the user needs to run these shell scripts with things like ./backup.sh
An install to /usr/bin or similar would be nice, and a brew package would be pretty sweet once the kinks are worked out.
Inspired by this news item: https://news.ycombinator.com/item?id=36467400
It seems like a good idea to keep an independently plain-text but encrypted version of a 1password vault around.
Cron command:
/Users/aaron/code/services/mystic-cryptic/backup.sh -d "/Users/aaron/Documents/journal" -o "/Users/aaron/Library/Mobile Documents/com~apple~CloudDocs/backups/" >> /Users/aaron/code/services/mystic-cryptic/logs/log.txt
Log:
/Users/aaron/code/services/mystic-cryptic/backup.sh: line 73: cd: /Users/aaron/temp: No such file or directory
find: /Users/aaron/Documents/journal: Operation not permitted
/Users/aaron/code/services/mystic-cryptic/backup.sh: line 82: cat: command not found
/Users/aaron/code/services/mystic-cryptic/backup.sh: line 85: rm: command not found
/Users/aaron/code/services/mystic-cryptic/backup.sh: line 109: ../.env: No such file or directory
The current PATH variable in crontab:
PATH=/opt/homebrew/bin:/usr/bin
I think this needs a better scheduling mechanism than the user's crontab.
Since this is primarily targetted for mac use, it would be nice to have a launchd plist file that could be installed and call this program, and the list of backups could be stored somewhere (maybe the plist file, or a separate file in application support). This would be a more mac-native approach.
Some of the current issues:
The backup.sh and restore.sh have wonky / inefficient error handling.
For example:
# first zip the directory
zip --quiet --test --recurse-paths -0 $file_name $dir_path
# stop if there was an issue creating the file
if [ ! -f $file_name.zip ]; then
echo "$file_name.zip does not exist, something went wrong at the zip stage"
exit 1
fi
Let's say the file exists from a previous run, but zip failed because the output file was already there (this is a contrived example, but similar to something that happened when testing). The code would continue because the file was present, it wouldn't detect that zip actually had an exit 1
error.
Bash scripts must have a better way to detect that a called script exited successfully (exit 0). This would simplify the use of file checks to determine if a command completed successfully.
I tried archiving a code folder, and I forgot that it contains a bunch of LLM models (700gb worth, in fact... Yikes!). I want to archive the code but not the models.
To experiment, I used find
to filter out files over 100mb, and it worked well. But it would be ideal to let the user specify what they want to filter out. Need to think of a good interaction for this. The simplest might be allowing the user to specify a size only to filter out. Alternatively, it could be a size and a file-matching pattern like regular expressions. I don't want the backup script to be too complicated though.
Example backup.sh code that filters everything above 100mb:
# first zip the directory
# exclude files over 100mb (+100000k) in size
find "$dir_path" -type f -size +100000k > exclude_list.txt
zip --quiet --test --recurse-paths -0 $file_name $dir_path -x@exclude_list.txt
echo "files over 100mb excluded, here's the list:"
cat exclude_list.txt | while read line; do
echo "$line"
done
rm exclude_list.txt
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.