binsonbuzz/unraid_rclone_mount

scripts to create rclone mounts on gdrive

unraid_rclone_mount's Issues

The mount is making unraid unable to stop array

I am using unraid 6.8.3

When clicking stop array, the status is stuck at Array Stopping Retry unmounting user share(s)...

there are two issues.

1. /mnt/user cannot be unmounted by the system. I have to manually do fusermount -uz /mnt/user
2. The script left an empty folder in /mnt/user0/mount_rclone. The system cannot delete it automatically and thus failed to umount /mnt/user0 as it is not empty.

Please see the log below
Feb 13 19:50:21 unraidServer emhttpd: shcmd (545): exit status: 32
Feb 13 19:50:21 unraidServer emhttpd: shcmd (546): rmdir /mnt/user
Feb 13 19:50:21 unraidServer root: rmdir: failed to remove '/mnt/user': Device or resource busy
Feb 13 19:50:21 unraidServer emhttpd: shcmd (546): exit status: 1
Feb 13 19:50:22 unraidServer emhttpd: shcmd (548): /usr/local/sbin/update_cron
Feb 13 19:50:22 unraidServer emhttpd: Retry unmounting user share(s)...
Feb 13 19:50:27 unraidServer emhttpd: shcmd (549): umount /mnt/user0
Feb 13 19:50:27 unraidServer root: umount: /mnt/user0: not mounted.
Feb 13 19:50:27 unraidServer emhttpd: shcmd (549): exit status: 32
Feb 13 19:50:27 unraidServer emhttpd: shcmd (550): rmdir /mnt/user0
Feb 13 19:50:27 unraidServer root: rmdir: failed to remove '/mnt/user0': Directory not empty
Feb 13 19:50:27 unraidServer emhttpd: shcmd (550): exit status: 1
Feb 13 19:50:27 unraidServer emhttpd: shcmd (551): umount /mnt/user
Feb 13 19:50:27 unraidServer root: umount: /mnt/user: target is busy.
Feb 13 19:50:27 unraidServer emhttpd: shcmd (551): exit status: 32
Feb 13 19:50:27 unraidServer emhttpd: shcmd (552): rmdir /mnt/user
Feb 13 19:50:27 unraidServer root: rmdir: failed to remove '/mnt/user': Device or resource busy
Feb 13 19:50:27 unraidServer emhttpd: shcmd (552): exit status: 1
Feb 13 19:50:27 unraidServer emhttpd: shcmd (554): /usr/local/sbin/update_cron
Feb 13 19:50:27 unraidServer emhttpd: Retry unmounting user share(s)...
Feb 13 19:50:32 unraidServer emhttpd: shcmd (555): umount /mnt/user0
Feb 13 19:50:32 unraidServer root: umount: /mnt/user0: not mounted.
Feb 13 19:50:32 unraidServer emhttpd: shcmd (555): exit status: 32
Feb 13 19:50:32 unraidServer emhttpd: shcmd (556): rmdir /mnt/user0
Feb 13 19:50:32 unraidServer root: rmdir: failed to remove '/mnt/user0': Directory not empty
Feb 13 19:50:32 unraidServer emhttpd: shcmd (556): exit status: 1
Feb 13 19:50:32 unraidServer emhttpd: shcmd (557): umount /mnt/user
Feb 13 19:50:32 unraidServer root: umount: /mnt/user: target is busy.
Feb 13 19:50:32 unraidServer emhttpd: shcmd (557): exit status: 32
Feb 13 19:50:32 unraidServer emhttpd: shcmd (558): rmdir /mnt/user
Feb 13 19:50:32 unraidServer root: rmdir: failed to remove '/mnt/user': Device or resource busy
Feb 13 19:50:32 unraidServer emhttpd: shcmd (558): exit status: 1
Feb 13 19:50:32 unraidServer emhttpd: shcmd (560): /usr/local/sbin/update_cron
Feb 13 19:50:32 unraidServer emhttpd: Retry unmounting user share(s)...
Feb 13 19:50:37 unraidServer emhttpd: shcmd (561): umount /mnt/user0
Feb 13 19:50:37 unraidServer root: umount: /mnt/user0: not mounted.
Feb 13 19:50:37 unraidServer emhttpd: shcmd (561): exit status: 32
Feb 13 19:50:37 unraidServer emhttpd: shcmd (562): rmdir /mnt/user0
Feb 13 19:50:37 unraidServer root: rmdir: failed to remove '/mnt/user0': Directory not empty
Feb 13 19:50:37 unraidServer emhttpd: shcmd (562): exit status: 1
Feb 13 19:50:37 unraidServer emhttpd: shcmd (563): umount /mnt/user
Feb 13 19:50:37 unraidServer root: umount: /mnt/user: target is busy.
Feb 13 19:50:37 unraidServer emhttpd: shcmd (563): exit status: 32
Feb 13 19:50:37 unraidServer emhttpd: shcmd (564): rmdir /mnt/user
Feb 13 19:50:37 unraidServer root: rmdir: failed to remove '/mnt/user': Device or resource busy
Feb 13 19:50:37 unraidServer emhttpd: shcmd (564): exit status: 1
Feb 13 19:50:37 unraidServer emhttpd: shcmd (566): /usr/local/sbin/update_cron
Feb 13 19:50:37 unraidServer emhttpd: Retry unmounting user share(s)...
Feb 13 19:50:42 unraidServer emhttpd: shcmd (567): umount /mnt/user0
Feb 13 19:50:42 unraidServer root: umount: /mnt/user0: not mounted.
Feb 13 19:50:42 unraidServer emhttpd: shcmd (567): exit status: 32
Feb 13 19:50:42 unraidServer emhttpd: shcmd (568): rmdir /mnt/user0
Feb 13 19:50:42 unraidServer root: rmdir: failed to remove '/mnt/user0': Directory not empty
Feb 13 19:50:42 unraidServer emhttpd: shcmd (568): exit status: 1
Feb 13 19:50:42 unraidServer emhttpd: shcmd (569): umount /mnt/user
Feb 13 19:50:42 unraidServer root: umount: /mnt/user: target is busy.
Feb 13 19:50:42 unraidServer emhttpd: shcmd (569): exit status: 32
Feb 13 19:50:42 unraidServer emhttpd: shcmd (570): rmdir /mnt/user
Feb 13 19:50:42 unraidServer root: rmdir: failed to remove '/mnt/user': Device or resource busy
Feb 13 19:50:42 unraidServer emhttpd: shcmd (570): exit status: 1
Feb 13 19:50:42 unraidServer emhttpd: shcmd (572): /usr/local/sbin/update_cron
Feb 13 19:50:42 unraidServer emhttpd: Retry unmounting user share(s)...

mount loses the connection after some time

Hello everyone, first of all I wanted to thank you for this great project.

I have installed the MergerFS plugin from the AppStore. Now I have the problem that the share is no longer up to date and remains in the state from the mount time.

If I execute the unmount script and execute the mount again I can avoid the problem, but this is also not a usable state.

Does anyone have any idea what the problem could be?

Works on Synology, with a couple small changes

I tried your script on my Synology NAS. It gave me a few errors but they were easy to fix. I already had rclone configured with a crypt mount working but I wanted mergerfs and your script has been, by FAR, the easiest way to get that working.

I had to copy my rclone.conf to /root/.config/rclone/rclone.conf, replace all instances of /mnt/user/ with a home path (just a preference), and I had to run the script with sudo.

You can see the changes in the attached file. I think if there was a path variable at the top with the others, it would make the script more universal.

Thanks for developing this, helped me out a lot!

unraid_rclone_mount.sh.txt

Getting Error when running mount script Unraid

Hey everyone has anyone seen this issue before, script has been working perfectly for a couple of years now, recently rebooted server and ran script to mount gdrvie. I am getting error regarding mergerfs not installed successfully -

pthread_setname_np
make[2]: *** [Makefile:125: build/fuse_loop.o] Error 1
make[2]: Leaving directory '/tmp/mergerfs/libfuse'
make[1]: *** [Makefile:104: objects] Error 2
make[1]: Leaving directory '/tmp/mergerfs/libfuse'
make: *** [Makefile:257: libfuse] Error 2
strip: 'build/mergerfs': No such file
/tmp/build-mergerfs: line 18: build/mergerfs: not found
cp: can't stat 'build/mergerfs': No such file or directory
mv: cannot stat '/mnt/user/appdata/other/rclone/mergerfs/mergerfs': No such file or directory
15.10.2023 10:53:10 INFO: *sleeping for 5 seconds
15.10.2023 10:53:15 ERROR: Mergerfs not installed successfully. Please check for errors. Exiting.
Script Finished Oct 15, 2023 10:53.15

Uninstall instructions

Hi,

If I want to remove this. what steps I need to do ? eg, how to uninstall mergerfs ?

tahnks

New Appdata Backup Plugin

I'm wondering if any changes will need to be made to account for the new Appdata Backup plugin for Unraid 6.12?

Setting LocalFilesShare to ignore causes a move error

Since LocalFilesShare is not a path the command below generates an invalid path.

unraid_rclone_mount/rclone_mount

Line 61 in 05689dd

LocalFilesLocation="$LocalFilesShare/$RcloneRemoteName" # Location for local files to be merged with rclone mount

Therefore, when using the invalid path the command below wont work

unraid_rclone_mount/rclone_mount

Line 211 in 05689dd

mv $MergerFSMountLocation $LocalFilesLocation

Logs:

05.12.2020 02:40:01 INFO: Not creating local folders as requested.
05.12.2020 02:40:01 INFO: Creating MergerFS folders.
05.12.2020 02:40:01 INFO: *** Starting mount of remote hai
05.12.2020 02:40:01 INFO: Checking if this script is already running.
05.12.2020 02:40:01 INFO: Script not running - proceeding.
05.12.2020 02:40:01 INFO: *** Checking if online
05.12.2020 02:40:02 PASSED: *** Internet online
05.12.2020 02:40:02 INFO: Success hai remote is already mounted.
05.12.2020 02:40:02 INFO: Mergerfs already installed, proceeding to create mergerfs mount
05.12.2020 02:40:02 INFO: Creating hai mergerfs mount.
mv: cannot move '/mnt/user/mount_mergerfs/hai' to 'ignore/hai': No such file or directory
* ERROR: unable to parse 'branches' - ignore/hai:/mnt/user/mount_rclone/hai
* ERROR: mountpoint not set
05.12.2020 02:40:02 INFO: Checking if hai mergerfs mount created.
05.12.2020 02:40:02 CRITICAL: hai mergerfs mount failed. Stopping dockers.
binhex-plexpass
binhex-rtorrentvpn
Script Finished Dec 05, 2020 02:40.02

Unraid 6.8.3 CA Script not running on Array start?

Am i missing something?

none of the scripts seem to run when i reboot my server, i have my array set to start automatically, and i have the script set to "At startup of array" but it never starts unless i start it manually.

I'm new to unraid so maybe i'm missing something simple?

Thanks!

Duplicate Folders

Hi! I'm running into an issue when I use this script to mount my GDrive inside of Unraid.

The folders that I'm trying to access in MergerFS seem to be duplicated. So my folder structure used to look like

/mnt/user/mount_mergerfs/gdrive_vfs/audiobooks
/mnt/user/mount_mergerfs/gdrive_vfs/tv
/mnt/user/mount_mergerfs/gdrive_vfs/movies
etc

It now appears like
/mnt/user/mount_mergerfs/gdrive_vfs/audiobooks
/mnt/user/mount_mergerfs/gdrive_vfs/audiobooks
/mnt/user/mount_mergerfs/gdrive_vfs/movies
/mnt/user/mount_mergerfs/gdrive_vfs/movies
/mnt/user/mount_mergerfs/gdrive_vfs/tv
/mnt/user/mount_mergerfs/gdrive_vfs/tv

Whenever I try to ssh into these folders, it will sometimes allow me in (to show me that both folders are identical). Whenever I try deleting or moving the folders, the changes affect my remote storage.

I've tried renaming my rclone mapping, however the duplication seems to carry over regardless of the name/folder.

How can I collapse these double folders into a single one? Or just remove one of these mappings so when I try to access /mnt/user/mount_mergerfs/gdrive_vfs/tv it only has 1 option to choose?

Or is it possible to fully remove any cached data and force rclone to pull down the folder structure from my remote storage? When I run "rclone lsd gdrive_vfs:" it populates a folder structure with single entries.

My dockers seem to be paralyzed by the choice of accessing 2 identically named folders.

Uploads fails do not understand why

Hey

I tried using these scripts today but the uploads failed for some reason. Could anyone assist? here is all the details of my problem:

https://www.reddit.com/r/unRAID/comments/q15ohc/need_help_setting_up_rclone_with_encrypted_gdrive/

Terrible issues setting up. 99% it's user error.

Retracted. For whatever reason, it just started working. No clue why.

My unraid log is filling up

Hello, i have a problem that is filling up my syslog
its about /mnt/user/mount_rclone

Can someone help me thanks

rclone mount script not working

Hi, I'm trying to move to these scripts from my current RClone config so I can use the Mergerfs but I'm having some issues.

My rclone config:

[bestvillamil]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
root_folder_id = XXX

[bestvillamilsecure]
type = crypt
remote = bestvillamil:secure
filename_encryption = standard
directory_name_encryption = true
password = XXX
password2 = XXX

[nowjon]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
root_folder_id = XXX

[nowjonsharedmedia]
type = crypt
remote = nowjon:/PDH/
filename_encryption = standard
password = XXX
password2 = XXX

My current mount script:

#!/bin/bash
#----------------------------------------------------------------------------

mkdir /mnt/disks/bestvillamil
mkdir /mnt/disks/bestvillamilsecure

mkdir /mnt/disks/nowjon
mkdir /mnt/disks/nowjonsharedmedia

rclone mount --max-read-ahead 1024k --allow-other -vv --drive-impersonate [email protected] bestvillamil: /mnt/disks/bestvillamil &

rclone mount
--allow-other
--cache-chunk-path=/mnt/cache/appdata/rClone/chunks
--cache-dir=/mnt/cache/appdata/rClone/vfs
--cache-db-path=/mnt/cache/appdata/rClone/db
--cache-info-age=60m
--cache-tmp-upload-path=/mnt/cache/appdata/rClone/upload
--cache-workers=10
--cache-writes
--checkers=10
--dir-cache-time=60m
--drive-chunk-size=64M
--drive-use-trash=false
--fast-list
--stats=0
bestvillamilsecure: /mnt/disks/bestvillamilsecure &

#Nowjon and Shared Storage
rclone mount --max-read-ahead 1024k --allow-other -vv nowjon: /mnt/disks/nowjon &
rclone mount --max-read-ahead 1024k --allow-other -vv nowjonsharedmedia: /mnt/disks/nowjonsharedmedia &

The script I edited, this doesn't have enough extra LocalFilesShare's, can I just add more?

#!/bin/bash

######################

Mount Script

######################

Version 0.96.7

######################

####### EDIT ONLY THESE SETTINGS #######

INSTRUCTIONS

1. Change the name of the rclone remote and shares to match your setup

2. NOTE: enter RcloneRemoteName WITHOUT ':'

3. Optional: include custom command and bind mount settings

4. Optional: include extra folders in mergerfs mount

REQUIRED SETTINGS

RcloneRemoteName="bestvillamilsecure" # Name of rclone remote mount WITHOUT ':'. NOTE: Choose your encrypted remote for sensitive data
RcloneMountShare="/mnt/disks/bestvillamilsecure" # where your rclone remote will be located without trailing slash e.g. /mnt/user/mount_rclone
LocalFilesShare="/mnt/user/Media/Downloads" # location of the local files and MountFolders you want to upload without trailing slash to rclone e.g. /mnt/user/local. Enter 'ignore' to disable
MergerfsMountShare="/mnt/disks/merge_bestvillamilsecure" # location without trailing slash e.g. /mnt/user/mount_mergerfs. Enter 'ignore' to disable
DockerStart="binhex-jellyfin binhex-plex binhex-emby binhex-delugevpn binhex-jackett sonarr radarr ombi" # list of dockers, separated by space, to start once mergerfs mount verified. Remember to disable AUTOSTART for dockers added in docker settings page
MountFolders={"Downloads/Deluge,Downloads/Deluge/Complete,Downloads/Deluge/Incomplete,Movies,TV Shows,Anime"} # comma separated list of folders to create within the mount

Note: Again - remember to NOT use ':' in your remote name above

OPTIONAL SETTINGS

Add extra paths to mergerfs mount in addition to LocalFilesShare

LocalFilesShare2="/mnt/user/Media/Movies" # without trailing slash e.g. /mnt/user/other__remote_mount/or_other_local_folder. Enter 'ignore' to disable
LocalFilesShare3="/mnt/user/Media/TV Shows"
LocalFilesShare4="/mnt/user/Media/Anime"

Add extra commands or filters

Command1="--rc"
Command2=""
Command3=""
Command4=""
Command5=""
Command6=""
Command7=""
Command8=""

CreateBindMount="N" # Y/N. Choose whether to bind traffic to a particular network adapter
RCloneMountIP="10.10.10.10" # My unraid IP is 172.30.12.2 so I create another similar IP address
NetworkAdapter="eth0" # choose your network adapter. eth0 recommended
VirtualIPNumber="2" # creates eth0:x e.g. eth0:1. I create a unique virtual IP addresses for each mount & upload so I can monitor and traffic shape for each of them

####### END SETTINGS #######

Here's the log from when it ran

2020/07/26 18:52:34 NOTICE: Encrypted drive 'bestvillamilsecure:': ChangeNotify was unable to decrypt "Scene Naming (Responses)": illegal base32 data at input byte 5
Script Starting Jul 26, 2020 19:00.01

Full logs for this script are available at /tmp/user.scripts/tmpScripts/rclone_Mergefs_Mount/log.txt

26.07.2020 19:00:01 INFO: Creating local folders.
26.07.2020 19:00:01 INFO: *** Starting mount of remote bestvillamilsecure
26.07.2020 19:00:01 INFO: Checking if this script is already running.
26.07.2020 19:00:01 INFO: Script not running - proceeding.
26.07.2020 19:00:01 INFO: *** Checking if online
26.07.2020 19:00:07 PASSED: *** Internet online
26.07.2020 19:00:07 INFO: Success bestvillamilsecure remote is already mounted.
26.07.2020 19:00:07 INFO: Mergerfs already installed, proceeding to create mergerfs mount
26.07.2020 19:00:07 INFO: Creating bestvillamilsecure mergerfs mount.
26.07.2020 19:00:07 INFO: Adding /mnt/user/Media/Movies to bestvillamilsecure mergerfs mount.
26.07.2020 19:00:07 INFO: Adding /mnt/user/Media/TV Shows to bestvillamilsecure mergerfs mount.
26.07.2020 19:00:07 INFO: Adding /mnt/user/Media/Anime to bestvillamilsecure mergerfs mount.
fuse: bad mount point `Shows:/mnt/user/Media/Anime': No such file or directory
26.07.2020 19:00:07 INFO: Checking if bestvillamilsecure mergerfs mount created.
26.07.2020 19:00:07 CRITICAL: bestvillamilsecure mergerfs mount failed. Stopping dockers.
binhex-jellyfin
binhex-plex
binhex-emby
binhex-delugevpn
binhex-jackett
sonarr
radarr
ombi
Script Finished Jul 26, 2020 19:00.11

Full logs for this script are available at /tmp/user.scripts/tmpScripts/rclone_Mergefs_Mount/log.txt

Please let me know how I can get this working :)

Error installing mergerfs

I'm not sure if this issue is just cdn related, when trying to run the mount script I get the following error:

Mergerfs not installed - installing now.
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/main/x86_64/APKINDEX.tar.gz
2022/02/14 20:50:04 INFO : vfs cache: cleaned: objects 3 (was 3) in use 0, to upload 0, uploading 0, total size 14.004Ki (was 14.004Ki)
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/community/x86_64/APKINDEX.tar.gz
ERROR: http://dl-cdn.alpinelinux.org/alpine/v3.10/main: temporary error (try again later)
WARNING: Ignoring APKINDEX.00740ba1.tar.gz: No such file or directory
ERROR: http://dl-cdn.alpinelinux.org/alpine/v3.10/community: temporary error (try again later)
WARNING: Ignoring APKINDEX.d8b2a6f4.tar.gz: No such file or directory
2 errors; 14 distinct packages available
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/main/x86_64/APKINDEX.tar.gz
ERROR: http://dl-cdn.alpinelinux.org/alpine/v3.10/main: temporary error (try again later)
WARNING: Ignoring APKINDEX.00740ba1.tar.gz: No such file or directory
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/community/x86_64/APKINDEX.tar.gz
ERROR: http://dl-cdn.alpinelinux.org/alpine/v3.10/community: temporary error (try again later)
WARNING: Ignoring APKINDEX.d8b2a6f4.tar.gz: No such file or directory
ERROR: unsatisfiable constraints:

Would love a workaround :-)

rclone_upload chunk variables may want adding to the config section

The following parts of the upload script may warrant being added as variables to the config sections - the choice in chunk size will affect the RAM usage which may be problematic on low RAM machines as rclone will fail due to running out.

It is probably also worth investigating what changing this setting will do to RAM consumption.

	--buffer-size 128M \
	--drive-chunk-size 128M \

How would I use this script to mount more than one gdrive (i.e. personal drive and a shared/team drive)

Hello there,

This looks really amazing! I am just trying to wrap my head around how all of it would work but from the VERY limited understanding I have about coding it seems that this would really simplify my life to mount the G-drives and get a jellyfin and/or plex server up and running.

The situation I have is this: I currently have two (maybe more) external hard drives and two separate google drives that I would like to mount together. One of my G drives is my personal drive and one is a shared/team one. I would like my jellyfin/plex docker servers to be able to find the files from both so that I can build a library with them as well as to be able to pull from the local storage I have. I think the end result would be to have a mergerfs mount point with my personal drive, one for my shared drive, and one for all of my local HDDs that jellyfin/plex/etc can read from so that I can access these different containers away from home. Oh and before I forget, I am running all of this off a raspberry pi 4 4gb.

Based on this scenario, would the scripts that you have come up with help me achieve this goal? And if so, what would be the best way to start? I am about to wipe the current SD card I am using in my RPI4 and likely start again so that I can get Nginx proxy manager working along with the other containers I plan on having. Your scripts seem like it would be the way to go.

Thank you for all your work and for reading this.