masonr/plexidrive

Scripts to facilitate the use of cloud storage (such as Google Drive) as storage for Plex media server

PLEXiDRIVE

Scripts to facilitate the use of cloud storage providers (i.e. Google Drive) as storage for Plex media using rclone

Purpose

The purpose of this project is to use Cloud Drives as a means of storage for Plex. These scripts can support any cloud drive services that are supported by rclone. The main use case of this project specifically targets using Google Drive unlimited accounts. Traditionally, using a Drive account with Plex runs into issues with exceeding Google's API call quota. This occurs during Plex scans of large media collections. To combat this, this project automates the uploading of media to a Drive account and automatically scans the individual directories where new media was placed. This means that only a small subset of the media library will be scanned as opposed to scanning the entire collection (requires automatic Plex scans to be switched off). The scripts also has the ability to upload media to multiple Google accounts for redundancy in a RAID 1-like manner. This can be useful if the Drive accounts have the potential to be banned or revoked (i.e. purchased on eBay, etc.).

Disclaimer

These scripts are use at your own risk, meaning I am not responsible for any issues or faults that may arise. I have tested these scripts on my own systems and verfied their functionality; however, due diligence is required by the end user. I am in no way affiliated with Google, Plex Inc., or rclone. I am not responsible if a ban is place on the user's Drive account due to abuse or excessive API calls.

Dependencies

1. rclone
2. Plex Media Server
3. plexdrive (optional)

Installation

1. Clone Git repo in home directory

   > ~$ git clone https://github.com/masonr/PLEXiDRIVE

2. Edit permissions to allow plex user full access

   > ~$ sudo chmod -R 777 PLEXiDRIVE

3. Install rclone and configure each Google Drive account

4. Move rclone into a directory found in the PATH environment variable and edit permissions

   > ~$ sudo mv rclone /usr/local/bin/
   > ~$ sudo chown root:root /usr/local/bin/rclone
   > ~$ sudo chmod 755 /usr/local/bin/rclone

5. Mount Google Drive(s) using rclone mount with options

   > ~$ sudo mkdir /mnt/gdrive-main
   > ~$ sudo rclone mount --allow-non-empty --allow-other gdrive-main:/ /mnt/gdrive-main &

   Edit path as needed and use rclone remote names configured in Step 3

   Alternatively, plexdrive should also be able to achieve mounting the remote drive without needing to change anything.

   Encrypted rclone mounts can also be used, but be sure to point your Plex libraries to the decrypted mounts and use the encrypted rclone mount names in the plexidrive config file.

6. Determine the Plex media section numbers for the Movies and TV Show libraries

   • Libraries must first be set up on the Plex server (map the Movies library to the rclone mounted path; same for TV Shows)

   > ~/PLEXiDRIVE$ sudo su -c 'export LD_LIBRARY_PATH=/usr/lib/plexmediaserver; /usr/lib/plexmediaserver/Plex\ Media\ Scanner --list' plex
   	1: Movies
   	2: TV Shows

   See command and example output above

   • Copy the corresponding library section numbers to the plexidrive.conf (plex_movies_section_num & plex_tvshow_section_num)

Important Notes

• Movies must be placed in the root of the Drive account in a folder called "Movies"
• TV Shows must be placed in the root of the Drive account in a folder called "TV Shows"
• TV Shows must be organized of the form: "(root)/Show Name/Season Number/files" (use an automation tool, such as SickRage or Sonarr for ease)
• The script will not delete empty TV Show folders after successful uploading
• Movies can be placed in individual folders or in the local Movies root directory
• In order to avoid a ban on the Google Drive account with large Plex libraries, the automatic media scans within Plex server settings must be switched off
• It's very important to use the exact notation as described for the config file parameters or the scripts may not work at all
• The plex-scan script must be run as root user (sudo ./plex-scan.sh) as the script must have the effective user as plex

Usage

Uploading media

Simply run the script below after configuring the Plex server and setting up the plexidrive.conf file

> ~/PLEXiDRIVE$ ./plexidrive.sh

Scanning Plex library for new files

> ~/PLEXiDRIVE$ sudo su -c './plex-scan.sh' plex

Cron jobs

In order to automate the uploading of media and Plex scans, cron jobs can be used. Add a cron job to the root crontab for the Plex scan, and to the local user's account for the media uploads.

Example cron job to run PLEXiDRIVE every 4 hours:

0 */4 * * * /bin/bash /path/to/PLEXiDRIVE/plexidrive.sh && su -c '/bin/bash /path/to/PLEXiDRIVE/plex-scan.sh' plex

Configuration (plexidrive.conf)

GDrive Settings

• num_of_gdrives: the number of Google Drive accounts to upload media files to
• drive_names: the name(s) of the Google Drive accounts

Options

• delete_after_upload: denotes if the local media files should be deleted after successful upload
• file_types: the file types to scan for when detecting files to upload
• rclone_config: (optional) full path to rclone config file

Plex Library Directories

• plex_tvshow_path: the path of the rclone mounted drive and folder where TV Shows will be found
• plex_movies_path: the path of the rclone mounted drive and folder where Movies will be found
• plex_movies_section_num: the library section number corresponding to Movies, found in installation step 9
• plex_tvshow_section_num: the library section number corresponding to TV Shows, found in installation step 9

Local Media Directories

• local_tvshow_path: the path where local TV Show media will be found
• local_movies_path: the path where local Movie media will be found

Enable/Disable Componenets

• enable_show_uploads: enable or disable uploading of TV media
• enable_movie_uploads: enable or disable uploading of Movie media

Example Config w/ One Google Drive

## GDrive Settings ##
num_of_gdrives=1
drive_names=('gdrive-main')

## Options ##
delete_after_upload=true # true/false
file_types="mkv|avi|mp4|m4v|mpg|wmv|flv"
rclone_config=""

## Plex Library Directories ##
plex_tvshow_path="/mnt/main/TV Shows" # no ending /
plex_movies_path="/mnt/main/Movies" # no ending /
plex_movies_section_num=1
plex_tvshow_section_num=2

## Local Media Directories ##
local_tvshow_path="/home/masonr/tv-shows/" # end with /
local_movies_path="/home/masonr/movies/" # end with /

## Enable/Disable Components ##
enable_show_uploads=true # true/false
enable_movie_uploads=true # true/false

Example Config w/ Two Google Drives

## GDrive Settings ##
num_of_gdrives=2
drive_names=('gdrive-main' 'gdrive-backup')

## Options ##
delete_after_upload=true # true/false
file_types="mkv|avi|mp4|m4v|mpg|wmv|flv|mpeg"
rclone_config="/home/masonr/.config/rclone/rclone.conf"

## Plex Library Directories ##
plex_tvshow_path="/mnt/main/TV Shows" # no ending /
plex_movies_path="/mnt/backup/Movies" # no ending /
plex_movies_section_num=1
plex_tvshow_section_num=2

## Local Media Directories ##
local_tvshow_path="/home/masonr/tv-shows/" # end with /
local_movies_path="/home/masonr/movies/" # end with /

## Enable/Disable Components ##
enable_show_uploads=true # true/false
enable_movie_uploads=true # true/false

plexidrive's Issues

TV Shows root directory entry does not exist for ! See step 8 of readme for directions. - Also, media folder numbers not being determined

when I run -
sudo ./plexidrive.sh
OR
sudo su -c 'export LD_LIBRARY_PATH=/usr/lib/plexmediaserver; /usr/lib/plexmediaserver/Plex\ Media\ Scanner --list' plex

I get - https://dl.dropboxusercontent.com/s/0fvgptisi75gzi7/TeamViewer_2017-06-03_10-20-38.png -

the structure of the folder directory is as you directed. - https://dl.dropboxusercontent.com/s/7jx2hziqq9vjvck/TeamViewer_2017-06-03_10-36-40.png

Thanks for all your help. I can't believe its one thing right after the other.

Allow Multiple Libraries per Media Collection Type

Allow multiple libraries for each type (TV Show & Movies).

Would need to define the lib type (tv shows vs movies) and each would need their own local path on the filesystem before being uploaded to keep track of what library to scan afterwards.

remote gdrive folders don't appear in the /mnt/gdrive-main

It seems that I can use rclone to lsd and ls and copy files. But the remote folders themselves don't appear in /mnt/gdrive-main. Are they supposed to appear somewhere else? according to the command -

sudo rclone mount --allow-non-empty --allow-other the_domain:/ /mnt/gdrive-main

"the_domain" should mount to /mnt/gdrive-main. here's a pic - https://dl.dropboxusercontent.com/s/rsyolrg0vzblsam/TeamViewer_2017-05-31_11-09-17.png

Investigate a Fully rclone Solution

Look into using rclone copy instead of the gdrive cli.

Would make the setup and running of the scripts much simpler. This alone makes it worth re-evaluating rclone copy vs gdrive upload.

As noted by another user, using own client/application id for rclone will reduce the throttling of API calls and download speeds.

plexidrive.sh shouldn't run if the script is already being executed

If plexidrive.sh checks to see if it's already running before executing, it would mean we can have a relatively frequent cronjob schedule.

custom user options

Awesome scripts. This is really what i've been looking for. Thank you!
Only thing i'm missing is to add some customizations to rclone upload, without editing your scripts. I use a few rclone commands for my uploads.
--bwlimit=10.0M --log-file=PATH --exclude ".placeholder"
Aswell as "find"-options -mtime +3
Min-Age 3d is because i want Plex to analyze the items (thumbnails etc) before moving to GDrive, to lower the GDrive access as much as possible.

Would be great to have this in your .conf file.
Maybe even dedicated line for logfile?

(another thing, please write down somewhere that " ~/ " is not working ;) )

Thank you

cannot fetch to ken: 400 Bad Request

When I run this -

> ~$ gdrive --config ~/.gdrive-main about

I get -

gdrive --config ~/.gdrive-main about
Failed to get about: Get https://www.googleapis.com/drive/v3/about?alt=json&fields=maxImportSizes%2CmaxUploadSize%2CstorageQuota%2Cuser: oauth2: cannot fetch token: 400 Bad Request
Response: {
  "error" : "invalid_grant"
}

[1]+  Stopped                 sudo rclone mount --allow-non-empty --allow-other the_matrix:/ /mnt/gdrive-main

Please not sure what to do. Anyone else having this issue?

./plexidrive.sh only prints file name

Hey,

I've setup your scripts as per your instructions and ran through them several times to make sure I haven't made a mistake, however when I run ./plexidrive.sh the only output I get is
root@cheesus ~/PLEXiDRIVE # ./plexidrive.sh
File: Quantico - S02E21 - RAINBOW HDTV-720p.mkv

Doesn't seem to process or upload the file at all, anything obvious I've done wrong, or anywhere I can find some logs to see what's up?

I can do gdrive list which shows my gdrive files so that connects fine, and I have rclone mounting the gdrive fine.

Thanks!

Ability to encrypt data and use encrypted rclone mounts

Will investigate to determine how much work would need to be done to implement this.

[Feature Request] Concurrent (parallel) uploads

Look into implementing parallel uploads.

Options:

1. If there's multiple Drive accounts, upload same file to all Drive accounts concurrently.
2. Spawn off x number of uploads at a time (different files).

Add config option to specify rclone.conf file to use

Be able to specify which rclone.conf file should be used. Will be useful if running the scripts as part of a cronjob under a different user (i.e. root, but conf is in local user's home directory).