Automated Media Server 👻

         _               _
    __ _| |__   ___  ___| |_
   / _` | '_ \ / _ \/ __| __/
  | (_| | | | | (_) \__ \ |_
   \__, |_| |_|\___/|___/\__|
   |___/      /   _ \
          (¯\| o (@) |/¯)
           \_  .___.  _/
            /   !_!   \
           /_.--._.--._\

About

This is an automated media server set up in docker containers via docker-compose. The goal of this project is to automate as much of the installation and configuration as possible while maintaining simplicity of the setup (no ansible playbooks or fancy bash scripts) to avoid making it a complete black box.

The end result of this setup is a media server with the following components

plex
filebot
- with a web interface for triggering amc script
transmission
- configured to call filebot container on torrent completion
- with combustion web UI
nzbget
- configured to call filebot container on nzb completion
sonarr
radarr
jackett
bazarr
tautulli
- configured to look at plex server logs
portainer
watchtower
nginx + letsencrypt
- for reverse proxying to reach your services at *.domain.tld
heimdall

The high-level steps for setup are as follows

Install docker and docker-compose
docker-compose up
add an indexer to jackett
configure sonarr / radarr to use the jackett indexer
configure sonarr / radarr to use an nzb indexer
configure sonarr / radarr to use transmission as their download client
configure sonarr / radarr to use nzbget as their nzb client
configure nzbget with your newsgroup provider
add libraries to plex

Once these steps have been completed, it is possible to add a TV Show or Movie to sonarr / radarr, have it automatically download them when available, and then it will be automatically copied into your plex libraries. Post-installation, this should be a fully-automated media server. The transmission container will automatically clean any files older than 30 days (configurable).

📓 This has only been tested on Ubuntu 18.04 LTS but it should work just fine on other linux distros. MacOS and Windows are unsupported. If you test it on MacOS or Windows and it works, let me know!

⚠️ A Note About Torrent Clients

I've removed the cofiguration for qbittorrent. If you want to keep using it, feel free to search through git history to recover the configuration, but it's no longer supported and has not been updated to use the new filebot container.

Network

Each service is available on its own ports:

Service	Port
transmission	5656
nzbget	6789
filebot	7676
sonarr	8989
radarr	7878
bazarr	6767
jackett	9117
plex	32400
portainer	9000
heimdall	8888
netdata	19999

📓To reach plex, append /web to the address e.g. 192.168.1.11:32400/web

All of the services are running in the default docker-compose network. From within services, they can access each other via their <service_name>:<port> as defined in docker-compose.yml. These are NOT the ports you've configured in your .env file. Those ports are mapped to the host device's network, but the services are operating within the network that docker-compose sets up, so they will not respect the rules forwarding ports to the host network.

Service	Port
transmission	5656
nzbget	6789
filebot	7676
sonarr	8989
radarr	7878
bazarr	6767
jackett	9117
plex	32400
tautulli	8181
heimdall	80
netdata	19999

Installation

Install Docker

https://docs.docker.com/install/#supported-platforms

Install Docker Compose

https://docs.docker.com/compose/install/#install-compose

Clone this repo

git clone https://github.com/ghostserverd/mediaserver-docker.git
cd mediaserver-docker

Build your `.env` file

cp .env_sample .env
id $USER # save the result of this for building your .env file below

Modify the .env file to specify the directory configurations. Note that these directories are for the host machine. They are mapped to various locations inside of their container by the docker-compose file.

The .env_sample file has some notes about the various configuration options. There are additional descriptions of each service's configuration below.

Directories

A good directory setup looks something like this

CONFIG_DIR=/some/directory/media_server/config
DOWNLOAD_DIR=/some/directory/media_server/downloads
MEDIA_DIR=/some/directory/media_server/media
TV_DIR=/some/directory/media_server/media/TV Shows
MOVIES_DIR=/some/directory/media_server/media/Movies
BASE_DIR=/some/directory/media_server

Note that there is a base directory that holds both media and downloads

/some/directory/media_server

Configuring a base directory with downloads and media present allows filebot to take advantage of hardlinks when renaming and moving files. Hardlinks are much quicker than actually copying the file, but they require that the source and destination be on the same device which is what the base directory accomplishes.

variable	description
CONFIG_DIR	where configuration for each of the services will live. You'll end up with multiple directories in here, one for each service. If you move this directory to a different volume with a different instance of the whole media server, it should retain your various configurations.
DOWNLOAD_DIR	where various services will download files to. ⚠️ This should not be on a small partition as it will contain media files.
MEDIA_DIR	where your media will be copied to. ⚠️ This should not be on a small partition as it will contain media files.
TV_DIR	where your TV shows will be placed by `filebot` on download completion. It should be a subdirectory of `MEDIA_DIR`
MOVIES_DIR	where your TV shows will be placed by `filebot` on download completion. It should be a subdirectory of `MEDIA_DIR`
BASE_DIR	a shared directory that houses `media` and `downloads` directories to be used for hardlinks

PUID and PGID

variable	description
PUID	the unix `UID` that will be passed to the various services. It can be discovered by running `id $USER` on the host machine.
PGID	is the unix `GID` that will be passed to the various services. It can be discovered by running `id $USER` on the host machine.

Deploy the mediaserver

Fill out the rest of the configuration options before deploying. See each individual service's section for configuration details.

docker-compose up

Append -d to run in detached mode. The first time you run a service, it is probably a good idea to not run in dettached mode (i.e. DON'T append -d) so you can watch all of the logs for issues.

Configuration

Configure Jackett

variable	description
JACKETT_PORT	the port that jacket will listen on

<server-ip>:9117

jackett should be configurable the same as any other installation of it. Feel free to skip these steps if you know how to configure jackett already.

Add an Indexer
- Click + Add Indexer
- Search for your desired tracker
- Click on the 🔧icon next to your desired tracker
- Sign in with your account information
- Click Okay
You should see Successfully configured <tracker> and a new entry for your tracker
Copy the API Key from the top right corner and save it somewhere
Click the Copy Torznab Feed on the tracker you just added and paste it somewhere to save it

Configure Sonarr

variable	description
SONARR_PORT	the port that sonarr will listen on

<server-ip>:8989

sonarr should be configurable the same as any other installation of it. Feel free to skip these steps if you know how to configure sonarr already.

⚠️ Do make sure that your download path is set to /data/completed/tv as that is the directory that the container has permissions to.

⚠️ It is also critical that you use transmission instead of the IP address when configuring the download client, as well as jackett instead of the IP when setting up your indexer. This is because this uses docker-compose networking which means each service is accessible at the name of the service, rather than the host or even local IP address.

Step-by-step for those who need it

Add an Indexer
- Click on the Settings button at the top
- Click on the Indexers tab
- Click the big + symbol
- Click the Custom button in the Torznab section
- Configure your Torznab feed
  - Name : the name of this indexer (doesn't matter, just name it the name of your tracker)
  - URL : the Copy Torznab Feed url from jackett that you saved earlier
    - ⚠️ It is necessary to replace the IP with jackett
      - http://jackett:9117/api/v2.0/indexers/<indexer>/results/torznab/ instead of
      - http://192.168.1.11:9117/api/v2.0/indexers/<indexer>/results/torznab/
  - API Key : the API Key from jackett that you saved earlier
- Click Test to verify that it is configured properly
- Click Save
Add a Download Client
- Click on the Settings button at the top
- Click on the Download Client tab
- Under Completed Download Handling toggle Enable from Yes to No
  - We'll be using filebot to handle our completed downloads
- Click the Save button at the top right
- Click the large + button
- Click on qBittorrent
- Configure your Download Client
  - Name : whatever you want; probably qBittorrent
  - Host : qbittorrent
  - Port : 6767 or whatever you have set for QBIT_WEBUI_PORT in your .env file
  - Username : admin or whatever you have set for QBIT_WEBUI_USER in your .env file
  - Password : adminadmin or whatever you have set for QBIT_WEBUI_PASS in your .env file
- Click Test to verify that it is configured properly
- Click Save
Add Some TV Shows
- Click on the Series button at the top
- Click + Add Series
- Start typing in the search bar. It will search automatically when you stop typing
- Configure the download path (you should only have to do this on the first show you add)
  - Click the dropdown that says Select Path
  - Click Add a different path
  - Click the 📁button on the right of the modal
  - Click tv # final path should be /tv/
  - Click Ok
  - Click the green ✔️that is now visible
  - Click the + sign
There are many other configuration options for sonarr that are not covered here. sonarr's webpage is here

Configure Radarr

variable	description
RADARR_PORT	the port that radarr will listen on

<server-ip>:7878

radarr should be configurable the same as any other installation of it. Feel free to skip these steps if you know how to configure radarr already.

⚠️ Do make sure that your download path is set to /data/completed/movies as that is the directory that the container has permissions to.

Step-by-step for those who need it

Add an Indexer
- Click on the Settings button at the top
- Click on the Indexers tab
- Click the big + symbol
- Click the Custom button in the Torznab section
- Configure your Torznab feed
  - Name : the name of this indexer (doesn't matter, just name it the name of your tracker)
  - URL : the Copy Torznab Feed url from jackett that you saved earlier
    - ⚠️ It is necessary to replace the IP with jackett
      - http://jackett:9117/api/v2.0/indexers/<indexer>/results/torznab/ instead of
      - http://192.168.1.11:9117/api/v2.0/indexers/<indexer>/results/torznab/
  - API Key : the API Key from Jackett that you saved earlier
- Click Test to verify that it is configured properly
- Click Save
Add a Download Client
- Click on the Settings button at the top
- Click on the Download Client tab
- Under Completed Download Handling toggle Enable from Yes to No
  - We'll be using filebot to handle our completed downloads
- Click the Save button at the top right
- Click the large + button
- Click on qBittorrent
- Configure your Download Client
  - Name : whatever you want; probably qBittorrent
  - Host : qbittorrent
  - Port : 6767 or whatever you have set for QBIT_WEBUI_PORT in your .env file
  - Username : admin or whatever you have set for QBIT_WEBUI_USER in your .env file
  - Password : adminadmin or whatever you have set for QBIT_WEBUI_PASS in your .env file
- Click Test to verify that it is configured properly
- Click Save
Add Some Movies
- Click on the Add Movies button at the top
- Start typing in the search bar. It will search automatically when you stop typing
- Configure the download path (you should only have to do this on the first movie you add)
  - Click the dropdown that says Select Path
  - Click Add a different path
  - Click the 📁button on the right of the modal
  - Click movies # final path should be /movies/
  - Click Ok
  - Click the green ✔️that is now visible
  - Click the + sign
There are many other configuration options for radarr that are not covered here. radarr's webpage is here

Configure filebot

<server-ip>:7676

variable	description
FILEBOT_PORT	the web interface port for the filebot container
FILEBOT_FORMAT	the filebot format expression to use
FILEBOT_ACTION	the action for filebot to take when renaming files
FILEBOT_CONFLICT	what filebot does when it sees a conflicting filename
FILEBOT_SERIES_DB	the database to use for TV metadata lookup
FILEBOT_ANIME_DB	the database to use for anime metadata lookup
FILEBOT_MOVIE_DB	the database to use for movie metadata lookup
FILEBOT_MUSIC_DB	the database to use for music metadata lookup
OPEN_SUB_USER	the opensubtitles username for filebot to use when downloading subtitles [NOT FUNCTIONAL]
OPEN_SUB_PASS	the opensubtitles password for filebot to use when downloading subtitles [NOT FUNCTIONAL]

The opensubtitles configuration is called by the container startup script, but opensubtitles still fails when filebot is actually run. Until I figure out how to make this work, stick to bazarr.

You can navigate to <server-ip>:7676 to see the filebot command that will be run when a download is completed. Do NOT expose this port to the internet as it is not password protected.

These are the filebot CLI options: https://www.filebot.net/cli.html. If there are additional options you want to be able to configure, open an issue.

⚠️ This has been updated to use the 4.9.x version of filebot by default. In order to properly register after you have purchased a license, copy your license.psm file to the filebot config directory on your host machine. The container will automatically register filebot with that license.

Configure transmission

variable	description
TRANS_WEBUI_USER	the username with which to log into transmission
TRANS_WEBUI_PASS	the password with which to log into transmission
TRANS_WEBUI_PORT	the port for transmission's web interface
TRANS_CONNECTION_PORT	the connection port for tranmission to use
TRANS_MAX_RETENTION	the time in seconds before a torrent is automatically removed
TRANS_MAX_RATIO	the ratio at which a torrent is automatically removed

<server-ip>:5656

It should not be necessary to configure transmission beyond the default configuration. The container writes a config with reasonable defaults. If you need access to additional transmission settings, feel free to open an issue.

The container is already automatically configured to call filebot to post-process a download.

Configure nzbget

<server-ip:7890

variable	description
NZBGET_PORT	the web interface port for nzbget

The container is already automatically configured to call filebot to post-process a download.

These are the automatic configurations in nzbget now:

The MainDir will automatically be set to /downloads/nzb
The ScriptDir will automatically be set to /usr/local/bin
The Extensions will automatically be set to nzbget-postprocess.sh
The ControlUsername will automatically be set to whatever is configured in NZBGET_WEB_USER iff it is set
The ControlPassword will automatically be set to whatever is configured in NZBGET_WEB_PASS iff it is set

The paths are configured that way to support calling filebot with the post-process script.

See the documentation for instructions on setting up the rest of nzbget.

Configure Plex

variable	description
PLUGIN_LIST	a list of plugins to install. supported plugins are `trakt` and `subzero`. leave empty to install no plugins
PLEX_WEB_PORT	the port for the plex web interface

<server-ip>:32400/web

Add some libraries
- TV Shows will be at /data/TV Shows assuming you followed the /media/TV Shows convention for TV_DIR
- Movies will be at /data/Movies assuming you followed the /media/Movies convention for MOVIES_DIR
⚠️ Set up your media agents to not use local files (hopefully this will be fixed in the future)
- There is a problem that I have not been able to fix yet where local TV Series art is not available. It appears the artwork downloaded by filebot is not readable by plex for an unkown reason (I don't believe it's permissions related, but if you have ideas, please open an issue).
- To get around this, uncheck Local Media Assets for all Agents under Settings > Server > Agents. Artwork will be downloaded by plex and accessible.
Kill and restart the containers after logging in to Plex if you have Plexpass

docker stop plex
docker-compose up -d plex

Reverse proxy / letsencrypt

The compose file includes a letsencrypt container which you can use to set up a reverse proxy to various services which will automatically provision an SSL certificate for you to use. If you do not want to use a reverse proxy, simply delete that entry from the compose file.

I recommend that you launch the full mediaserver once without the reverse proxy enabled so you can set up authentication for sonarr, radarr, bazarr, nzbget, tautulli, netdata, and heimdall

The reverse proxy is configured to use subdomain routing by default. It copies in appropriate configurations for each service with these names

service	config name
sonarr	s.subdomain.conf
radarr	r.subdomain.conf
transmission	t.subdomain.conf
nzbget	n.subdomain.conf
plex	p.subdomain.conf
tautulli	u.subdomain.conf
netdata	m.subdomain.conf
heimdall	h.subdomain.conf
bazarr	b.subdomain.conf

Config files for each subdomain will be present in the config directory on your host machine if you wish to change the configurations. The directory is config/letsencrypt/nginx/proxy-confs. If you wish to use different subdomains (e.g. plex.domain.tld instead of p.domain.tld) you need to change the configuration for the subdomain in that directory, and update the LE_SUBDOMAINS to include the new subdomain.

Note that the first time you run the letsencrypt container, it can take some time for it to register an SSL cert.

variable	description
LE_HOSTNAME	the hostname you are using to host the reverse proxy
LE_EMAIL	the email for which your letsencrypt SSL cert will be registered
LE_SUBDOMAINS	the subdomains for which to register the SSL cert

VPN networks for transmission, nzbget, and others

I have built a wireguard container following the principles described here and have successfully tested using docker-compose to force all traffic for nzbget through the wireguard VPN using a config from http://mullvad.net/.

Download or build a wireguard config file from your VPN provider. For example, mullvad has a wireguard config generator at https://mullvad.net/en/download/wireguard-config/. Once you have the config file (e.g. mine is called wgnet0.conf, copy that file into your config/wireguard directory. The container will automatically find the config file and build the VPN network for you.

⚠️ Do NOT use your VPN's killswitch in the wireguard config file. This will disallow traffic from the docker network which will make it inaccessible from the reverse proxy container. There is a rudimentary killswitch implemented in the wireguard container already. I may improve this over time.

docker-compose.yml has been updated with a commented out wireguard service. There is also a commented out nzbget service configured to use the wireguard container's network as an example. This will force nzbget to proxy all traffic through the wireguard container, and thus through the VPN. You can follow the same pattern with the transmission container, or even sonarr and radarr if you want all queries to torrent trackers to go through the VPN.

To proxy an additional container through the wireguard container's network, add the service name (e.g. transmission) to the list of network aliases defined in the wireguard service.

    networks:
      default:
        aliases:
          - nzbget
          - transmission

Then update the new service's definition (e.g. transmission) to remove the port mapping list, and add

network_mode: "service:wireguard"
depends_on:
  - wireguard

Because the wireguard has an alias to transmission, it is now accessible on transmission:5656 just like it was before. You will also want to add the port mapping that were originally in the transmission service definition to the wireguard service definition.

The wireguard container now has a proper route back to the local network over eth0 so services that are using the wireguard network should now be accessible via your local subnet. You need to specify this subnet in the LOCAL_NETWORK environment variable. Thanks to htilly and cmulk who's containers I shamelessly copied and modified. Once you have set up the route back to the local subnet, you can properly port foward through the VPN. See https://mullvad.net/en/help/port-forwarding-and-mullvad/ for details on port forwarding with mullvad.

Why do you need to mount `docker.sock` for the wireguard container?

Mounting docker.sock is normally an anti-pattern for containers. However, when the wireguard container is started, wg-quick rewrites /etc/resolv.conf with the DNS address specified in the interface's .conf file. This breaks DNS resolution for docker services from within any container that is using the wireguard container as its network. In practice, this means that from within e.g. the transmission or nzbget containers, curl filebot:7676 will fail DNS resolution. This breaks the auto-config between downlaoders and filebot.

To work around this issue, the wireguard container now inspects the network (docker network inspect mediaserver-docker_default) and writes /etc/hosts entries for each service it finds in the network. This means that if one of your service IPs changes for any reason, you'll need to restart the wireguard container in order to be able to properly resolve DNS for docker services again.

I believe it is possible to fix this issue using dnsmasq instead of writing /etc/hosts which would remove the requirement to read from docker.sock, but I have not been able to get it to work, so I'm leaving this hack in place for now.

If you do not mount docker.sock, the wireguard container will still run, but any containers in the wireguard network will be unable to resolve DNS for other docker services. If that's fine with you, you don't need to mount docker.sock.

Update: mounting `docker.sock` is no longer necessary

The wireguard container has been updated to use dnsmasq to conditionally route DNS requests.

The container also now scrapes the wireguard interface file for its DNS server and write /etc/dnsmasq.conf with that as the final fallback DNS address.

There are now three options for enabling docker DNS resolution from within the wireguard container:

If docker.sock is mounted, write /etc/hosts with any service in the network (this is really just for backwards comptability and is the same behavior as described above.)
If LOCAL_TLD is set (e.g. to local) write /etc/dnsmasq.conf to use 127.0.0.11 for that TLD. Also, write /etc/resolv.conf to search LOCAL_TLD so that containers can access the addresses of the services without having to know to append .local to match the rule in /etc/dnsmasq.conf. This will require aliases with the TLD in each of the containers that need to be accessible from within the wireguard network. An alias should look something like this (e.g. for LOCAL_TLD=ghost).

filebot:
...
  networks:
    default:
      aliases:
        - filebot.ghost

which will result in filebot being accessible from containers within the wireguard network.

If SERVICE_NAMES is set (a list of services to make available from within the wireguard network), write each service name individually to /etc/dnsmasq.conf to force 127.0.0.11 as the DNS server for each service address. This is nice because you don't have to write an alias for each service to make available, but you do need to list out all of the services. A sample SERVICE_NAMES variable is set in the docker-compose.yml file. It shouldn't need to be modified, but if there is a reason to, open an issue and I can make it pull from the .env file.

The last three settings are mutually exclusive. Only one of the mechanisms should be used. Of the last two, I'm not really sure which one I prefer, but they're both better than mounting docker.sock in my opinion.

Thank You

Linuxserver

Most of these containers are config wrappers around LinuxServer.io containers. Without their amazing linuxserver containers, none of this would have been possible. If you find this automated media server useful, go donate to them! They probably deserve it more than I do.

forum.linuxserver.io
IRC on freenode at #linuxserver.io
Podcast covers everything to do with getting the most from your Linux Server plus a focus on all things Docker and containerisation!
Donate

Filebot

This would also not be possible without filebot. This has been updated to use the latest 4.9.x version of filebot. It supports automatic registration as long as you provide a license file.

ASCII word generation

Thanks to patorjk for his ascii text generator

combustion UI for transmission

Thanks to secretmapper for the combustion UI for transmission

Combustion

Donate

If you really want to donate to me, you can do that here

Future Plans

Consider switching from nginx to traefik
Auto-configuration for linking radarr and sonarr to transmission
Auto-configuration for linking radarr and sonarr to nzbget
Auto-configuration for plex libraries
Improve documentation (maybe blog post with pictures)
~~Better configuration options for nzbget~~
~~Wireguard as a network container for downloaders~~
~~Additional containers (tautulli, muximux, portainer)~~
~~Add reverse proxy support (traefik?)~~
~~Upgrade filebot to 4.8.2 and make it easy to license~~

michsior14 / mediaserver-docker Goto Github PK

mediaserver-docker's Introduction