smarteon/puppet-rclone

Puppet Rclone management module

Puppet Rclone management module

Puppet module helping with installation and configuration of Rclone

Disclaimer: This module is in early stage of development - everything are subject to change. Any feedback or help is welcomed.

Description

Use this module to automate installation and configuration of Rclone - cloud rsync tool.

Setup

Setup Requirements

Depends on following modules:

• puppet/archive

Beginning with rclone

class { 'rclone':
  ensure => 'desired rclone version'
}

# configure s3
rclone::config::s3 { 'my_S3':
  access_key_id     => 'AKI...',
  secret_access_key => '...',
  os_user           => 'my_user',
  region            => 'eu-west-1',
}

Usage

See reference guide

Limitations

Module in early stage, anything can break anytime.

Development

Prerequisites: Installed ruby, gem and bundler and also PDK can help.

pdk bundle update
pdk bundle install

Run unit tests

pdk bundle exec rake spec

or with PDK

pdk test unit

Generate reference

bundle exec rake strings:generate:reference

Releasing

Build and release the module to puppet forge (note currently validation and changelog doesn't work)

pdk build
pdk release

Tag the release in git

git tag -a v<version> -m "message"
git push --tags

Bump the version to next devel and make pull request.

puppet-rclone's Issues

Google Drive Service Account Credential Not Working

Hi,

I've tried many combinations of embedding the json service account credential in a profile manifest and hiera yaml file, trying heredoc in the manifest like so:

$service_account_credential = @(CREDENTIAL:json /L)
{"type": "service_account",
"project_id": "rclone-12345",
[...]}
| - CREDENTIAL

or in the hiera yaml file like:

--
role: backup
rclone::gdrive::os_user: root
rclone::gdrive::config_name: googledrive
rclone::gdrive::client_id: '1234'
rclone::gdrive::client_secret: ''
rclone::gdrive::team_drive: teamdrive
rclone::gdrive::root_folder_id: asdflkasdjflskdfjasldkj-
rclone::gdrive::scope: drive
rclone::gdrive::service_account_credentials: >
{type: 'service_account',
project_id: 'rclone-12345',
private_key_id: 'alsdjflkasdjflkasdjfl;askdjf;lasdjfa;lsdkjfsdl',
private_key: "-----BEGIN PRIVATE KEY-----\n[...]\n-----END PRIVATE KEY-----\n",
client_email: '[email protected]',
client_id: '1234',
auth_uri: 'https://accounts.google.com/o/oauth2/auth',
token_uri: 'https://oauth2.googleapis.com/token',
auth_provider_x509_cert_url: 'https://www.googleapis.com/oauth2/v1/certs',
client_x509_cert_url: 'https://somegoogleurl'}

In either case, the error output from a puppet client linux host is as follows:

me@host:~$ sudo /opt/puppetlabs/bin/puppet agent --test --environment=myenvironment
Info: Using configured environment 'myenvironment'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Caching catalog for backup.lygos.com
Info: Applying configuration version 'puppet-myenvironment-1234'
Notice: Hiera returned role: backup
Notice: /Stage[main]/Main/Notify[Hiera returned role: backup]/message: defined 'message' as 'Hiera returned role: backup'
Notice: /Stage[main]/Nrpe::Install/Package[nagios-plugins]/ensure: created (corrective)
Error: Could not find command '\ \ "project_id":\ "rclone-12345",'
Error: /Stage[main]/Profile::Backup/Rclone::Config::Gdrive[drive_remote]/Rclone::Config[googledrive]/Exec[rclone create remote googledrive for user root]/returns: change from 'notrun' to ['0'] failed: Could not find command '\ \ "project_id":\ "rclone-12345",' (corrective)

Not sure why it appears that puppet is attempting to exec a command formed by the string \ \ "project_id":\ "rclone-12345",' (?).

An example of how to embed the service_account_credentials in either a manifest or hiera file would be most welcome.
I know that the credentials work when I manually install them on a host and run rclone to Google Drive.

Thanks for your work on the rclone module!

Marshall