aplyca/ansible-role-s3fs

Install and configure s3fs

ansible-role-s3fs's Issues

Aplyca.S3fs : Make sure s3fs dependencies are present is not working

TASK [Aplyca.S3fs : Make sure s3fs dependencies are present] *******************************************************************************************************************************************************************************************************************
Thursday 12 November 2020 19:18:22 +0800 (0:00:00.033) 0:00:24.746 *****
fatal: [odoo-worker-nodes]: FAILED! => {}

MSG:

'ansible.parsing.yaml.objects.AnsibleUnicode object' has no attribute 'dependencies'

Use import_tasks instead of include

Use import_tasks instead of include to combine tasks from multiple files. The include syntax is deprecated.

https://docs.ansible.com/ansible/2.9/user_guide/playbooks_reuse_includes.html

Mounting not working, no buckets created

This is my playbook:
s3fs.yml

---
-
  hosts: all
  become: true
  vars_files:
    - vars/aws.yml
  roles:
    - role: Aplyca.S3fs
      buckets:
        - mountpoint: /mnt/s3fs
          bucket: "{{aws_bucket_name}}"
          accessKeyId: "{{aws_access_key_id}}"
          secretAccessKey: "{{aws_secret_access_key}}"
          options: "allow_other,use_cache=/tmp,max_stat_cache_size=100000,uid=33,gid=33,umask=002"          

ansible-playbook s3fs.yml runs perfectly with no issues however mounting is simply not happening.
It seems that no buckets are actually being created (in debug: buckets = [])
Moreover, /etc/passwd-s3fs does not get created

Here's the debug log: ansible-role-s3fs-bucket-not-found-2017-07-25.txt

Vars:

• ansible: 2.3.1.0
• ansible.cfg: hash_behaviour = merge
• target OS: Ubuntu Sever 16.04

Add NTP as dependency

In order to keep the system in sync with the s3 service to avoid synchronization errors.

Sync up the system clock when restarting s3fs service to avoid s3 error

See: http://stackoverflow.com/questions/4770635/s3-error-the-difference-between-the-request-time-and-the-current-time-is-too-la

Problems with variables from defaults/ and vars/

Hey there.

I stumbled upon the issue mentioned here: #3
So i set the hash_behaviour=merge

It did not help!!! Same error:

FAILED! => {
    "msg": "'dict object' has no attribute 'dependencies'"
}

During this my s3fs dict shows (with merge enabled and disabled):

    "s3fs": {
        "buckets": [
            {
                "accessKeyId": "accessKeyId", 
                "bucket": "s3fs", 
                "mountpoint": "/srv/mount", 
                "options": "allow_other,use_cache=/tmp,max_stat_cache_size=100000,uid=33,gid=33,umask=002", 
                "secretAccessKey": "secretAccessKey"
            }
        ]
    }

I removed the s3fs bucket configuration. Now something gets confiured - vars/

{
    "s3fs": {
        "daemon": "/usr/local/bin/s3fs", 
        "dependencies": [
            "unzip", 
            "fuse", 
            "libssl-dev", 
            "pkg-config", 
            "build-essential", 
            "libfuse-dev", 
            "libcurl4-openssl-dev", 
            "libxml2-dev", 
            "mime-support", 
            "automake", 
            "libtool", 
            "ntp"
        ], 
        "passwd_file": "/etc/passwd-s3fs"
    }
}

But now it fails with

FAILED! => {
    "msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'source'\n\nThe error appears to have been in '/[...]/roles/Aplyca.S3fs/tasks/setup.yml': line 11, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Download s3fs source\n  ^ here\n\nexception type: <class 'ansible.errors.AnsibleUndefinedVariable'>\nexception: 'dict object' has no attribute 'source'"
}

Please help! Copying all variables to the playbook i use the role in, is not a long-term soluiton!

On ansible 2.4.3.0 with the playbook conaining only:

  roles:
    - role: aplyca.s3fs
      s3fs:
        buckets:
          - mountpoint: "{{ someMountpoint }}"
            bucket: s3fs
            accessKeyId: "accessKeyId"
            secretAccessKey: "secretAccessKey"
            options: "allow_other,use_cache=/tmp,max_stat_cache_size=100000,uid=33,gid=33,umask=002"

Don't require hash behavior setting to be changed

The Ansible docs recommend against setting the hash behavior to merge. It'd be nice if this role was usable with the officially recommended Ansible config settings. From a brief look at the code, it seems like flattening the top-level config setting out to separate items would allow Ansible's usual precedence rules to do the right thing. That is, instead of s3fs.source, use s3fs_source, etc.

Happy to put together a PR if that approach would be acceptable.

defaults/main.yml is overwritten by vars/main.yml

Attempting to install s3 deps, the following error is displayed:

task path: /private/etc/ansible/roles/Aplyca.s3fs/tasks/setup.yml:11
fatal: []: FAILED! => {"failed": true, "msg": "'dict object' has no attribute 'source'"}

This is due to vars being of a higher prescedence to defaults.

Error while trying to download s3fs

When I try to run your role I get the this error:

TASK [aplyca.s3fs : Download s3fs source] **************************************
fatal: [ec2-52-73-143-8.compute-1.amazonaws.com]: FAILED! => {"failed": true, "msg": "the field 'args' has an invalid value, which appears to include a variable that is undefined. The error was: 'dict object' has no attribute 'source'\n\nThe error appears to have been in '/private/etc/ansible/roles/aplyca.s3fs/tasks/setup.yml': line 11, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Download s3fs source\n  ^ here\n"}

here is the ansible-playbook I am running:

---
- hosts: localhost
  vars:
    aws_instance_type: "r4.2xlarge"
    # needs to be hvm:ebs-ssd instance type
    aws_ami: "ami-ad4b33bb"
    redundancy: 1
  tasks:

  - name: Provision servers for django
    ec2:
      instance_type: "{{ aws_instance_type }}"
      image: "{{ aws_ami }}"
      region: "us-east-1"
      keypair: "{{ lookup('env','AWS_KEY_PAIR') }}"
      count: "{{ redundancy }}"
      wait: True
    register: django_ec2

  - name: Add new instance to host group
    add_host:
      hostname: "{{ item.public_dns_name }}"
      groupname: django_instances
    with_items: "{{ django_ec2.instances }}"

  - name: Wait for ssh to come up
    wait_for:
      host: "{{ item.public_dns_name }}"
      delay: 30
      timeout: 600
      port: 22
      state: started
    with_items: "{{ django_ec2.instances }}"

- name: Configure django servers
  hosts: django_instances
  user: ubuntu
  gather_facts: True
  vars:
    ansible_ssh_private_key_file: "{{ lookup('env','AWS_SSH_KEY') }}"
    s3fs:
      buckets:
        - mountpoint: /opt/chronam/data/batches/
          bucket: chronam-data-batches
          options: "umask=0022,uid=1000,gid=1000,allow_other,nonempty,use_cache=/tmp"
  roles:
    - aplyca.s3fs
  tasks:
  - name: install dependencies
    apt: name={{ item }} update_cache=True
    with_items:
      - python2.7-dev
      - python-pip
      - libmysqlclient-dev 
      - apache2 
      - libapache2-mod-wsgi 
      - libxml2-dev 
      - libxslt-dev 
      - libjpeg-dev 
      - git-core 
      - graphicsmagick
    become: True

  - name: Git clone chronam code
    git:
      repo: 'https://github.com/LibraryOfCongress/chronam.git'
      dest: /opt/chronam
    become: True

  - name: Set correct permissions
    file:
      recurse: True
      path: /opt/chronam
      owner: ubuntu
      group: ubuntu
    become: True

  - name: Create chronam directories
    file: path="{{ item }}" state="directory" 
    with_items:
      - /opt/chronam/static
      - /opt/chronam/.python-eggs

  - name: Copy LC specific python dependencies
    copy: 
      src: "files/{{ item }}" 
      dest: "/opt/chronam/{{ item }}"
    with_items:
      - django-lc-0.0.3.tar.gz
      - minicts-0.0.1.tar.gz

  - name: Install Chronam
    script: scripts/install_chronam.sh

  - name: create cache and bib directories
    file:
      path: "/opt/chronam/data/{{ item }}"
      state: directory
    with_items:
      - cache
      - bib

  - name: Modify chronam to use new SOLR
    lineinfile:
      path: /opt/chronam/settings_template.py
      regexp: "SOLR = .*"
      line: "SOLR = 'http://{{ groups['solr_instances'][0] }}:8080/solr'"

  - name: Modify chronam to use new MySQL database
    lineinfile:
      path: /opt/chronam/settings_template.py
      insertafter: "'PASSWORD':.*"
      regexp: "'HOST':.*"
      line: "        'HOST': '{{ groups['mysql_instances'][0] }}',"

  - name: Modify chronam to use correct MySQL password
    lineinfile:
      path: /opt/chronam/settings_template.py
      regexp: "'PASSWORD':.*"
      line: "        'PASSWORD': '{{ hostvars['localhost']['db_password'] }}',"

  - name: Modify apache2 config to allow access to root directory
    replace:
      path: /etc/apache2/apache2.conf
      regexp: "(<[dD]irectory />[^<]*)Require all denied"
      replace: '\1Require all granted'
    become: True

  - name: restart apache
    service:
      name: apache2
      state: restarted
    become: True

  - name: Run initial chronam config script
    script: scripts/django_initial_setup.sh

  - name: Remove temp directory
    file:
      path: /var/tmp/django_cache
      state: absent
    become: True

can't install using ansible-galaxy

Whenever I try to install this role I get the following error:

ansible-galaxy install aplyca.s3fs
- downloading role 's3fs', owned by aplyca
- downloading role from https://github.com/aplyca/ansible-role-s3fs/archive/v1.1.0.tar.gz
 [ERROR]: failed to download the file: HTTP Error 404: Not Found

 [WARNING]: - aplyca.s3fs was NOT installed successfully.

ERROR! - you can use --ignore-errors to skip failed roles and finish processing the list.

version

ansible-galaxy --version
ansible-galaxy 2.3.0.0
  config file = /Users/jscancella/.ansible.cfg
  configured module search path = Default w/o overrides
  python version = 2.7.13 (default, Dec 18 2016, 07:03:39) [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)]