sleighzy/ansible-zookeeper

Ansible role for installing and configuring Apache ZooKeeper

ansible-zookeeper's Issues

Role installation failing on Ubuntu Ansible Controller v. 2.9.6

Hello,

First of all, I would like to thank you for sharing this role with the community.
I am using Ansible 2.9.6. on a Ubuntu 20.04.4 (ansible controller) and I am getting the following error when trying to execute the role installation:

TASK [sleighzy.zookeeper : Load OS-specific variables] **************************************************************************************************************************************
task path: /home/xxx/.ansible/roles/sleighzy.zookeeper/tasks/main.yaml:7
[WARNING]: Invalid request to find a file that matches a "null" value
failed: [xxx] (item=/home/localadmin/.ansible/roles/sleighzy.zookeeper/vars/../vars/Debian.yml) => {
"ansible_facts": {},
"ansible_included_var_files": [],
"ansible_loop_var": "item",
"changed": false,
"item": "/home/localadmin/.ansible/roles/sleighzy.zookeeper/vars/../vars/Debian.yml",
"message": "Could not find file on the Ansible Controller.\nIf you are using a module and expect the file to exist on the remote, see the remote_src option"
}

PLAY RECAP **********************************************************************************************************************************************************************************
xxx : ok=3 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0

The target machine is running under the same OS.

The issue seems to be related to the ansible.builtin.include_vars module and if I am changing it with include_vars the installation of the role will work correctly.

################
Ansible version:

localadmin@xxx:~/Ansible$ ansible --version
ansible 2.9.6
config file = /etc/ansible/ansible.cfg
configured module search path = ['/home/localadmin/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python3/dist-packages/ansible
executable location = /usr/bin/ansible
python version = 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0]

################
OS version:

localadmin@xxx:~/Ansible$ cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.4 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.4 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

From what I could see, I am already using the latest version of Ansible available for Ubuntu. Is this something that can be fixed?

Add a sample in readme to allow "zookeeper_servers" property from hosts list

Hi,

This plugin was very helpful. Thanks...

One suggestion -> I found that readme has shown how you can add "zookeeper_servers" for distributed setup. However, I think it will be easy for someone if you can add a sample like the one below (this will allow someone to have the same YAML which can work with different hosts file).

  roles:
    - role: sleighzy.zookeeper
      vars:
        zookeeper_servers_use_inventory_hostname: true
        zookeeper_servers: "{{ groups['zookeeper-nodes'] }}"

Also in the Kafka setup the following will be helpful: Most of the time the setup would need the list of ZK

- role: sleighzy.kafka
      vars:        
        kafka_zookeeper_connect: "{{ groups['zookeeper-nodes'] | product([':2181']) | map('join') | list | join(',') }}"        

Add firewall rule for Debian using ufw

Hi

To be able to manage ufw rules in Debian, you can use the community.general.ufw module from the community.general collection.
task file: ufw.yml
name: "Add ufw rules"
community.general.ufw:
rule: allow
port: "{{ item }}"
proto: tcp
with_items:
- "{{ (zookeeper_client_port }}"
- "{{ zookeeper_leader_port }}"
- "{{ zookeeper_election_port }}"

Add to the main.yaml task file:

• include_tasks: ufw.yml
  when: (ansible_os_family == "Debian") and zookeeper_ufw

Kind regards
Jens M. Kofoed

Error: JAVA_HOME is not set and java could not be found in PATH.

Hi
I get an error in the zookeeper logs

Error: JAVA_HOME is not set and java could not be found in PATH.

Support zookeeper > 3.4.14

I'm trying to install zookeeper 3.5.5 but the download mirror is not working.

Details:

Code

http://apache.mirror.anlx.net/zookeeper/zookeeper-3.5.5/
Name: apache-zookeeper-3.5.5.tar.gz

VS

http://apache.mirror.anlx.net/zookeeper/zookeeper-3.4.14/
Name: zookeeper-3.4.14.tar.gz

Maybe one possible solution is passing the filename as a custom variable

What do you think?

inventory file

First of all, thank you for this great resource.
I world like to have an example of inventory file to get zookeeper working fine.
I installed zookeeper with a file I did myself. All worked fine , however all nodes are in standalone mode.
thanks in advance

Change hardcoded ports in firewalld.yml file

Hi,

Many thanks for a wonderful role for installing and configure zookeeper. It saves my hours to write my own.
A suggestions is to change the hardcoded ports in the firewalld.yml file:
From:
name: Add firewalld rules
firewalld:
port: "{{ item }}"
permanent: yes
state: enabled
loop:
- 2181/tcp
- 2888/tcp
- 3888/tcp
notify: Reload firewalld

To:
name: Add firewalld rules
firewalld:
port: "{{ item }}"
permanent: yes
state: enabled
with_items:
- "{{ (zookeeper_client_port }}/tcp"
- "{{ zookeeper_leader_port }}/tcp"
- "{{ zookeeper_election_port }}/tcp"
notify: Reload firewalld

Kind regards
Jens M. Kofoed

sleighzy.zookeeper return AnsibleUndefinedVariable

i'm trying to install and run zookeeper using this ansile galaxy, it returns
fatal:FAILED! => {"changed": false, "msg": "AnsibleUndefinedVariable: {{ groups['zookeeper-nodes'] }}: 'dict object' has no attribute 'zookeeper-nodes'"}

Can't install role with custom name

Starting galaxy role install process
- extracting community.zookeeper to ...
[WARNING]: - community.zookeeper was NOT installed successfully: Invalid linkname for tarfile member: path ...ansible-zookeeper is not a subpath of the role ...community.zookeeper

Seems related to https://github.com/sleighzy/ansible-zookeeper/blob/master/tests/roles/ansible-zookeeper being a symlink, can this be fixed?

FAILED! => {"changed": false, "msg": "AnsibleUndefinedVariable: 'ansible.vars.hostvars.HostVarsVars object' has no attribute 'zookeeper_id'"}

Hello, i'm currently use your ansible-galaxy playbook in my local vm, and that error showing, message error :

fatal: [target1]: FAILED! => {"changed": false, "msg": "AnsibleUndefinedVariable: 'ansible.vars.hostvars.HostVarsVars object' has no attribute 'zookeeper_id'"}

And after i check it, i thinks because u change code in this PR : #13

and, when i change the code locally from :

server.{{ hostvars[host].zookeeper_id }}={{ hostvars[host].ansible_nodename }}:{{ zookeeper_leader_port }}:{{ zookeeper_election_port }}

to

server.{{ hostvars[host].zookeeper_id | default(zookeeper_id) }}={{ hostvars[host].ansible_nodename }}:{{ zookeeper_leader_port }}:{{ zookeeper_election_port }}

the error not showing again, and my playbook success. does that mean you have to rollback the code of this section?

'ansible.vars.hostvars.HostVarsVars object' has no attribute 'zookeeper_id'"}

TASK [ansible-zookeeper : Template configuration file to zoo.cfg] ***********************************************************************************************
fatal: [35.192.83.190]: FAILED! => {"changed": false, "msg": "AnsibleUndefinedVariable: 'ansible.vars.hostvars.HostVarsVars object' has no attribute 'zookeeper_id'"}

Add the ability to template any parameters into the zoo.cfg file

I'm installing zookeeper using these scripts which are great! However there are some use cases that aren't covered. For example, I'd like to add Quorum SSL however there is a limited set of variable that are templated into zoo.cfg.

To make it more generic, I propose a generic dictionary that can be passed. Jinja would then go through this dictionary and apply the key: value pairing into the zoo.cfg

For example, for SSLQuorum I would need the following:

        serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory
        sslQuorum=true
        ssl.quorum.keyStore.location=keystore/location.p12
        ssl.quorum.keyStore.password=changeit

I'll make the PR but i wanted to know if you @sleighzy have any comments?
For example:

    - role: sleighzy.zookeeper
      zookeeper_cfg_params:
        serverCnxnFactory: org.apache.zookeeper.server.NettyServerCnxnFactory
        sslQuorum: true
        ssl.quorum.keyStore.location: keystore/location.p12
        ssl.quorum.keyStore.password: changeit
      become: yes

I'm open to the variable naming too but I think zookeeper_cfg_params covers it

Error Task

Got an error see screenshot, Java version 11, in journalctl -xe it shows:
Jul 27 16:02:09 centos71 systemd[1]: Failed to start Apache Zookeeper.
Jul 27 16:02:09 centos71 systemd[1]: Unit zookeeper.service entered failed state.
Jul 27 16:02:09 centos71 systemd[1]: zookeeper.service failed.
Jul 27 16:02:09 centos71 systemd[1]: zookeeper.service holdoff time over, scheduling restart.
Jul 27 16:02:09 centos71 systemd[1]: Stopped Apache Zookeeper.
Jul 27 16:02:09 centos71 systemd[1]: start request repeated too quickly for zookeeper.service
Jul 27 16:02:09 centos71 systemd[1]: Failed to start Apache Zookeeper.
Jul 27 16:02:09 centos71 systemd[1]: Unit zookeeper.service entered failed state.
Jul 27 16:02:09 centos71 systemd[1]: zookeeper.service failed.

Incorrect cfg for clustering?

First - thanks for putting together this great resource.

I've found that if I try and 'cluster' by doing something like

• name: "Kafka"
  hosts: "{{ kafka_tag }}"
  remote_user: "{{ os_config['remote_user'] }}"
  become: yes
  vars:
  kafka_version: "2.8.1"
  zookeeper_servers: "{{ groups[kafka_tag] }}"
  zookeeper_id: "{{ groups[kafka_tag].index(inventory_hostname) + 1}}"
  vars_files:
  • vars/cluster-config.yml
  • vars/constants.yml
  • vars/os-level-config.yml
    roles:
    • sleighzy.kafka
    • sleighzy.zookeeper

the zoo.cfg ends up as

server.1=ip-10-0-0-65.ec2.internal:2888:3888
server.1=ip-10-0-1-234.ec2.internal:2888:3888
server.1=ip-10-0-2-207.ec2.internal:2888:3888

whereas it should be something like

server.1=ip-10-0-0-65.ec2.internal:2888:3888
server.2=ip-10-0-1-234.ec2.internal:2888:3888
server.3=ip-10-0-2-207.ec2.internal:2888:3888

otherwise all the servers end up in standalone mode.

See https://mail-archives.apache.org/mod_mbox/zookeeper-user/202111.mbox/browser

I don't know if this is easily fixable? I can try and look into it if you like.

Custom location for `zoo.cfg.j2` template or change ansible_nodename

Hello Simon,

I'm trying not to change the downloaded role as this might make it harder to keep it in sync w/ the future versions.

We have an inventory like:

---
zk_cluster_devel:
  hosts:
    "zk-cluster-[01:03].example.com":
      provider: aws
      ......................

As seen above the machine is in AWS and the default hostname is generated in the form ip-internal-ip which is not valid as it does not resolve to anything.
I tried inventory_hostname but it added zk-cluster-01.example.com and then ZK complained w/ "Have smaller server identifier, so dropping the connection".

The only viable solution was hostvars[host].ansible_all_ipv4_addresses[0].
(I would have tried ansible_default_ipv4 but the article https://medium.com/opsops/ansible-default-ipv4-is-not-what-you-think-edb8ab154b10 discouraged me.)

I'll make the PR, but I would like to know what path would you like me to take:

• custom location for zoo.cfg.j2 (like https://github.com/geerlingguy/ansible-role-gitlab/blob/master/defaults/main.yml#L9) or
• change ansible_nodename to ansible_all_ipv4_addresses[0]
• change ansible_nodename but to something configurable w/ ansible_nodename as default value

Thanx!