samdoran/ansible-collection-macos

Ansible collection of roles and modules for use on macOS

ansible-collection-macos's Issues

How to source the Parallels SDK version

https://github.com/samdoran/ansible-collection-macos/tree/main/roles/parallels_sdk#parallels-sdk lists a parallels_sdk_version variable but doesn't explain how to acquire it when there's a need to update or have a fresh install.

Any insights on this?

README Galaxy badge link outdated

TODO: update it to https://galaxy.ansible.com/ui/repo/published/samdoran/macos/

[BUG] `parallels` role incompatible with ansible-core 2.14+

This line https://github.com/samdoran/ansible-collection-macos/blob/67923ac/roles/parallels/tasks/install.yml#L95 triggers the following error:

TASK [samdoran.macos.parallels : Get Parallels version from `pd-autodeploy.zip`] *****************************************************************************
task path: ~/.ansible/collections/ansible_collections/samdoran/macos/roles/parallels/tasks/install.yml:83
fatal: [parallels-07]: FAILED! => {"changed": false, "msg": "Unsupported parameters for (ansible.legacy.command) module: warn. Supported parameters include: _raw_params, _uses_shell, argv, chdir, creates, executable, removes, stdin, stdin_add_newline, strip_empty_ends."}

Per https://stackoverflow.com/a/74560322/595220, warn should be removed. This will require cutting a new release, once fixed.

[BUG] Xcode CLI tools installation is getting skipped on fresh macOS 13.0 (stable) installs

For some reason, the check for the presence of Xcode CLI tools is misbehaving, causing the installation to be skipped, followed by a failing check for the python3 install that fails and opens a window asking for the Xcode CLI tools install.
Logging this here for now. It appears that the SSH output somehow leaks into the task results, causing this.
The string that the check is expecting is present in the stdout value, though, not stderr.

$ ansible-playbook guest-bootstrap-python.yml -vvv \
    --ask-pass \
    --ask-become-pass \
    --limit ansible-core-ci-template-macos-13.0-experiment-7
ansible-playbook 2.10.17
  config file = [...]/ansible.cfg
  configured module search path = ['~/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = ~/.pyenv/versions/venv-py3.8.6/lib/python3.8/site-packages/ansible
  executable location = ~/.pyenv/versions/venv-py3.8.6/bin/ansible-playbook
  python version = 3.8.6 (default, Oct 26 2020, 13:09:51) [GCC 10.2.0]
Using [...]/ansible.cfg as config file
SSH password: 
BECOME password[defaults to SSH password]: 
host_list declined parsing [...]/inventory.yml as it did not pass its verify_file() method
script declined parsing [...]/inventory.yml as it did not pass its verify_file() method
Parsed [...]/inventory.yml inventory source with yaml plugin
Skipping callback 'default', as we already have a stdout callback.
Skipping callback 'minimal', as we already have a stdout callback.
Skipping callback 'oneline', as we already have a stdout callback.

PLAYBOOK: guest-bootstrap-python.yml *************************************************************************************************************************
1 plays in guest-bootstrap-python.yml

PLAY [Pre-provision a fresh macOS 12.3+ install with Python 3] ***********************************************************************************************
META: ran handlers

TASK [samdoran.macos.command_line_tools : Check for Xcode Command Line Tools] ********************************************************************************
task path: ~/.ansible/collections/ansible_collections/samdoran/macos/roles/command_line_tools/tasks/bootstrap.yml:1
<ansible-core-ci-template-macos-13.0-experiment-7> ESTABLISH SSH CONNECTION FOR USER: administrator
<ansible-core-ci-template-macos-13.0-experiment-7> SSH: EXEC sshpass -d11 ssh -C -o ControlMaster=auto -o ControlPersist=600s -o 'User="administrator"' -o ConnectTimeout=10 -J [email protected]:9934 -o ControlPath=~/.ansible/cp/cd44b32e0e -tt ansible-core-ci-template-macos-13.0-experiment-7 'sudo -H -S  -p "[sudo via ansible, key=opiqtcjruxfbodaghsusojogpwexnrwy] password:" -u root /bin/sh -c '"'"'echo BECOME-SUCCESS-opiqtcjruxfbodaghsusojogpwexnrwy ; stat /Library/Developer/CommandLineTools'"'"''
Escalation succeeded
<ansible-core-ci-template-macos-13.0-experiment-7> (1, b'\r\nstat: /Library/Developer/CommandLineTools: stat: No such file or directory\r\n', b'Shared connection to ansible-core-ci-template-macos-13.0-experiment-7 closed.\r\n')
<ansible-core-ci-template-macos-13.0-experiment-7> Failed to connect to the host via ssh: Shared connection to ansible-core-ci-template-macos-13.0-experiment-7 closed.
ok: [ansible-core-ci-template-macos-13.0-experiment-7] => {
    "changed": false,
    "failed_when_result": false,
    "msg": "non-zero return code",
    "rc": 1,
    "stderr": "Shared connection to ansible-core-ci-template-macos-13.0-experiment-7 closed.\r\n",
    "stderr_lines": [
        "Shared connection to ansible-core-ci-template-macos-13.0-experiment-7 closed."
    ],
    "stdout": "\r\nstat: /Library/Developer/CommandLineTools: stat: No such file or directory\r\n",
    "stdout_lines": [
        "",
        "stat: /Library/Developer/CommandLineTools: stat: No such file or directory"
    ]
}

TASK [samdoran.macos.command_line_tools : Create hidden install file] ****************************************************************************************
task path: ~/.ansible/collections/ansible_collections/samdoran/macos/roles/command_line_tools/tasks/bootstrap.yml:14
skipping: [ansible-core-ci-template-macos-13.0-experiment-7] => {
    "changed": false,
    "skip_reason": "Conditional result was False"
}

[...]

TASK [samdoran.macos.command_line_tools : Test default Python] ***********************************************************************************************
task path: ~/.ansible/collections/ansible_collections/samdoran/macos/roles/command_line_tools/tasks/bootstrap.yml:47
<ansible-core-ci-template-macos-13.0-experiment-7> ESTABLISH SSH CONNECTION FOR USER: administrator
<ansible-core-ci-template-macos-13.0-experiment-7> SSH: EXEC sshpass -d11 ssh -C -o ControlMaster=auto -o ControlPersist=600s -o 'User="administrator"' -o ConnectTimeout=10 -J [email protected]:9934 -o ControlPath=~/.ansible/cp/cd44b32e0e ansible-core-ci-template-macos-13.0-experiment-7 '/bin/sh -c '"'"'echo ~administrator && sleep 0'"'"''
<ansible-core-ci-template-macos-13.0-experiment-7> (0, b'/Users/administrator\n', b'')
<ansible-core-ci-template-macos-13.0-experiment-7> ESTABLISH SSH CONNECTION FOR USER: administrator
<ansible-core-ci-template-macos-13.0-experiment-7> SSH: EXEC sshpass -d11 ssh -C -o ControlMaster=auto -o ControlPersist=600s -o 'User="administrator"' -o ConnectTimeout=10 -J [email protected]:9934 -o ControlPath=~/.ansible/cp/cd44b32e0e ansible-core-ci-template-macos-13.0-experiment-7 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /Users/administrator/.ansible/tmp `"&& mkdir "` echo /Users/administrator/.ansible/tmp/ansible-tmp-1667402978.8919308-1397010-168807032675972 `" && echo ansible-tmp-1667402978.8919308-1397010-168807032675972="` echo /Users/administrator/.ansible/tmp/ansible-tmp-1667402978.8919308-1397010-168807032675972 `" ) && sleep 0'"'"''
<ansible-core-ci-template-macos-13.0-experiment-7> (0, b'ansible-tmp-1667402978.8919308-1397010-168807032675972=/Users/administrator/.ansible/tmp/ansible-tmp-1667402978.8919308-1397010-168807032675972\n', b'')
Using module file ~/.pyenv/versions/venv-py3.8.6/lib/python3.8/site-packages/ansible/modules/setup.py
<ansible-core-ci-template-macos-13.0-experiment-7> PUT ~/.ansible/tmp/ansible-local-13960056h9ow_yc/tmpdp3hnodm TO /Users/administrator/.ansible/tmp/ansible-tmp-1667402978.8919308-1397010-168807032675972/AnsiballZ_setup.py
<ansible-core-ci-template-macos-13.0-experiment-7> SSH: EXEC sshpass -d11 scp -C -o ControlMaster=auto -o ControlPersist=600s -o 'User="administrator"' -o ConnectTimeout=10 -J [email protected]:9934 -o ControlPath=~/.ansible/cp/cd44b32e0e ~/.ansible/tmp/ansible-local-13960056h9ow_yc/tmpdp3hnodm '[ansible-core-ci-template-macos-13.0-experiment-7]:/Users/administrator/.ansible/tmp/ansible-tmp-1667402978.8919308-1397010-168807032675972/AnsiballZ_setup.py'
<ansible-core-ci-template-macos-13.0-experiment-7> (0, b'', b'')
<ansible-core-ci-template-macos-13.0-experiment-7> ESTABLISH SSH CONNECTION FOR USER: administrator
<ansible-core-ci-template-macos-13.0-experiment-7> SSH: EXEC sshpass -d11 ssh -C -o ControlMaster=auto -o ControlPersist=600s -o 'User="administrator"' -o ConnectTimeout=10 -J [email protected]:9934 -o ControlPath=~/.ansible/cp/cd44b32e0e ansible-core-ci-template-macos-13.0-experiment-7 '/bin/sh -c '"'"'chmod u+x /Users/administrator/.ansible/tmp/ansible-tmp-1667402978.8919308-1397010-168807032675972/ /Users/administrator/.ansible/tmp/ansible-tmp-1667402978.8919308-1397010-168807032675972/AnsiballZ_setup.py && sleep 0'"'"''
<ansible-core-ci-template-macos-13.0-experiment-7> (0, b'', b'')
<ansible-core-ci-template-macos-13.0-experiment-7> ESTABLISH SSH CONNECTION FOR USER: administrator
<ansible-core-ci-template-macos-13.0-experiment-7> SSH: EXEC sshpass -d11 ssh -C -o ControlMaster=auto -o ControlPersist=600s -o 'User="administrator"' -o ConnectTimeout=10 -J [email protected]:9934 -o ControlPath=~/.ansible/cp/cd44b32e0e -tt ansible-core-ci-template-macos-13.0-experiment-7 '/bin/sh -c '"'"'sudo -H -S  -p "[sudo via ansible, key=wvcyutmdohfracealzyzdgggsucdqdlh] password:" -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-wvcyutmdohfracealzyzdgggsucdqdlh ; /usr/bin/python3 /Users/administrator/.ansible/tmp/ansible-tmp-1667402978.8919308-1397010-168807032675972/AnsiballZ_setup.py'"'"'"'"'"'"'"'"' && sleep 0'"'"''
Escalation succeeded
<ansible-core-ci-template-macos-13.0-experiment-7> (1, b"\r\nxcode-select: note: no developer tools were found at '/Applications/Xcode.app', requesting install. Choose an option in the dialog to download the command line developer tools.\r\n", b'Shared connection to ansible-core-ci-template-macos-13.0-experiment-7 closed.\r\n')
<ansible-core-ci-template-macos-13.0-experiment-7> Failed to connect to the host via ssh: Shared connection to ansible-core-ci-template-macos-13.0-experiment-7 closed.
<ansible-core-ci-template-macos-13.0-experiment-7> ESTABLISH SSH CONNECTION FOR USER: administrator
<ansible-core-ci-template-macos-13.0-experiment-7> SSH: EXEC sshpass -d11 ssh -C -o ControlMaster=auto -o ControlPersist=600s -o 'User="administrator"' -o ConnectTimeout=10 -J [email protected]:9934 -o ControlPath=~/.ansible/cp/cd44b32e0e ansible-core-ci-template-macos-13.0-experiment-7 '/bin/sh -c '"'"'rm -f -r /Users/administrator/.ansible/tmp/ansible-tmp-1667402978.8919308-1397010-168807032675972/ > /dev/null 2>&1 && sleep 0'"'"''
<ansible-core-ci-template-macos-13.0-experiment-7> (0, b'', b'')
fatal: [ansible-core-ci-template-macos-13.0-experiment-7]: FAILED! => {
    "ansible_facts": {},
    "changed": false,
    "failed_modules": {
        "ansible.legacy.setup": {
            "failed": true,
            "module_stderr": "Shared connection to ansible-core-ci-template-macos-13.0-experiment-7 closed.\r\n",
            "module_stdout": "\r\nxcode-select: note: no developer tools were found at '/Applications/Xcode.app', requesting install. Choose an option in the dialog to download the command line developer tools.\r\n",
            "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
            "rc": 1
        }
    },
    "msg": "The following modules failed to execute: ansible.legacy.setup\n"
}

PLAY RECAP ***************************************************************************************************************************************************
ansible-core-ci-template-macos-13.0-experiment-7 : ok=1    changed=0    unreachable=0    failed=1    skipped=4    rescued=0    ignored=0

[Improvement idea] Check for Xcode CLI tools via `pkgutil`

I've been googling about the installer command and stumbled upon this post https://hvops.com/articles/ansible-mac-osx/. Turns out that pkgutil --pkg-info=com.apple.pkg.CLTools_Executables (for macOS X it's com.apple.pkg.DeveloperToolsCLI) exposes metadata that can be used to check whether Xcode CLI tools are present.

$ pkgutil --file-info /Library/Developer/CommandLineTools/usr/bin/python3
volume: /
path: /Library/Developer/CommandLineTools/usr/bin/python3

pkgid: com.apple.pkg.CLTools_Executables
pkg-version: 13.4.0.0.1.1651278267
install-time: 1656705474
uid: 0
gid: 0
mode: 120755

$ pkgutil --file-info /Library/Developer/CommandLineTools
volume: /
path: /Library/Developer/CommandLineTools

pkgid: com.apple.pkg.CLTools_SDK_macOS12
pkg-version: 13.4.0.0.1.1651278267
install-time: 1656705475
uid: 0
gid: 0
mode: 40755

pkgid: com.apple.pkg.CLTools_Executables
pkg-version: 13.4.0.0.1.1651278267
install-time: 1656705474
uid: 0
gid: 0
mode: 40755

pkgid: com.apple.pkg.CLTools_SDK_macOS110
pkg-version: 13.4.0.0.1.1651278267
install-time: 1656705476
uid: 0
gid: 0
mode: 40755

pkgid: com.apple.pkg.CLTools_SwiftBackDeploy
pkg-version: 13.4.0.0.1.1651278267
install-time: 1656705478
uid: 0
gid: 0
mode: 40755

pkgid: com.apple.pkg.CLTools_macOS_SDK
pkg-version: 13.4.0.0.1.1651278267
install-time: 1656705478
uid: 0
gid: 0
mode: 40755

$ pkgutil --pkg-info=com.apple.pkg.CLTools_Executables
package-id: com.apple.pkg.CLTools_Executables
version: 13.4.0.0.1.1651278267
volume: /
location: /
install-time: 1656705474
groups: com.apple.FindSystemFiles.pkg-group
$ echo $?
0

$ pkgutil --pkg-info=com.apple.pkg.DeveloperToolsCLI
No receipt for 'com.apple.pkg.DeveloperToolsCLI' found at '/'.
$ echo $?
1

It seems like using it could be a more elegant/robust solution for making this check, compared to using stat. Have you considered using it?

[MacOS 12.3+] Chicken-egg problem: python vs Xcode CLI tools

On a fresh macos VM install (12.4 in my case), https://github.com/samdoran/ansible-collection-macos/tree/main/roles/python suggests that Xcode CLI tools need to be installed before Python.
It looks like this wouldn't work for macos 12.3+ since it has /usr/bin/python removed and /usr/bin/python3 is just a stub that needs Xcode CLI tools to operate.
The way these roles are written assumes presence of a Python interpreter on the target node. To activate it, one needs to install Xcode CLI tools but installing them through normal Ansible modules requires Python that is not there yet.
As part of going through the image building process in ansible-core-ci, I've faced this issue and my solution was to essentially rewrite https://github.com/samdoran/ansible-collection-macos/blob/main/roles/command_line_tools/tasks/main.yml using ansible.builtin.raw. Having it in a pre-provisioning playbook allows to get the target Ansible-ready and proceed as usual.

So I guess I wonder if this collection would benefit from doing the same — either rewriting the roles not to rely on the pre-existing Python interpreter or providing an extra role specifically to cover this use-case.