christophetd/adaz

:wrench: Deploy customizable Active Directory labs in Azure - automatically.

Adaz: Active Directory Hunting Lab in Azure

This project allows you to easily spin up Active Directory labs in Azure with domain-joined workstations, Windows Event Forwarding, Kibana, and Sysmon using Terraform/Ansible.

It exposes a high-level configuration file for your domain to allow you to customize users, groups and workstations.

dns_name: hunter.lab
dc_name: DC-1

initial_domain_admin:
 username: hunter
 password: MyAdDomain!

organizational_units: {}

users:
- username: christophe
- username: dany

groups:
- dn: CN=Hunters,CN=Users
 members: [christophe]

default_local_admin:
 username: localadmin
 password: Localadmin!

workstations:
- name: XTOF-WKS
 local_admins: [christophe]
- name: DANY-WKS
 local_admins: [dany]

enable_windows_firewall: yes

Features

• Windows Event Forwarding pre-configured
• Audit policies pre-configured
• Sysmon installed
• Logs centralized in an Elasticsearch instance which can easily be queried from the Kibana UI
• Domain easily configurable via YAML configuration file

Here's an incomplete and biaised comparison with DetectionLab:

	Adaz	DetectionLab
Public cloud support	Azure	AWS, Azure (beta)
Expected time to spin up a lab	15-20 minutes	25 minutes
Log management & querying	Elasticsearch+Kibana	Splunk Enterprise
WEF	✔️	✔️
Audit policies	✔️	✔️
Sysmon	✔️	✔️
YAML domain configuration file	✔️	🚫
Multiple Windows 10 workstations support	✔️	🚫
VirtualBox/VMWare support	🚫	✔️
osquery / fleet	🚫(vote!)	✔️
Powershell transcript logging	🚫 (vote!)	✔️
IDS logs	🚫 (vote!)	✔️

Use-cases

• Detection engineering: Having access to clean lab with a standard is a great way to understand what traces common attacks and lateral movement techniques leave behind.

• Learning Active Directory: I often have the need to test GPOs or various AD features (AppLocker, LAPS...). Having a disposable lab is a must for this.

Screenshots

Getting started

Prerequisites

• An Azure subscription. You can create one for free and you get $200 of credits for the first 30 days. Note that this type of subscription has a limit of 4 vCPUs per region, which still allows you to run 1 domain controller and 2 workstations (with the default lab configuration).

• A SSH key in ~/.ssh/id_rsa.pub. Your private key must either be added to ssh-agent (typically, by running ssh-add ~/.ssh/id_rsa once and adding eval "$(ssh-agent -s)" in your .bashrc) or not encrypted with a passphrase.

• Terraform >= 0.12

• Azure CLI

• You must be logged in to your Azure account by running az login. Yu can use az account list to confirm you have access to your Azure subscription

Installation

• Clone this repository

git clone https://github.com/christophetd/Adaz.git

• Create a virtual env and install Ansible dependencies

# Note: the virtual env needs to be in ansible/venv
python3 -m venv ansible/venv 
source ansible/venv/bin/activate
pip install -r ansible/requirements.txt
deactivate

• Initialize Terraform

cd terraform
terraform init

Usage

Optionally edit domain.yml according to your needs (reference here), then run:

terraform apply

Resource creation and provisioning takes 15-20 minutes. Once finished, you will have an output similar to:

dc_public_ip = 13.89.191.140
kibana_url = http://52.176.3.250:5601
what_next =
####################
###  WHAT NEXT?  ###
####################

Check out your logs in Kibana:
http://52.176.3.250:5601

RDP to your domain controller:
xfreerdp /v:13.89.191.140 /u:hunter.lab\\hunter '/p:Hunt3r123.' +clipboard /cert-ignore

RDP to a workstation:
xfreerdp /v:52.176.5.229 /u:localadmin '/p:Localadmin!' +clipboard /cert-ignore


workstations_public_ips = {
  "DANY-WKS" = "52.165.182.15"
  "XTOF-WKS" = "52.176.5.229"
}

Don't worry if during the provisioning you see a few messages looking like FAILED - RETRYING: List Kibana index templates (xx retries left)

By default, resources are deployed in the West Europe region under a resource group ad-hunting-lab. You can control the region with a Terraform variable:

terraform apply -var 'region=East US 2'

Documentation

• Frequently Asked Questions
• domain.yml reference
• Audit policies enabled
• Detailed architecture
• Troubleshooting common issues
• Common Operations: adding users, destroying the lab, etc.
• Project structure and directory organization
• Terraform variables available

Community

Talks / posts referencing Adaz:

• Purple Teaming Cloud Identity: Simulation Labs for Red and Blue Teams
• (French) HoneyWISE : stratégie d’exploitation d’honeytokens en environnement Active Directory

Roadmap

I will heavily rely on the number of thumbs up votes you will leave on feature-proposal issues for the next features!

Suggestions and bugs

Feel free to open an issue or to tweet @christophetd.

adaz's Issues

kibana not provisioned, 404 error ?

What is not working?
Provisioning works for everything except kibana. VM is created with nic etc, but no service.
I have warning during terraform provisioning

What OS are you using?
Mac OS X

Full Terraform / Ansible output?

azurerm_virtual_machine.es_kibana (local-exec): Executing: ["/bin/sh" "-c" "/bin/bash -c 'source venv/bin/activate && ANSIBLE_HOST_KEY_CHECKING=false ansible-playbook elasticsearch-kibana.yml -v'"]
azurerm_virtual_machine.es_kibana (local-exec): Using /Users/mathieuchateau/Local/Adaz-main/ansible/ansible.cfg as config file
azurerm_virtual_machine.es_kibana (local-exec): [WARNING]:  * Failed to parse /Users/mathieuchateau/Local/Adaz-
azurerm_virtual_machine.es_kibana (local-exec): main/ansible/inventory_azure_rm.yml with auto plugin: a batched request failed
azurerm_virtual_machine.es_kibana (local-exec): with status code 404, url
azurerm_virtual_machine.es_kibana (local-exec): /subscriptions/XXXXXXXX/resourceGroups/ad-hunting-
azurerm_virtual_machine.es_kibana (local-exec): lab/providers/Microsoft.Compute/virtualMachines
azurerm_virtual_machine.es_kibana (local-exec): [WARNING]:  * Failed to parse /Users/mathieuchateau/Local/Adaz-
azurerm_virtual_machine.es_kibana (local-exec): main/ansible/inventory_azure_rm.yml with yaml plugin: Plugin configuration YAML
azurerm_virtual_machine.es_kibana (local-exec): file, not YAML inventory
azurerm_virtual_machine.es_kibana (local-exec): [WARNING]:  * Failed to parse /Users/mathieuchateau/Local/Adaz-
azurerm_virtual_machine.es_kibana (local-exec): main/ansible/inventory_azure_rm.yml with ini plugin: Invalid host pattern
azurerm_virtual_machine.es_kibana (local-exec): 'plugin:' supplied, ending in ':' is not allowed, this character is reserved to
azurerm_virtual_machine.es_kibana (local-exec): provide a port.
azurerm_virtual_machine.es_kibana (local-exec): [WARNING]: Unable to parse /Users/mathieuchateau/Local/Adaz-
azurerm_virtual_machine.es_kibana (local-exec): main/ansible/inventory_azure_rm.yml as an inventory source
azurerm_virtual_machine.es_kibana (local-exec): [WARNING]: No inventory was parsed, only implicit localhost is available
azurerm_virtual_machine.es_kibana (local-exec): [WARNING]: provided hosts list is empty, only localhost is available. Note that
azurerm_virtual_machine.es_kibana (local-exec): the implicit localhost does not match 'all'
azurerm_virtual_machine.es_kibana (local-exec): [WARNING]: Could not match supplied host pattern, ignoring: elasticsearch

Error while running Terraform apply. Using WSL to run the scripts.

0f158/resourceGroups/ad-hunting-lab/providers/Microsoft.Compute/virtualMachines/DANY-WKS]

Error: Error running command '/bin/bash -c 'source venv/bin/activate && ansible-playbook domain-controllers.yml --tags=common,base -v'': exit status 1. Output: /bin/bash: venv/bin/activate: No such file or directory

Error: Error running command '/bin/bash -c 'source venv/bin/activate && ANSIBLE_HOST_KEY_CHECKING=false ansible-playbook elasticsearch-kibana.yml -v'': exit status 1. Output: /bin/bash: venv/bin/activate: No such file or directory

Add a Suricata IDS instance sending logs to Elasticsearch

Provide cost measurements for other regions

At least an Europe and an Asia region

Is it possible to add more servers and workstations?

I see that there are 3 workstations created in this lab? Is it possible to add more workstations? if so, which config file should i change? Also, is it possible to add servers to the mix to join the domain. some documentation along those lines will be helpful.

Allow specifying keyboard layout of machines

Currently: manually run Set-WinUserLanguageList -LanguageList fr-CH, en-US -Force

Allow specificying the organizational unit of workstations in domain.yml

Provide Fleet server with osquery configured on workstations

Add a (optional?) Kali box to the workstation subnet

A Kali box would be helpful in simulating various attacks on the network.

Enable powershell transcript logs

Support multiple domain controllers

Currently, only having a single domain controller is supported. It might be useful for some use-cases to support having multiple DCs

More errors while trying to create. Created a lab, didn't work properly so deleted and tried again.

azurerm_virtual_machine.workstation[0]: Refreshing state... [id=/subscriptions/2f8c4cef-a6a5-473a-8535-ce9d5100f158/resourceGroups/ad-hunting-lab/providers/Microsoft.Compute/virtualMachines/XTOF-WKS]
azurerm_virtual_machine.workstation[1]: Refreshing state... [id=/subscriptions/2f8c4cef-a6a5-473a-8535-ce9d5100f158/resourceGroups/ad-hunting-lab/providers/Microsoft.Compute/virtualMachines/DANY-WKS]
azurerm_virtual_machine.es_kibana: Refreshing state... [id=/subscriptions/2f8c4cef-a6a5-473a-8535-ce9d5100f158/resourceGroups/ad-hunting-lab/providers/Microsoft.Compute/virtualMachines/es-kibana]
azurerm_virtual_machine.dc: Refreshing state... [id=/subscriptions/2f8c4cef-a6a5-473a-8535-ce9d5100f158/resourceGroups/ad-hunting-lab/providers/Microsoft.Compute/virtualMachines/domain-controller]

Error: Network Interface "ad-lab-nic" (Resource Group "ad-hunting-lab") was not found!

Error: Network Interface "es-kibana-nic" (Resource Group "ad-hunting-lab") was not found!



Error: Network Interface "ad-lab-wks-1-nic" (Resource Group "ad-hunting-lab") was not found!



Error: Network Interface "ad-lab-wks-0-nic" (Resource Group "ad-hunting-lab") was not found!



Error: Invalid index

  on data.tf line 9, in data "azurerm_public_ip" "workstation":
   9:   name = azurerm_public_ip.workstation[count.index].name
    |----------------
    | azurerm_public_ip.workstation is empty tuple
    | count.index is 0

The given key does not identify an element in this collection value.


Error: Invalid index

  on data.tf line 9, in data "azurerm_public_ip" "workstation":
   9:   name = azurerm_public_ip.workstation[count.index].name
    |----------------
    | azurerm_public_ip.workstation is empty tuple
    | count.index is 1

The given key does not identify an element in this collection value.


Error: Invalid index

  on outputs.tf line 23, in output "what_next":
  23: xfreerdp /v:${azurerm_public_ip.workstation[0].ip_address} /u:${local.domain.default_local_admin.username} '/p:${local.domain.default_local_admin.password}' +clipboard /cert-ignore
    |----------------
    | azurerm_public_ip.workstation is empty tuple

The given key does not identify an element in this collection value.

prajganesh@DESKTOP-I8M1347:~/Adaz/terraform$

Error: Error making request: Get "http://ipv4.icanhazip.com": dial tcp 10.25.207.164:80: i/o timeout

What is not working?
A clear and concise description of what the bug is.

Error: Error making request: Get "http://ipv4.icanhazip.com": dial tcp 10.25.207.164:80: i/o timeout
│
│ with data.http.public_ip,
│ on data.tf line 21, in data "http" "public_ip":
│ 21: data "http" "public_ip" {

---

I corrected this by adding the following to data.tf

data "http" "public_ip" {
url = "https://api.ipify.org"
}

Write CI pipeline and integration tests with InSpec

c.f. https://github.com/inspec/inspec-azure

terraform needs ssh key passphrase, but mixed with terraform output

When doing an apply, terraform will prompt for passphrase to access ssh key.

What happens it that terraform does not stop waiting for it, it continues to try provisioning. passphrase request is mixed in the terraform output. If missed, deploy will fail.

example:

Not sure if there is a cleaner way, but a wait would help catch maybe ?

- name: Sleep for 30 seconds and continue with play
      wait_for: 
        timeout: 30

Cannot Install Elastic VM

Hi there, for some reason I cannot install elastic vm. Follow the error:

TASK [winlogbeat : Find Elasticsearch IP] **************************************
fatal: [domain-controller_1493]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'elasticsearch'\n\nThe error appears to be in '/home/francesco/lab/Adaz/ansible/roles/winlogbeat/tasks/main.yml': line 8, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Find Elasticsearch IP\n  ^ here\n"}

RUNNING HANDLER [wef-collector : restart wec service] **************************

PLAY RECAP *********************************************************************
domain-controller_1493     : ok=28   changed=22   unreachable=0    failed=1    skipped=3    rescued=0    ignored=0   





Error: compute.VirtualMachinesClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="InvalidParameter" Message="The value of parameter linuxConfiguration.ssh.publicKeys.keyData is invalid." Target="linuxConfiguration.ssh.publicKeys.keyData"

  on elasticsearch_kibana.tf line 24, in resource "azurerm_virtual_machine" "es_kibana":
  24: resource "azurerm_virtual_machine" "es_kibana" {

Can you please point me what´s wrong?

Thanks

Failed: Ensure domain groups exist

Hey thanks for your project. I am getting this error and I could not figured out how to fix it:

> ansible-playbook domain-controllers.yml --skip-tags=base -v
[...]
TASK [domain-controller : Ensure domain groups exist] *******************************************************************************************************************************************************************************************************
failed: [domain-controller_86da] (item={'dn': 'CN=Hunters,CN=Users', 'members': ['christophe', 'dany']}) => {"ansible_loop_var": "item", "changed": false, "created": false, "item": {"dn": "CN=Hunters,CN=Users", "members": ["christophe", "dany"]}, "msg": "the group path CN=Users,,DC=hunter,DC=lab does not exist, please specify a valid LDAP path"}

Note the double commas in CN=Users,,DC=hunter.

My domain.yml:

# Domain groups
 groups:
 - dn: CN=Hunters,CN=Users
   members: [christophe, dany]

I am sure the problem is in here:

path: "{{ item.dn.split(',') | reject('search', item.dn.split(',')[0]) | map('regex_replace', '(.*)', '\\1,') | join('') }}{{ domain_root_path }}"

Any suggestions?

Maybe SecurityOnion can shorten work effort

Hi,
I´m comming here by accident and see a few feature requests because Kibana, Suricata, Sigma rules, ...
Just as hint: Do you know Security Onion? Its an open source Saltstack based system, where ELK, Kibana, Surricata, Sigma rules, ... is integrated to a whole ecosystem.
Maybe this helps you keeping the effort low.
https://github.com/Security-Onion-Solutions/securityonion
https://docs.securityonion.net/en/2.3/
(I´m not in any relation to SecurityOnion, just a user of)

VirtualBox/VMWare support

es_kibana timed out

Hey @christophetd - congrats on launching this! I'm attempting to bring it online, but I was just met with the following:

azurerm_virtual_machine.es_kibana (local-exec): fatal: [es-kibana_31cd]: FAILED! => {"changed": false, "elapsed": 601, "msg": "timed out waiting for ping module test success: Failed to connect to the host via ssh: [email protected]: Permission denied (publickey)."}

azurerm_virtual_machine.es_kibana (local-exec): PLAY RECAP *********************************************************************
azurerm_virtual_machine.es_kibana (local-exec): es-kibana_31cd             : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0



Error: Error running command '/bin/bash -c 'source venv/bin/activate && ANSIBLE_HOST_KEY_CHECKING=false ansible-playbook elasticsearch-kibana.yml -v'': exit status 2. Output: Using /Users/clong/git-repos/Adaz/ansible/ansible.cfg as config file

PLAY [Configure Elasticsearch and Kibana] **************************************

TASK [elasticsearch-kibana : wait_for_connection] ******************************
fatal: [es-kibana_31cd]: FAILED! => {"changed": false, "elapsed": 601, "msg": "timed out waiting for ping module test success: Failed to connect to the host via ssh: [email protected]: Permission denied (publickey)."}

PLAY RECAP *********************************************************************
es-kibana_31cd             : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

Is there maybe a timeout value that needs to be adjusted here?

Pre-install common Sysinternal tools (Procmon, Process Hacker...)

End-to-end tests with Terratest

Allow dynamically specifying a resource group name

It looks like AZURE_RESOURCE_GROUPS doesn't actually restrict the set of hosts considered by the AzureRM inventory plugin

Ansible Domain Controller playbook fails due to outdated pywinrm version

Macos w/ Python 3.11.5

TASK [domain-controller : Ensure necessary Microsoft features are installed] ************************************************************************************************************* failed: [domain-controller_4da5] (item=AD-Domain-Services) => {"ansible_loop_var": "item", "item": "AD-Domain-Services", "msg": "ntlm: module 'inspect' has no attribute 'getargspec'", "unreachable": true} failed: [domain-controller_4da5] (item=RSAT-AD-AdminCenter) => {"ansible_loop_var": "item", "item": "RSAT-AD-AdminCenter", "msg": "ntlm: module 'inspect' has no attribute 'getargspec'", "unreachable": true} failed: [domain-controller_4da5] (item=RSAT-ADDS-Tools) => {"ansible_loop_var": "item", "item": "RSAT-ADDS-Tools", "msg": "ntlm: module 'inspect' has no attribute 'getargspec'", "unreachable": true} fatal: [domain-controller_4da5]: UNREACHABLE! => {"changed": false, "msg": "All items completed", "results": [{"ansible_loop_var": "item", "item": "AD-Domain-Services", "msg": "ntlm: module 'inspect' has no attribute 'getargspec'", "unreachable": true}, {"ansible_loop_var": "item", "item": "RSAT-AD-AdminCenter", "msg": "ntlm: module 'inspect' has no attribute 'getargspec'", "unreachable": true}, {"ansible_loop_var": "item", "item": "RSAT-ADDS-Tools", "msg": "ntlm: module 'inspect' has no attribute 'getargspec'", "unreachable": true}]}

Upgrade pywinrm==0.4.1 to pywinrm==0.4.3 in ansible/requirements.txt

Could not match supplied host pattern, ignoring: elasticsearch

The setup looks to be working fine and I receive the what's next output, but I am not able to access the Kibana dashboard (ERR_CONNECTION_REFUSED).
The only ports open on the Elastic/Kibana machine are 22,53, and 25324 (OMSAgent), so maybe Elastic and Kibana wasn't installed and started on the machine?

I went through the Terraform output and found multiple warnings stating;

`azurerm_virtual_machine.es_kibana (local-exec): Executing: ["/bin/sh" "-c" "/bin/bash -c 'source venv/bin/activate && ANSIBLE_HOST_KEY_CHECKING=false ansible-playbook elasticsearch-kibana.yml -v'"]
azurerm_virtual_machine.workstation[0]: Still creating... [1m40s elapsed]
azurerm_virtual_machine.workstation[1]: Still creating... [1m40s elapsed]
azurerm_virtual_machine.es_kibana (local-exec): [WARNING]: Ansible is being run in a world writable directory
azurerm_virtual_machine.es_kibana (local-exec): (/home/nexxic/Adaz/ansible), ignoring it as an ansible.cfg source. For more
azurerm_virtual_machine.es_kibana (local-exec): information see
azurerm_virtual_machine.es_kibana (local-exec): https://docs.ansible.com/ansible/devel/reference_appendices/config.html#cfg-in-
azurerm_virtual_machine.es_kibana (local-exec): world-writable-dir
azurerm_virtual_machine.dc: Still creating... [1m50s elapsed]
azurerm_virtual_machine.es_kibana (local-exec): **No config file found; using defaults**
azurerm_virtual_machine.es_kibana (local-exec): [WARNING]: No inventory was parsed, only implicit localhost is available
azurerm_virtual_machine.es_kibana (local-exec): [WARNING]: provided hosts list is empty, only localhost is available. Note that
azurerm_virtual_machine.es_kibana (local-exec): the implicit localhost does not match 'all'
azurerm_virtual_machine.es_kibana: Still creating... [1m50s elapsed]
azurerm_virtual_machine.es_kibana (local-exec): **[WARNING]: Could not match supplied host pattern, ignoring: elasticsearch**

azurerm_virtual_machine.es_kibana (local-exec): PLAY [Configure Elasticsearch and Kibana] **************************************
azurerm_virtual_machine.es_kibana (local-exec): skipping: no hosts matched

azurerm_virtual_machine.es_kibana (local-exec): PLAY RECAP *********************************************************************`

And the same for Domain_controllers:

null_resource.provision_rest_of_dc_after_creation: Provisioning with 'local-exec'...
null_resource.provision_rest_of_dc_after_creation (local-exec): Executing: ["/bin/sh" "-c" "/bin/bash -c 'source venv/bin/activate && ansible-playbook domain-controllers.yml --skip-tags=base -v'"]
null_resource.provision_rest_of_dc_after_creation (local-exec): [WARNING]: Ansible is being run in a world writable directory
null_resource.provision_rest_of_dc_after_creation (local-exec): (/home/nexxic/Adaz/ansible), ignoring it as an ansible.cfg source. For more
null_resource.provision_rest_of_dc_after_creation (local-exec): information see
null_resource.provision_rest_of_dc_after_creation (local-exec): https://docs.ansible.com/ansible/devel/reference_appendices/config.html#cfg-in-
null_resource.provision_rest_of_dc_after_creation (local-exec): world-writable-dir
null_resource.provision_rest_of_dc_after_creation (local-exec): No config file found; using defaults
null_resource.provision_rest_of_dc_after_creation (local-exec): [WARNING]: No inventory was parsed, only implicit localhost is available
null_resource.provision_rest_of_dc_after_creation (local-exec): [WARNING]: provided hosts list is empty, only localhost is available. Note that
null_resource.provision_rest_of_dc_after_creation (local-exec): the implicit localhost does not match 'all'
null_resource.provision_rest_of_dc_after_creation (local-exec): [WARNING]: Could not match supplied host pattern, ignoring: domain_controllers

Any idea what the problem might be? Looks like it's not finding the correct configuration for the machine?

VPN instead of inbound rules

how about VPN in the ES box for example to join the network instead of the NSG rules which might affect the flexibility if you keep connecting from different network.

Implement additional network segmentation between workstations and servers subnet

By creating new NSG and attaching them to the subnets

how to apply ansible afterward first run?

I guess I am officially a noob on this.. :)

When using you config, without changing anything except Azure Region, all works:
-Kibana ready and receiving logs
-DC VM with AD accounts,
-WKS domain joined and forwarding logs

If i add/remove WKS VM and do another terraform apply, it's created again, but no domain join etc.. Nothing in console output about an error, just not doing it. Tried multiple times, same result.

It appears ansible is called through resource "provision_workstation_once_dc_has_been_created".
After fist run this ressource is kept provisioned, so ansible is not called on new VM later.

During hardening and others tests, it's neat to get a new VM and try again.

I found at least a workaround by destroying it and creating WKS VM again:
terraform destroy --target null_resource.provision_workstation_once_dc_has_been_created

Then ansible is applied again. But may break others already deployed WKS I guess (did not try)

Build integration with Sigma rules

Suggestion:

• Clone Sigma rules repository or allow to specify custom ones
• Convert them to Elastalert format using sigmac
• Run Elastalert on the Elasticsearch/Kibana VM
• Output alerts to Elasticsearch

"context deadline exceeded" on instantiation

I had a random error when running terraform apply -var "region=West Europe" which I never had before:

Error: Error flattening `storage_os_disk`: &errors.errorString{s:"Error retrieving Disk \"wks-1-os-disk\" (Resource Group \"AD-HUNTING-LAB\"): compute.DisksClient#Get: Failure sending request: StatusCode=0 -- Original Error: context deadline exceeded"}

  on workstations.tf line 23, in resource "azurerm_virtual_machine" "workstation":
  23: resource "azurerm_virtual_machine" "workstation" {



Error: Error flattening `storage_os_disk`: &errors.errorString{s:"Error retrieving Disk \"wks-0-os-disk\" (Resource Group \"AD-HUNTING-LAB\"): compute.DisksClient#Get: Failure sending request: StatusCode=0 -- Original Error: context deadline exceeded"}

  on workstations.tf line 23, in resource "azurerm_virtual_machine" "workstation":
  23: resource "azurerm_virtual_machine" "workstation" {

This specific instantiation was particularly slow, maybe it was a temporary Azure glitch?

MicrosoftWindowsDesktop:Windows-10:19h1-pron:latest' is not available.

What is not working?

After running terraform apply; get the following error.

╷
│ Error: compute.VirtualMachinesClient#CreateOrUpdate: Failure sending request: StatusCode=404 -- Original Error: Code="PlatformImageNotFound" Message="The platform image 'MicrosoftWindowsDesktop:Windows-10:19h1-pron:latest' is not available. Verify that all fields in the storage profile are correct. For more details about storage profile information, please refer to https://aka.ms/storageprofile" Target="imageReference"
│
│ with azurerm_virtual_machine.workstation[1],
│ on workstations.tf line 23, in resource "azurerm_virtual_machine" "workstation":
│ 23: resource "azurerm_virtual_machine" "workstation" {
│
╵
╷
│ Error: compute.VirtualMachinesClient#CreateOrUpdate: Failure sending request: StatusCode=404 -- Original Error: Code="PlatformImageNotFound" Message="The platform image 'MicrosoftWindowsDesktop:Windows-10:19h1-pron:latest' is not available. Verify that all fields in the storage profile are correct. For more details about storage profile information, please refer to https://aka.ms/storageprofile" Target="imageReference"
│
│ with azurerm_virtual_machine.workstation[0],
│ on workstations.tf line 23, in resource "azurerm_virtual_machine" "workstation":
│ 23: resource "azurerm_virtual_machine" "workstation" {
│
╵

What OS are you using?
Mac OS X

Your domain.yml file?
If you customized the domain.yml file, please include it below

No, customisation

Full Terraform / Ansible output?
If applicable, please include the full Terraform / Ansible output

Add @domain.tld to group members in win_group_membership

c.f. #23 and https://github.com/christophetd/Adaz/blob/master/ansible/roles/domain-member/tasks/main.yml#L37-L41

Provide Packer resources and procedures to build a base image for DC and workstations

Error while installing Ansible requirements.txt. Using WSL for installation

Building wheels for collected packages: pywinrm, ansible, antlr4-python3-runtime, jsmin, vsts-cd-manager, jsondiff, sshtunnel, PyYAML
  Building wheel for pywinrm (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/prajganesh/ansible/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-ub3dmkbj/pywinrm/setup.py'"'"'; __file__='"'"'/tmp/pip-install-ub3dmkbj/pywinrm/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-m7xs3cur
       cwd: /tmp/pip-install-ub3dmkbj/pywinrm/
  Complete output (6 lines):
  usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: setup.py --help [cmd1 cmd2 ...]
     or: setup.py --help-commands
     or: setup.py cmd --help

  error: invalid command 'bdist_wheel'
  ----------------------------------------
  ERROR: Failed building wheel for pywinrm
  Running setup.py clean for pywinrm
  Building wheel for ansible (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/prajganesh/ansible/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-ub3dmkbj/ansible/setup.py'"'"'; __file__='"'"'/tmp/pip-install-ub3dmkbj/ansible/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-pasgs4wq
       cwd: /tmp/pip-install-ub3dmkbj/ansible/
  Complete output (6 lines):
  usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: setup.py --help [cmd1 cmd2 ...]
     or: setup.py --help-commands
     or: setup.py cmd --help

  error: invalid command 'bdist_wheel'
  ----------------------------------------
  ERROR: Failed building wheel for ansible
  Running setup.py clean for ansible
  Building wheel for antlr4-python3-runtime (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/prajganesh/ansible/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-ub3dmkbj/antlr4-python3-runtime/setup.py'"'"'; __file__='"'"'/tmp/pip-install-ub3dmkbj/antlr4-python3-runtime/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-_h6lgcbv
       cwd: /tmp/pip-install-ub3dmkbj/antlr4-python3-runtime/
  Complete output (6 lines):
  usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: setup.py --help [cmd1 cmd2 ...]
     or: setup.py --help-commands
     or: setup.py cmd --help

  error: invalid command 'bdist_wheel'
  ----------------------------------------
  ERROR: Failed building wheel for antlr4-python3-runtime
  Running setup.py clean for antlr4-python3-runtime
  Building wheel for jsmin (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/prajganesh/ansible/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-ub3dmkbj/jsmin/setup.py'"'"'; __file__='"'"'/tmp/pip-install-ub3dmkbj/jsmin/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-i5i6ofti
       cwd: /tmp/pip-install-ub3dmkbj/jsmin/
  Complete output (6 lines):
  usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: setup.py --help [cmd1 cmd2 ...]
     or: setup.py --help-commands
     or: setup.py cmd --help

  error: invalid command 'bdist_wheel'
  ----------------------------------------
  ERROR: Failed building wheel for jsmin
  Running setup.py clean for jsmin
  Building wheel for vsts-cd-manager (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/prajganesh/ansible/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-ub3dmkbj/vsts-cd-manager/setup.py'"'"'; __file__='"'"'/tmp/pip-install-ub3dmkbj/vsts-cd-manager/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-_tkbcahe
       cwd: /tmp/pip-install-ub3dmkbj/vsts-cd-manager/
  Complete output (6 lines):
  usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: setup.py --help [cmd1 cmd2 ...]
     or: setup.py --help-commands
     or: setup.py cmd --help

  error: invalid command 'bdist_wheel'
  ----------------------------------------
  ERROR: Failed building wheel for vsts-cd-manager
  Running setup.py clean for vsts-cd-manager
  Building wheel for jsondiff (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/prajganesh/ansible/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-ub3dmkbj/jsondiff/setup.py'"'"'; __file__='"'"'/tmp/pip-install-ub3dmkbj/jsondiff/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-jbe07zkb
       cwd: /tmp/pip-install-ub3dmkbj/jsondiff/
  Complete output (6 lines):
  usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: setup.py --help [cmd1 cmd2 ...]
     or: setup.py --help-commands
     or: setup.py cmd --help

  error: invalid command 'bdist_wheel'
  ----------------------------------------
  ERROR: Failed building wheel for jsondiff
  Running setup.py clean for jsondiff
  Building wheel for sshtunnel (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/prajganesh/ansible/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-ub3dmkbj/sshtunnel/setup.py'"'"'; __file__='"'"'/tmp/pip-install-ub3dmkbj/sshtunnel/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-539h7qj5
       cwd: /tmp/pip-install-ub3dmkbj/sshtunnel/
  Complete output (6 lines):
  usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: setup.py --help [cmd1 cmd2 ...]
     or: setup.py --help-commands
     or: setup.py cmd --help

  error: invalid command 'bdist_wheel'
  ----------------------------------------
  ERROR: Failed building wheel for sshtunnel
  Running setup.py clean for sshtunnel
  Building wheel for PyYAML (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/prajganesh/ansible/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-ub3dmkbj/PyYAML/setup.py'"'"'; __file__='"'"'/tmp/pip-install-ub3dmkbj/PyYAML/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-di9vph47
       cwd: /tmp/pip-install-ub3dmkbj/PyYAML/
  Complete output (6 lines):
  usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: setup.py --help [cmd1 cmd2 ...]
     or: setup.py --help-commands
     or: setup.py cmd --help

  error: invalid command 'bdist_wheel'
  ----------------------------------------
  ERROR: Failed building wheel for PyYAML

Ansible installation fails due to an error with `jsmin~=2.2.2`

Hi @christophetd! Thanks for making this awesome tool!

I ran into this issue during setup:

What is not working?
Ansible installation fails due to an error with jsmin~=2.2.2 (a dependency of azure-cli==2.5.1).

$ pip install -r ansible/requirements.txt
…
Collecting jsmin~=2.2.2
…
  error in jsmin setup command: use_2to3 is invalid.
…
ERROR: Could not find a version that satisfies the requirement jsmin~=2.2.2 (from azure-cli) 

Cause of the error
Quoting from Azure/azure-cli#19468 (comment):

• setuptools deprecated use_2to3 in v58.0.0
• azure-cli requires jsmin
• jsmin requires use_2to3, so it doesn't work with the latest setuptools

This new version of setuptools seems to be bundled in Python 3.9.8, 3.10.1, and all version released since Nov 2021.

Workaround
A quick workaround is to pin setuptools to a version <58:

pip install setuptools==57.5.0

The long-term fix is likely to update azure-cli to a version >=2.28.1, which is when the bug was fixed.

What OS are you using?
macOS 11.6

Full output

~/src $ git clone https://github.com/christophetd/Adaz.git
Cloning into 'Adaz'...
remote: Enumerating objects: 190, done.
remote: Counting objects: 100% (25/25), done.
remote: Compressing objects: 100% (24/24), done.
remote: Total 190 (delta 9), reused 2 (delta 0), pack-reused 165
Receiving objects: 100% (190/190), 357.57 KiB, done.
Resolving deltas: 100% (49/49), done.
~/src $ cd Adaz
~/src/Adaz (master) $ python3 -m venv ansible/venv
~/src/Adaz (master) $ source ansible/venv/bin/activate
~/src/Adaz (master) $ pip install -r ansible/requirements.txt --no-cache-dir
Collecting pywinrm==0.4.1
  Downloading pywinrm-0.4.1.tar.gz (36 kB)
  Preparing metadata (setup.py) ... done
Collecting requests==2.23.0
  Downloading requests-2.23.0-py2.py3-none-any.whl (58 kB)
Collecting msrest==0.6.13
  Downloading msrest-0.6.13-py2.py3-none-any.whl (83 kB)
Collecting msrestazure==0.6.3
  Downloading msrestazure-0.6.3-py2.py3-none-any.whl (40 kB)
Collecting azure-cli==2.5.1
  Downloading azure_cli-2.5.1-py3-none-any.whl (1.6 MB)
Collecting ansible==2.9.9
  Downloading ansible-2.9.9.tar.gz (14.2 MB)
  Preparing metadata (setup.py) ... done
Collecting xmltodict
  Downloading xmltodict-0.12.0-py2.py3-none-any.whl (9.2 kB)
Collecting requests_ntlm>=0.3.0
  Downloading requests_ntlm-1.1.0-py2.py3-none-any.whl (5.7 kB)
Collecting six
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
  Downloading urllib3-1.25.11-py2.py3-none-any.whl (127 kB)
Collecting idna<3,>=2.5
  Downloading idna-2.10-py2.py3-none-any.whl (58 kB)
Collecting chardet<4,>=3.0.2
  Downloading chardet-3.0.4-py2.py3-none-any.whl (133 kB)
Collecting certifi>=2017.4.17
  Downloading certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting requests-oauthlib>=0.5.0
  Downloading requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
  Downloading isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting adal<2.0.0,>=0.6.0
  Downloading adal-1.2.7-py2.py3-none-any.whl (55 kB)
Collecting azure-mgmt-eventhub~=3.0.0
  Downloading azure_mgmt_eventhub-3.0.0-py2.py3-none-any.whl (95 kB)
Collecting azure-mgmt-recoveryservicesbackup~=0.6.0
  Downloading azure_mgmt_recoveryservicesbackup-0.6.0-py2.py3-none-any.whl (194 kB)
Collecting azure-mgmt-marketplaceordering~=0.1
  Downloading azure_mgmt_marketplaceordering-0.2.1-py2.py3-none-any.whl (21 kB)
Collecting scp~=0.13.2
  Downloading scp-0.13.6-py2.py3-none-any.whl (8.2 kB)
Collecting azure-mgmt-hdinsight~=1.4.0
  Downloading azure_mgmt_hdinsight-1.4.0-py2.py3-none-any.whl (54 kB)
Collecting azure-mgmt-media>=1.1.1,~=1.1
  Downloading azure_mgmt_media-1.1.1-py2.py3-none-any.whl (341 kB)
Collecting azure-mgmt-devtestlabs~=2.2
  Downloading azure_mgmt_devtestlabs-2.2.0-py2.py3-none-any.whl (194 kB)
Collecting azure-mgmt-relay~=0.1.0
  Downloading azure_mgmt_relay-0.1.0-py2.py3-none-any.whl (36 kB)
Collecting azure-keyvault~=1.1
  Downloading azure_keyvault-1.1.0-py2.py3-none-any.whl (352 kB)
Collecting azure-mgmt-reservations==0.6.0
  Downloading azure_mgmt_reservations-0.6.0-py2.py3-none-any.whl (33 kB)
Collecting azure-batch~=9.0
  Downloading azure_batch-9.0.0-py2.py3-none-any.whl (229 kB)
Collecting azure-mgmt-security~=0.1.0
  Downloading azure_mgmt_security-0.1.0-py2.py3-none-any.whl (144 kB)
Collecting azure-cli-telemetry<2.0,>=1.0.2
  Downloading azure_cli_telemetry-1.0.6-py3-none-any.whl (10 kB)
Collecting azure-mgmt-compute~=12.0
  Downloading azure_mgmt_compute-12.1.0-py2.py3-none-any.whl (1.6 MB)
Collecting paramiko<3.0.0,>=2.0.8
  Downloading paramiko-2.9.2-py2.py3-none-any.whl (210 kB)
Collecting azure-mgmt-containerregistry~=3.0.0rc11
  Downloading azure_mgmt_containerregistry-3.0.0rc17-py2.py3-none-any.whl (660 kB)
Collecting azure-mgmt-containerservice~=9.0.1
  Downloading azure_mgmt_containerservice-9.0.1-py2.py3-none-any.whl (499 kB)
Collecting azure-mgmt-authorization~=0.52.0
  Downloading azure_mgmt_authorization-0.52.0-py2.py3-none-any.whl (112 kB)
Collecting azure-graphrbac~=0.60.0
  Downloading azure_graphrbac-0.60.0-py2.py3-none-any.whl (139 kB)
Collecting azure-mgmt-containerinstance~=1.4
  Downloading azure_mgmt_containerinstance-1.5.0-py2.py3-none-any.whl (96 kB)
Collecting azure-mgmt-botservice~=0.2.0
  Downloading azure_mgmt_botservice-0.2.0-py2.py3-none-any.whl (109 kB)
Collecting azure-mgmt-sql~=0.18.0
  Downloading azure_mgmt_sql-0.18.0-py2.py3-none-any.whl (375 kB)
Collecting azure-mgmt-managementgroups~=0.1
  Downloading azure_mgmt_managementgroups-0.2.0-py2.py3-none-any.whl (59 kB)
Collecting mock~=4.0
  Downloading mock-4.0.3-py3-none-any.whl (28 kB)
Collecting azure-mgmt-web~=0.44.0
  Downloading azure_mgmt_web-0.44.0-py2.py3-none-any.whl (1.0 MB)
Collecting urllib3[secure]~=1.18
  Downloading urllib3-1.26.8-py2.py3-none-any.whl (138 kB)
Collecting azure-mgmt-servicefabric~=0.4.0
  Downloading azure_mgmt_servicefabric-0.4.0-py2.py3-none-any.whl (62 kB)
Collecting azure-mgmt-trafficmanager~=0.51.0
  Downloading azure_mgmt_trafficmanager-0.51.0-py2.py3-none-any.whl (58 kB)
Collecting azure-mgmt-datalake-store~=0.5.0
  Downloading azure_mgmt_datalake_store-0.5.0-py2.py3-none-any.whl (88 kB)
Collecting azure-mgmt-iotcentral~=3.0.0
  Downloading azure_mgmt_iotcentral-3.0.0-py2.py3-none-any.whl (18 kB)
Collecting azure-mgmt-cdn==4.1.0rc1
  Downloading azure_mgmt_cdn-4.1.0rc1-py2.py3-none-any.whl (73 kB)
Collecting azure-mgmt-signalr~=0.3.0
  Downloading azure_mgmt_signalr-0.3.0-py2.py3-none-any.whl (53 kB)
Collecting jsondiff==1.2.0
  Downloading jsondiff-1.2.0.tar.gz (8.0 kB)
  Preparing metadata (setup.py) ... done
Collecting azure-mgmt-consumption~=2.0
  Downloading azure_mgmt_consumption-2.0.0-py2.py3-none-any.whl (46 kB)
Collecting azure-mgmt-deploymentmanager~=0.2.0
  Downloading azure_mgmt_deploymentmanager-0.2.0-py2.py3-none-any.whl (41 kB)
Collecting azure-mgmt-redis~=7.0.0rc1
  Downloading azure_mgmt_redis-7.0.0-py2.py3-none-any.whl (44 kB)
Collecting azure-mgmt-managedservices~=1.0
  Downloading azure_mgmt_managedservices-1.0.0-py2.py3-none-any.whl (38 kB)
Collecting azure-mgmt-policyinsights~=0.4.0
  Downloading azure_mgmt_policyinsights-0.4.0-py2.py3-none-any.whl (41 kB)
Collecting sshtunnel~=0.1.4
  Downloading sshtunnel-0.1.5-py2.py3-none-any.whl (23 kB)
Collecting azure-mgmt-appconfiguration~=0.4.0
  Downloading azure_mgmt_appconfiguration-0.4.0-py2.py3-none-any.whl (28 kB)
Collecting azure-mgmt-keyvault~=2.2.0
  Downloading azure_mgmt_keyvault-2.2.0-py2.py3-none-any.whl (89 kB)
Collecting vsts-cd-manager>=1.0.2,~=1.0.0
  Downloading vsts-cd-manager-1.0.2.tar.gz (13 kB)
  Preparing metadata (setup.py) ... done
Collecting azure-mgmt-cognitiveservices~=5.0.0
  Downloading azure_mgmt_cognitiveservices-5.0.0-py2.py3-none-any.whl (31 kB)
Collecting azure-mgmt-imagebuilder~=0.2.1
  Downloading azure_mgmt_imagebuilder-0.2.1-py2.py3-none-any.whl (66 kB)
Collecting azure-cosmos>=3.0.2,~=3.0
  Downloading azure_cosmos-3.2.0-py2.py3-none-any.whl (106 kB)
Collecting pyOpenSSL>=17.1.0
  Downloading pyOpenSSL-21.0.0-py2.py3-none-any.whl (55 kB)
Collecting azure-mgmt-storage~=9.0.0
  Downloading azure_mgmt_storage-9.0.0-py2.py3-none-any.whl (525 kB)
Collecting azure-mgmt-batchai~=2.0
  Downloading azure_mgmt_batchai-2.0.0-py2.py3-none-any.whl (174 kB)
Collecting azure-mgmt-iothub~=0.11.0
  Downloading azure_mgmt_iothub-0.11.0-py2.py3-none-any.whl (401 kB)
Collecting azure-mgmt-dns~=2.1
  Downloading azure_mgmt_dns-2.1.0-py2.py3-none-any.whl (134 kB)
Collecting azure-mgmt-msi~=0.2
  Downloading azure_mgmt_msi-0.2.0-py2.py3-none-any.whl (17 kB)
Collecting azure-mgmt-billing~=0.2
  Downloading azure_mgmt_billing-0.2.0-py2.py3-none-any.whl (25 kB)
Collecting javaproperties==0.5.1
  Downloading javaproperties-0.5.1-py2.py3-none-any.whl (19 kB)
Collecting azure-multiapi-storage~=0.3.1
  Downloading azure_multiapi_storage-0.3.7-py2.py3-none-any.whl (2.4 MB)
Collecting azure-mgmt-batch~=7.0
  Downloading azure_mgmt_batch-7.0.0-py2.py3-none-any.whl (81 kB)
Collecting azure-mgmt-recoveryservices~=0.4.0
  Downloading azure_mgmt_recoveryservices-0.4.0-py2.py3-none-any.whl (77 kB)
Collecting azure-cli-command-modules-nspkg~=2.0
  Downloading azure_cli_command_modules_nspkg-2.0.3-py3-none-any.whl (1.9 kB)
Collecting azure-mgmt-servicebus~=0.6.0
  Downloading azure_mgmt_servicebus-0.6.0-py2.py3-none-any.whl (120 kB)
Collecting azure-loganalytics~=0.1.0
  Downloading azure_loganalytics-0.1.1-py2.py3-none-any.whl (16 kB)
Collecting cryptography<3.0.0,>=2.3.1
  Downloading cryptography-2.9.2-cp35-abi3-macosx_10_9_x86_64.whl (1.8 MB)
Collecting azure-mgmt-kusto~=0.3.0
  Downloading azure_mgmt_kusto-0.3.0-py2.py3-none-any.whl (73 kB)
Collecting azure-cli-nspkg>=2.0.0
  Downloading azure_cli_nspkg-3.0.4-py3-none-any.whl (1.8 kB)
Collecting azure-mgmt-sqlvirtualmachine~=0.5.0
  Downloading azure_mgmt_sqlvirtualmachine-0.5.0-py2.py3-none-any.whl (34 kB)
Collecting azure-storage-blob<2.0.0,>=1.3.1
  Downloading azure_storage_blob-1.5.0-py2.py3-none-any.whl (75 kB)
Collecting azure-mgmt-loganalytics~=0.5.0
  Downloading azure_mgmt_loganalytics-0.5.0-py2.py3-none-any.whl (71 kB)
Collecting azure-mgmt-datalake-analytics~=0.2.1
  Downloading azure_mgmt_datalake_analytics-0.2.1-py2.py3-none-any.whl (146 kB)
Collecting azure-functions-devops-build~=0.0.22
  Downloading azure_functions_devops_build-0.0.22-py3-none-any.whl (47 kB)
Collecting azure-mgmt-resource==9.0.0
  Downloading azure_mgmt_resource-9.0.0-py2.py3-none-any.whl (807 kB)
Collecting antlr4-python3-runtime~=4.7.2
  Downloading antlr4-python3-runtime-4.7.2.tar.gz (112 kB)
  Preparing metadata (setup.py) ... done
Collecting websocket-client~=0.56.0
  Downloading websocket_client-0.56.0-py2.py3-none-any.whl (200 kB)
Collecting azure-mgmt-redhatopenshift==0.1.0
  Downloading azure_mgmt_redhatopenshift-0.1.0-py2.py3-none-any.whl (23 kB)
Collecting azure-mgmt-eventgrid~=2.2
  Downloading azure_mgmt_eventgrid-2.2.0-py2.py3-none-any.whl (109 kB)
Collecting azure-mgmt-iothubprovisioningservices~=0.2.0
  Downloading azure_mgmt_iothubprovisioningservices-0.2.0-py2.py3-none-any.whl (60 kB)
Collecting azure-mgmt-rdbms~=2.2.0
  Downloading azure_mgmt_rdbms-2.2.0-py2.py3-none-any.whl (231 kB)
Collecting azure-cli-core==2.5.1.*
  Downloading azure_cli_core-2.5.1-py3-none-any.whl (140 kB)
Collecting azure-mgmt-privatedns~=0.1.0
  Downloading azure_mgmt_privatedns-0.1.0-py2.py3-none-any.whl (44 kB)
Collecting azure-mgmt-netapp~=0.8.0
  Downloading azure_mgmt_netapp-0.8.0-py2.py3-none-any.whl (40 kB)
Collecting azure-mgmt-apimanagement~=0.1.0
  Downloading azure_mgmt_apimanagement-0.1.0-py2.py3-none-any.whl (542 kB)
Collecting azure-mgmt-network~=10.1.0
  Downloading azure_mgmt_network-10.1.0-py2.py3-none-any.whl (8.1 MB)
Collecting pytz==2019.1
  Downloading pytz-2019.1-py2.py3-none-any.whl (510 kB)
Collecting jsmin~=2.2.2
  Downloading jsmin-2.2.2.tar.gz (12 kB)
  Preparing metadata (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /Users/user/src/Adaz/ansible/venv/bin/python3.9 -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/yj/m9_s7pl516jgd8wvwb0snqk40000gn/T/pip-install-e7uhszi_/jsmin_18eb338660e6417dba5daa1ee53be54a/setup.py'"'"'; __file__='"'"'/private/var/folders/yj/m9_s7pl516jgd8wvwb0snqk40000gn/T/pip-install-e7uhszi_/jsmin_18eb338660e6417dba5daa1ee53be54a/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/yj/m9_s7pl516jgd8wvwb0snqk40000gn/T/pip-pip-egg-info-mfhzutpd
       cwd: /private/var/folders/yj/m9_s7pl516jgd8wvwb0snqk40000gn/T/pip-install-e7uhszi_/jsmin_18eb338660e6417dba5daa1ee53be54a/
  Complete output (1 lines):
  error in jsmin setup command: use_2to3 is invalid.
  ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/17/73/615d1267a82ed26cd7c124108c3c61169d8e40c36d393883eaee3a561852/jsmin-2.2.2.tar.gz#sha256=b6df99b2cd1c75d9d342e4335b535789b8da9107ec748212706ef7bbe5c2553b (from https://pypi.org/simple/jsmin/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
INFO: pip is looking at multiple versions of msrestazure to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of msrest to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of  to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of requests to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of pywinrm to determine which version is compatible with other requirements. This could take a while.
ERROR: Could not find a version that satisfies the requirement jsmin~=2.2.2 (from azure-cli) (from versions: 2.0, 2.0.1, 2.0.2, 2.0.2.post1, 2.0.3, 2.0.5, 2.0.6, 2.0.7, 2.0.8, 2.0.9, 2.0.10, 2.0.11, 2.1.0, 2.1.1, 2.1.2, 2.1.3, 2.1.4, 2.1.5, 2.1.6, 2.2.0, 2.2.1, 2.2.2, 3.0.0)
ERROR: No matching distribution found for jsmin~=2.2.2
~/src/Adaz (master) $ pip list
Package    Version
---------- -------
pip        21.3.1
setuptools 59.0.1
~/src/Adaz (master) $ python3 --version
Python 3.9.9

Pre-install Invoke-AtomicRedTeam on workstations

https://github.com/redcanaryco/invoke-atomicredteam

Couldn't get it working, help would be appreciated

What is not working?

I spent hours already but I can't get it working.

On my Win10 local machine:
I created an Azure subscription - successfully
Downloaded and installed Azure CLI for Windows (MSI) - successfully
Opened cmd.exe and then executed az login to connect to my account - successfully
Created an (OpenSSH) SSH key using cmd.exe > ssh-keygen - successfully
opened cmd.exe then executed winget install git.git - successfully
opened Git CMD then executed git clone https://github.com/christophetd/Adaz.git - successfully

I opened a command prompt, navigated to C:\Users\VegWorld\Adaz and then executed python3 -m venv ansible/venv
that didn't seem to work, so I navigated to \Adaz\ansible and then executed py -m venv env
A folder named env was created inside the folder ansible. I manually renamed it to venv
executed python3 source ansible/venv/bin/activate > no output/error was shown...
Then, I executed pip install -r ansible/requirements.txt which took a lot of time, and it finished with errors.
What went wrong? help would be greatly appreciated

What OS are you using?
Windows 10

Your domain.yml file?
If you customized the domain.yml file, please include it below

Full Terraform / Ansible output?

Search "error" (6 hits in 1 file) (Notepad++)
(6 hits)
Line 447: ERROR: msal 1.0.0 has requirement PyJWT[crypto]<2,>=1.0.0, but you'll have pyjwt 2.3.0 which is incompatible.
Line 448: ERROR: pyopenssl 21.0.0 has requirement cryptography>=3.3, but you'll have cryptography 2.9.2 which is incompatible.
Line 464: Running setup.py install for ansible ... error
Line 465: ERROR: Command errored out with exit status 1:
Line 6828: error: symbolic link privilege not held
Line 6830: ERROR: Command errored out with exit status 1: 'c:\users\vegworld\appdata\local\programs\python\python37-32\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\vegworld\AppData\Local\Temp\pip-install-q7an93pr\ansible\setup.py'"'"'; file='"'"'C:\Users\vegworld\AppData\Local\Temp\pip-install-q7an93pr\ansible\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record 'C:\Users\vegworld\AppData\Local\Temp\pip-record-5gxbcum5\install-record.txt' --single-version-externally-managed --compile --install-headers 'c:\users\vegworld\appdata\local\programs\python\python37-32\Include\ansible' Check the logs for full command output.

WARNING: You are using pip version 20.0.2; however, version 21.3.1 is available.
You should consider upgrading via the 'c:\users\vegworld\appdata\local\programs\python\python37-32\python.exe -m pip install --upgrade pip' command.

(I can provide the full text file if needed, too big for GitHub)

Terraform Destroy fails

I keep running into errors when trying to destroy the lab:

$ terraform destroy -force
Error: Error in function call

  on outputs.tf line 29, in output "workstations_public_ips":
  29:   value = zipmap(azurerm_virtual_machine.workstation.*.name, azurerm_public_ip.workstation.*.ip_address)
    |----------------
    | azurerm_public_ip.workstation is tuple with 2 elements
    | azurerm_virtual_machine.workstation is empty tuple

Call to function "zipmap" failed: number of keys (0) does not match number of
values (2).

$ terraform destroy -force
data.http.public_ip: Refreshing state...
azurerm_resource_group.main: Refreshing state... [id=/subscriptions/c0877b41-b058-4f58-9dc0-9c8174929611/resourceGroups/ad-hunting-lab]
azurerm_public_ip.workstation[1]: Refreshing state... [id=/subscriptions/c0877b41-b058-4f58-9dc0-9c8174929611/resourceGroups/ad-hunting-lab/providers/Microsoft.Network/publicIPAddresses/ad-lab-wks-1-ingress]
azurerm_public_ip.workstation[0]: Refreshing state... [id=/subscriptions/c0877b41-b058-4f58-9dc0-9c8174929611/resourceGroups/ad-hunting-lab/providers/Microsoft.Network/publicIPAddresses/ad-lab-wks-0-ingress]
data.azurerm_public_ip.main: Refreshing state...
data.azurerm_public_ip.elasticsearch: Refreshing state...
data.azurerm_public_ip.workstation[1]: Refreshing state...
data.azurerm_public_ip.workstation[0]: Refreshing state...

Error: Error: Public IP "ad-lab-ingress" (Resource Group "ad-hunting-lab") was not found

  on data.tf line 1, in data "azurerm_public_ip" "main":
   1: data "azurerm_public_ip" "main" {



Error: Error: Public IP "elasticsearch-ingress" (Resource Group "ad-hunting-lab") was not found

  on data.tf line 13, in data "azurerm_public_ip" "elasticsearch":
  13: data "azurerm_public_ip" "elasticsearch" {

Adaz not working with newly released Terraform version 0.13

What is not working?
When running terraform plan or terraform apply, I receive the following errors:

`
$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

data.http.public_ip: Refreshing state...

---

Error: Error: Public IP "ad-lab-ingress" (Resource Group "ad-hunting-lab") was not found

on data.tf line 1, in data "azurerm_public_ip" "main":
1: data "azurerm_public_ip" "main" {

Error: Error: Public IP "ad-lab-wks-0-ingress" (Resource Group "ad-hunting-lab") was not found

on data.tf line 6, in data "azurerm_public_ip" "workstation":
6: data "azurerm_public_ip" "workstation" {

Error: Error: Public IP "ad-lab-wks-1-ingress" (Resource Group "ad-hunting-lab") was not found

on data.tf line 6, in data "azurerm_public_ip" "workstation":
6: data "azurerm_public_ip" "workstation" {

Error: Error: Public IP "elasticsearch-ingress" (Resource Group "ad-hunting-lab") was not found

on data.tf line 13, in data "azurerm_public_ip" "elasticsearch":
13: data "azurerm_public_ip" "elasticsearch" {
`

What OS are you using?
I have confirmed this on WSLv2 running Ubuntu 20.04 and an Ubuntu 19.10 VPS.

Your domain.yml file?
Have not changed from what's in the repo.

Full Terraform / Ansible output?
As above.

Enable powershell module logging

Kibana setup failed

The Kibana setup fails. So why does apt then fail with a GPG error?

Logfile

Error: Error running command '/bin/bash -c 'source venv/bin/activate && ANSIBLE_HOST_KEY_CHECKING=false ansible-playbook elasticsearch-kibana.yml -v'': exit status 2. Output: Using /home/user/Adaz/ansible/ansible.cfg as config file

PLAY [Configure Elasticsearch and Kibana] **************************************

TASK [elasticsearch-kibana : wait_for_connection] ******************************
ok: [es-kibana_5589] => {"changed": false, "elapsed": 12}

TASK [elasticsearch-kibana : include_tasks] ************************************
included: /home/user/Adaz/ansible/roles/elasticsearch-kibana/tasks/elasticsearch.yml for es-kibana_5589

TASK [elasticsearch-kibana : Ensure Elasticsearch package is installed] ********
[WARNING]: Updating cache and auto-installing missing dependency: python-apt
[DEPRECATION WARNING]: Distribution Ubuntu 18.04 on host es-kibana_5589 should
use /usr/bin/python3, but is using /usr/bin/python for backward compatibility
with prior Ansible releases. A future Ansible release will default to using the
 discovered platform python for this host. See https://docs.ansible.com/ansible
/2.9/reference_appendices/interpreter_discovery.html for more information. This
 feature will be removed in version 2.12. Deprecation warnings can be disabled
by setting deprecation_warnings=False in ansible.cfg.
fatal: [es-kibana_5589]: FAILED! => {"ansible_facts": {"discovered_interpreter_python": "/usr/bin/python"}, "changed": false, "cmd": "apt-get update", "msg": "W: GPG error: http://archive.ubuntu.com/ubuntu bionic InRelease: Splitting up /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_bionic_InRelease into data and signature failed\nE: The repository 'http://archive.ubuntu.com/ubuntu bionic InRelease' is not signed.", "rc": 100, "stderr": "W: GPG error: http://archive.ubuntu.com/ubuntu bionic InRelease: Splitting up /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_bionic_InRelease into data and signature failed\nE: The repository 'http://archive.ubuntu.com/ubuntu bionic InRelease' is not signed.\n", "stderr_lines": ["W: GPG error: http://archive.ubuntu.com/ubuntu bionic InRelease: Splitting up /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_bionic_InRelease into data and signature failed", "E: The repository 'http://archive.ubuntu.com/ubuntu bionic InRelease' is not signed."], "stdout": "Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]\nHit:2 http://archive.ubuntu.com/ubuntu bionic InRelease\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]\nErr:2 http://archive.ubuntu.com/ubuntu bionic InRelease\n  Splitting up /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_bionic_InRelease into data and signature failed\nGet:5 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1399 kB]\nGet:6 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [276 kB]\nGet:7 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [166 kB]\nGet:8 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [22.1 kB]\nGet:9 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1079 kB]\nGet:10 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [241 kB]\nGet:11 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [12.9 kB]\nGet:12 http://security.ubuntu.com/ubuntu bionic-security/multiverse Translation-en [2964 B]\nGet:13 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1726 kB]\nGet:14 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [367 kB]\nGet:15 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [184 kB]\nGet:16 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [24.6 kB]\nGet:17 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1683 kB]\nGet:18 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [354 kB]\nGet:19 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [31.9 kB]\nGet:20 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [6980 B]\nGet:21 http://archive.ubuntu.com/ubuntu bionic-backports/main amd64 Packages [10.0 kB]\nGet:22 http://archive.ubuntu.com/ubuntu bionic-backports/main Translation-en [4764 B]\nGet:23 http://archive.ubuntu.com/ubuntu bionic-backports/universe amd64 Packages [10.3 kB]\nGet:24 http://archive.ubuntu.com/ubuntu bionic-backports/universe Translation-en [4588 B]\nReading package lists...\n", "stdout_lines": ["Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]", "Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]", "Err:2 http://archive.ubuntu.com/ubuntu bionic InRelease", "  Splitting up /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_bionic_InRelease into data and signature failed", "Get:5 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1399 kB]", "Get:6 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [276 kB]", "Get:7 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [166 kB]", "Get:8 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [22.1 kB]", "Get:9 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1079 kB]", "Get:10 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [241 kB]", "Get:11 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [12.9 kB]", "Get:12 http://security.ubuntu.com/ubuntu bionic-security/multiverse Translation-en [2964 B]", "Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1726 kB]", "Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [367 kB]", "Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [184 kB]", "Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [24.6 kB]", "Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1683 kB]", "Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [354 kB]", "Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [31.9 kB]", "Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [6980 B]", "Get:21 http://archive.ubuntu.com/ubuntu bionic-backports/main amd64 Packages [10.0 kB]", "Get:22 http://archive.ubuntu.com/ubuntu bionic-backports/main Translation-en [4764 B]", "Get:23 http://archive.ubuntu.com/ubuntu bionic-backports/universe amd64 Packages [10.3 kB]", "Get:24 http://archive.ubuntu.com/ubuntu bionic-backports/universe Translation-en [4588 B]", "Reading package lists..."]}

PLAY RECAP *********************************************************************
es-kibana_5589             : ok=2    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

Look at this line

fatal: [es-kibana_5589]: FAILED! 

Maybe related to #15

Change RAM and create Linux machines

1. Is there a way to change the size of VMs?
2. is there a way to add few linux machines to the lab?