azure/terraform-azurerm-caf-enterprise-scale

Azure landing zones Terraform module

Azure landing zones Terraform module

Detailed information about how to use, configure and extend this module can be found on our Wiki:

• Home
• User Guide
• Examples
• Frequently Asked Questions
• Troubleshooting

Overview

The Azure landing zones Terraform module is designed to accelerate deployment of platform resources based on the Azure landing zones conceptual architecture using Terraform.

This is currently split logically into the following capabilities within the module (links to further guidance on the Wiki):

Module capability	Scope	Design area
Core Resources	Management group and subscription organization	Resource organization
Management Resources	Management subscription	Management
Connectivity Resources	Connectivity subscription	Network topology and connectivity
Identity Resources	Identity subscription	Identity and access management

Using a very simple initial configuration, the module will deploy a management group hierarchy based on the above diagram. This includes the recommended governance baseline, applied using Azure Policy and Access control (IAM) resources deployed at the management group scope. The default configuration can be easily extended to meet differing requirements, and includes the ability to deploy platform resources in the management and connectivity subscriptions.

NOTE: In addition to setting input variables to control which resources are deployed, the module requires setting a Provider Configuration block to enable deployment across multiple subscriptions.

Although resources are logically grouped to simplify operations, the modular design of the module also allows resources to be deployed using different Terraform workspaces. This allows customers to address concerns around managing large state files, or assigning granular permissions to pipelines based on the principle of least privilege. (more information coming soon in the Wiki)

Terraform versions

This module has been tested using Terraform 1.3.1 and AzureRM Provider 3.74.0 as a baseline, and various versions to up the latest at time of release. In some cases, individual versions of the AzureRM provider may cause errors. If this happens, we advise upgrading to the latest version and checking our troubleshooting guide before raising an issue.

NOTE: The module now requires a minimum Terraform version of 1.3.1 to support the GA release of optional() Object Type Attributes and the required fix for issue #31844.

Usage

We recommend starting with the following configuration in your root module to learn what resources are created by the module and how it works.

This will deploy the core components only.

NOTE: For production use we highly recommend using the Terraform Registry and pinning to the latest stable version, as per the example below. Pinning to the main branch in GitHub will give you the latest updates quicker, but increases the likelihood of unplanned changes to your environment and unforeseen issues.

main.tf

# Configure Terraform to set the required AzureRM provider
# version and features{} block.

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = ">= 3.74.0"
    }
  }
}

provider "azurerm" {
  features {}
}

# Get the current client configuration from the AzureRM provider.
# This is used to populate the root_parent_id variable with the
# current Tenant ID used as the ID for the "Tenant Root Group"
# management group.

data "azurerm_client_config" "core" {}

# Use variables to customize the deployment

variable "root_id" {
  type    = string
  default = "es"
}

variable "root_name" {
  type    = string
  default = "Enterprise-Scale"
}

variable "default_location" {
  type    = string
}

# Declare the Azure landing zones Terraform module
# and provide a base configuration.

module "enterprise_scale" {
  source  = "Azure/caf-enterprise-scale/azurerm"
  version = "<version>" # change this to your desired version, https://www.terraform.io/language/expressions/version-constraints

  default_location = var.default_location

  providers = {
    azurerm              = azurerm
    azurerm.connectivity = azurerm
    azurerm.management   = azurerm
  }

  root_parent_id = data.azurerm_client_config.core.tenant_id
  root_id        = var.root_id
  root_name      = var.root_name

}

NOTE: For additional guidance on how to customize your deployment using the advanced configuration options for this module, please refer to our User Guide and the additional examples in our documentation.

Permissions

Please refer to our Module Permissions guide on the Wiki.

Examples

The following list outlines some of our most popular examples:

• Examples - Level 100
  • Deploy Default Configuration
  • Deploy Demo Landing Zone Archetypes
• Examples - Level 200
  • Deploy Custom Landing Zone Archetypes
  • Deploy Connectivity Resources
  • Deploy Identity Resources
  • Deploy Management Resources
  • Assign a Built-in Policy
• Examples - Level 300
  • Deploy Connectivity Resources With Custom Settings
  • Deploy Identity Resources With Custom Settings
  • Deploy Management Resources With Custom Settings
  • Expand Built-in Archetype Definitions
  • Create Custom Policies, Policy Sets and Assignments

For the complete list of our latest examples, please refer to our Examples page on the Wiki.

Release notes

Please see the releases page for the latest module updates.

Upgrade guides

For upgrade guides from previous versions, please refer to the following links:

• Upgrade from v3.3.0 to v4.0.0
• Upgrade from v2.4.1 to v3.0.0
• Upgrade from v1.1.4 to v2.0.0
• Upgrade from v0.4.0 to v1.0.0
• Upgrade from v0.3.3 to v0.4.0
• Upgrade from v0.1.2 to v0.2.0
• Upgrade from v0.0.8 to v0.1.0

Documentation

Requirements

The following requirements are needed by this module:

• terraform (>= 1.3.1)

• azapi (>= 1.7.0)

• azurerm (>= 3.74.0)

• random (>= 3.1.0)

• time (>= 0.7.0)

Modules

The following Modules are called:

connectivity_resources

Source: ./modules/connectivity

Version:

identity_resources

Source: ./modules/identity

Version:

management_group_archetypes

Source: ./modules/archetypes

Version:

management_resources

Source: ./modules/management

Version:

role_assignments_for_policy

Source: ./modules/role_assignments_for_policy

Version:

Required Inputs

The following input variables are required:

default_location

Description: Must be specified, e.g eastus. Will set the Azure region in which region bound resources will be deployed. Please see: https://azure.microsoft.com/en-gb/global-infrastructure/geographies/

Type: string

root_parent_id

Description: The root_parent_id is used to specify where to set the root for all Landing Zone deployments. Usually the Tenant ID when deploying the core Enterprise-scale Landing Zones.

Type: string

Optional Inputs

The following input variables are optional (have default values):

archetype_config_overrides

Description: If specified, will set custom Archetype configurations for the core ALZ Management Groups.
Does not work for management groups specified by the 'custom_landing_zones' input variable.
To override the default configuration settings for any of the core Management Groups, add an entry to the archetype_config_overrides variable for each Management Group you want to customize.
To create a valid archetype_config_overrides entry, you must provide the required values in the archetype_config_overrides object for the Management Group you wish to re-configure.
To do this, simply create an entry similar to the root example below for one or more of the supported core Management Group IDs:

• root
• decommissioned
• sandboxes
• landing-zones
• platform
• connectivity
• management
• identity
• corp
• online
• sap

map(
  object({
    archetype_id     = string
    enforcement_mode = map(bool)
    parameters       = map(any)
    access_control   = map(list(string))
  })
)

e.g.

  archetype_config_overrides = {
    root = {
      archetype_id = "root"
      enforcement_mode = {
        "Example-Policy-Assignment" = true,
      }
      parameters = {
        Example-Policy-Assignment = {
          parameterStringExample = "value1",
          parameterBoolExample   = true,
          parameterNumberExample = 10,
          parameterListExample = [
            "listItem1",
            "listItem2",
          ]
          parameterObjectExample = {
            key1 = "value1",
            key2 = "value2",
          }
        }
      }
      access_control = {
        Example-Role-Definition = [
          "00000000-0000-0000-0000-000000000000", # Object ID of user/group/spn/mi from Microsoft Entra ID
          "11111111-1111-1111-1111-111111111111", # Object ID of user/group/spn/mi from Microsoft Entra ID
        ]
      }
    }
  }

Type: any

Default: {}

configure_connectivity_resources

Description: If specified, will customize the "Connectivity" landing zone settings and resources.

Notes for the configure_connectivity_resources variable:

• settings.hub_network_virtual_network_gateway.config.address_prefix
  • Only support adding a single address prefix for GatewaySubnet subnet
• settings.hub_network_virtual_network_gateway.config.gateway_sku_expressroute
  • If specified, will deploy the ExpressRoute gateway into the GatewaySubnet subnet
• settings.hub_network_virtual_network_gateway.config.gateway_sku_vpn
  • If specified, will deploy the VPN gateway into the GatewaySubnet subnet
• settings.hub_network_virtual_network_gateway.config.advanced_vpn_settings.private_ip_address_allocation
  • Valid options are "", "Static" or "Dynamic". Will set private_ip_address_enabled and private_ip_address_allocation as needed.
• settings.azure_firewall.config.address_prefix
  • Only support adding a single address prefix for AzureFirewallManagementSubnet subnet

Type:

object({
    settings = optional(object({
      hub_networks = optional(list(
        object({
          enabled = optional(bool, true)
          config = object({
            address_space                = list(string)
            location                     = optional(string, "")
            link_to_ddos_protection_plan = optional(bool, false)
            dns_servers                  = optional(list(string), [])
            bgp_community                = optional(string, "")
            subnets = optional(list(
              object({
                name                      = string
                address_prefixes          = list(string)
                network_security_group_id = optional(string, "")
                route_table_id            = optional(string, "")
              })
            ), [])
            virtual_network_gateway = optional(object({
              enabled = optional(bool, false)
              config = optional(object({
                address_prefix           = optional(string, "")
                gateway_sku_expressroute = optional(string, "")
                gateway_sku_vpn          = optional(string, "")
                advanced_vpn_settings = optional(object({
                  enable_bgp                       = optional(bool, null)
                  active_active                    = optional(bool, null)
                  private_ip_address_allocation    = optional(string, "")
                  default_local_network_gateway_id = optional(string, "")
                  vpn_client_configuration = optional(list(
                    object({
                      address_space = list(string)
                      aad_tenant    = optional(string, null)
                      aad_audience  = optional(string, null)
                      aad_issuer    = optional(string, null)
                      root_certificate = optional(list(
                        object({
                          name             = string
                          public_cert_data = string
                        })
                      ), [])
                      revoked_certificate = optional(list(
                        object({
                          name       = string
                          thumbprint = string
                        })
                      ), [])
                      radius_server_address = optional(string, null)
                      radius_server_secret  = optional(string, null)
                      vpn_client_protocols  = optional(list(string), null)
                      vpn_auth_types        = optional(list(string), null)
                    })
                  ), [])
                  bgp_settings = optional(list(
                    object({
                      asn         = optional(number, null)
                      peer_weight = optional(number, null)
                      peering_addresses = optional(list(
                        object({
                          ip_configuration_name = optional(string, null)
                          apipa_addresses       = optional(list(string), null)
                        })
                      ), [])
                    })
                  ), [])
                  custom_route = optional(list(
                    object({
                      address_prefixes = optional(list(string), [])
                    })
                  ), [])
                }), {})
              }), {})
            }), {})
            azure_firewall = optional(object({
              enabled = optional(bool, false)
              config = optional(object({
                address_prefix                = optional(string, "")
                address_management_prefix     = optional(string, "")
                enable_dns_proxy              = optional(bool, true)
                dns_servers                   = optional(list(string), [])
                sku_tier                      = optional(string, "Standard")
                base_policy_id                = optional(string, "")
                private_ip_ranges             = optional(list(string), [])
                threat_intelligence_mode      = optional(string, "Alert")
                threat_intelligence_allowlist = optional(map(list(string)), {})
                availability_zones = optional(object({
                  zone_1 = optional(bool, true)
                  zone_2 = optional(bool, true)
                  zone_3 = optional(bool, true)
                }), {})
              }), {})
            }), {})
            spoke_virtual_network_resource_ids      = optional(list(string), [])
            enable_outbound_virtual_network_peering = optional(bool, false)
            enable_hub_network_mesh_peering         = optional(bool, false)
          })
        })
      ), [])
      vwan_hub_networks = optional(list(
        object({
          enabled = optional(bool, true)
          config = object({
            address_prefix = string
            location       = string
            sku            = optional(string, "")
            routes = optional(list(
              object({
                address_prefixes    = list(string)
                next_hop_ip_address = string
              })
            ), [])
            routing_intent = optional(object({
              enabled = optional(bool, false)
              config = optional(object({
                routing_policies = optional(list(object({
                  name         = string
                  destinations = list(string)
                })), [])
              }), {})
            }), {})
            expressroute_gateway = optional(object({
              enabled = optional(bool, false)
              config = optional(object({
                scale_unit = optional(number, 1)
                allow_non_virtual_wan_traffic = optional(bool, false)
              }), {})
            }), {})
            vpn_gateway = optional(object({
              enabled = optional(bool, false)
              config = optional(object({
                bgp_settings = optional(list(
                  object({
                    asn         = number
                    peer_weight = number
                    instance_0_bgp_peering_address = optional(list(
                      object({
                        custom_ips = list(string)
                      })
                    ), [])
                    instance_1_bgp_peering_address = optional(list(
                      object({
                        custom_ips = list(string)
                      })
                    ), [])
                  })
                ), [])
                routing_preference = optional(string, "Microsoft Network")
                scale_unit         = optional(number, 1)
              }), {})
            }), {})
            azure_firewall = optional(object({
              enabled = optional(bool, false)
              config = optional(object({
                enable_dns_proxy              = optional(bool, true)
                dns_servers                   = optional(list(string), [])
                sku_tier                      = optional(string, "Standard")
                base_policy_id                = optional(string, "")
                private_ip_ranges             = optional(list(string), [])
                threat_intelligence_mode      = optional(string, "Alert")
                threat_intelligence_allowlist = optional(map(list(string)), {})
                availability_zones = optional(object({
                  zone_1 = optional(bool, true)
                  zone_2 = optional(bool, true)
                  zone_3 = optional(bool, true)
                }), {})
              }), {})
            }), {})
            spoke_virtual_network_resource_ids        = optional(list(string), [])
            secure_spoke_virtual_network_resource_ids = optional(list(string), [])
            enable_virtual_hub_connections            = optional(bool, false)
          })
        })
      ), [])
      ddos_protection_plan = optional(object({
        enabled = optional(bool, false)
        config = optional(object({
          location = optional(string, "")
        }), {})
      }), {})
      dns = optional(object({
        enabled = optional(bool, true)
        config = optional(object({
          location = optional(string, "")
          enable_private_link_by_service = optional(object({
            azure_api_management                 = optional(bool, true)
            azure_app_configuration_stores       = optional(bool, true)
            azure_arc                            = optional(bool, true)
            azure_automation_dscandhybridworker  = optional(bool, true)
            azure_automation_webhook             = optional(bool, true)
            azure_backup                         = optional(bool, true)
            azure_batch_account                  = optional(bool, true)
            azure_bot_service_bot                = optional(bool, true)
            azure_bot_service_token              = optional(bool, true)
            azure_cache_for_redis                = optional(bool, true)
            azure_cache_for_redis_enterprise     = optional(bool, true)
            azure_container_registry             = optional(bool, true)
            azure_cosmos_db_cassandra            = optional(bool, true)
            azure_cosmos_db_gremlin              = optional(bool, true)
            azure_cosmos_db_mongodb              = optional(bool, true)
            azure_cosmos_db_sql                  = optional(bool, true)
            azure_cosmos_db_table                = optional(bool, true)
            azure_data_explorer                  = optional(bool, true)
            azure_data_factory                   = optional(bool, true)
            azure_data_factory_portal            = optional(bool, true)
            azure_data_health_data_services      = optional(bool, true)
            azure_data_lake_file_system_gen2     = optional(bool, true)
            azure_database_for_mariadb_server    = optional(bool, true)
            azure_database_for_mysql_server      = optional(bool, true)
            azure_database_for_postgresql_server = optional(bool, true)
            azure_digital_twins                  = optional(bool, true)
            azure_event_grid_domain              = optional(bool, true)
            azure_event_grid_topic               = optional(bool, true)
            azure_event_hubs_namespace           = optional(bool, true)
            azure_file_sync                      = optional(bool, true)
            azure_hdinsights                     = optional(bool, true)
            azure_iot_dps                        = optional(bool, true)
            azure_iot_hub                        = optional(bool, true)
            azure_key_vault                      = optional(bool, true)
            azure_key_vault_managed_hsm          = optional(bool, true)
            azure_kubernetes_service_management  = optional(bool, true)
            azure_machine_learning_workspace     = optional(bool, true)
            azure_managed_disks                  = optional(bool, true)
            azure_media_services                 = optional(bool, true)
            azure_migrate                        = optional(bool, true)
            azure_monitor                        = optional(bool, true)
            azure_purview_account                = optional(bool, true)
            azure_purview_studio                 = optional(bool, true)
            azure_relay_namespace                = optional(bool, true)
            azure_search_service                 = optional(bool, true)
            azure_service_bus_namespace          = optional(bool, true)
            azure_site_recovery                  = optional(bool, true)
            azure_sql_database_sqlserver         = optional(bool, true)
            azure_synapse_analytics_dev          = optional(bool, true)
            azure_synapse_analytics_sql          = optional(bool, true)
            azure_synapse_studio                 = optional(bool, true)
            azure_web_apps_sites                 = optional(bool, true)
            azure_web_apps_static_sites          = optional(bool, true)
            cognitive_services_account           = optional(bool, true)
            microsoft_power_bi                   = optional(bool, true)
            signalr                              = optional(bool, true)
            signalr_webpubsub                    = optional(bool, true)
            storage_account_blob                 = optional(bool, true)
            storage_account_file                 = optional(bool, true)
            storage_account_queue                = optional(bool, true)
            storage_account_table                = optional(bool, true)
            storage_account_web                  = optional(bool, true)
          }), {})
          private_link_locations                                 = optional(list(string), [])
          public_dns_zones                                       = optional(list(string), [])
          private_dns_zones                                      = optional(list(string), [])
          enable_private_dns_zone_virtual_network_link_on_hubs   = optional(bool, true)
          enable_private_dns_zone_virtual_network_link_on_spokes = optional(bool, true)
          virtual_network_resource_ids_to_link                   = optional(list(string), [])
        }), {})
      }), {})
    }), {})
    location = optional(string, "")
    tags     = optional(any, {})
    advanced = optional(any, {})
  })

Default: {}

configure_identity_resources

Description: If specified, will customize the "Identity" landing zone settings.

Type:

object({
    settings = optional(object({
      identity = optional(object({
        enabled = optional(bool, true)
        config = optional(object({
          enable_deny_public_ip             = optional(bool, true)
          enable_deny_rdp_from_internet     = optional(bool, true)
          enable_deny_subnet_without_nsg    = optional(bool, true)
          enable_deploy_azure_backup_on_vms = optional(bool, true)
        }), {})
      }), {})
    }), {})
  })

Default: {}

configure_management_resources

Description: If specified, will customize the "Management" landing zone settings and resources.

Type:

object({
    settings = optional(object({
      log_analytics = optional(object({
        enabled = optional(bool, true)
        config = optional(object({
          retention_in_days                                 = optional(number, 30)
          enable_monitoring_for_vm                          = optional(bool, true)
          enable_monitoring_for_vmss                        = optional(bool, true)
          enable_solution_for_agent_health_assessment       = optional(bool, true)
          enable_solution_for_anti_malware                  = optional(bool, true)
          enable_solution_for_change_tracking               = optional(bool, true)
          enable_solution_for_service_map                   = optional(bool, false)
          enable_solution_for_sql_assessment                = optional(bool, true)
          enable_solution_for_sql_vulnerability_assessment  = optional(bool, true)
          enable_solution_for_sql_advanced_threat_detection = optional(bool, true)
          enable_solution_for_updates                       = optional(bool, true)
          enable_solution_for_vm_insights                   = optional(bool, true)
          enable_solution_for_container_insights            = optional(bool, true)
          enable_sentinel                                   = optional(bool, true)
        }), {})
      }), {})
      security_center = optional(object({
        enabled = optional(bool, true)
        config = optional(object({
          email_security_contact                                = optional(string, "security_contact@replace_me")
          enable_defender_for_apis                              = optional(bool, true)
          enable_defender_for_app_services                      = optional(bool, true)
          enable_defender_for_arm                               = optional(bool, true)
          enable_defender_for_containers                        = optional(bool, true)
          enable_defender_for_cosmosdbs                         = optional(bool, true)
          enable_defender_for_cspm                              = optional(bool, true)
          enable_defender_for_dns                               = optional(bool, true)
          enable_defender_for_key_vault                         = optional(bool, true)
          enable_defender_for_oss_databases                     = optional(bool, true)
          enable_defender_for_servers                           = optional(bool, true)
          enable_defender_for_servers_vulnerability_assessments = optional(bool, true)
          enable_defender_for_sql_servers                       = optional(bool, true)
          enable_defender_for_sql_server_vms                    = optional(bool, true)
          enable_defender_for_storage                           = optional(bool, true)
        }), {})
      }), {})
    }), {})
    location = optional(string, "")
    tags     = optional(any, {})
    advanced = optional(any, {})
  })

Default: {}

create_duration_delay

Description: Used to tune terraform apply when faced with errors caused by API caching or eventual consistency. Sets a custom delay period after creation of the specified resource type.

Type:

object({
    azurerm_management_group      = optional(string, "30s")
    azurerm_policy_assignment     = optional(string, "30s")
    azurerm_policy_definition     = optional(string, "30s")
    azurerm_policy_set_definition = optional(string, "30s")
    azurerm_role_assignment       = optional(string, "0s")
    azurerm_role_definition       = optional(string, "60s")
  })

Default: {}

custom_landing_zones

Description: If specified, will deploy additional Management Groups alongside Enterprise-scale core Management Groups.
Although the object type for this input variable is set to any, the expected object is based on the following structure:

variable "custom_landing_zones" {
  type = map(
    object({
      display_name               = string
      parent_management_group_id = string
      subscription_ids           = list(string)
      archetype_config = object({
        archetype_id   = string
        parameters     = map(any)
        access_control = map(list(string))
      })
    })
  )

The decision not to hard code the structure in the input variable type is by design, as it allows Terraform to handle the input as a dynamic object type.
This was necessary to allow the parameters value to be correctly interpreted.
Without this, Terraform would throw an error if each parameter value wasn't a consistent type, as it would incorrectly identify the input as a tuple which must contain consistent type structure across all entries.

Note the id of the custom landing zone will be appended to var.root_id. The maximum length of the resulting name must be less than 90 characters.

The custom_landing_zones object is used to deploy additional Management Groups within the core Management Group hierarchy.
The main object parameters are display_name, parent_management_group_id, subscription_idsand archetype_config.

• display_name is the name assigned to the Management Group.

• parent_management_group_id is the name of the parent Management Group and must be a valid Management Group ID.

• subscription_ids is an object containing a list of Subscription IDs to assign to the current Management Group.

• archetype_config is used to configure the configuration applied to each Management Group. This object must contain valid entries for the archetype_id parameters, and access_control attributes.

The following example shows how you would add a simple Management Group under the myorg-landing-zones Management Group, where root_id = "myorg" and using the default_empty archetype definition.

  custom_landing_zones = {
    myorg-customer-corp = {
      display_name               = "MyOrg Customer Corp"
      parent_management_group_id = "myorg-landing-zones"
      subscription_ids           = [
        "00000000-0000-0000-0000-000000000000",
        "11111111-1111-1111-1111-111111111111",
      ]
      archetype_config = {
        archetype_id   = "default_empty"
        parameters     = {}
        access_control = {}
      }
    }
  }

Type: any

Default: {}

custom_policy_roles

Description: If specified, the custom_policy_roles variable overrides which Role Definition ID(s) (value) to assign for Policy Assignments with a Managed Identity, if the assigned "policyDefinitionId" (key) is included in this variable.

Type: map(list(string))

Default: {}

default_tags

Description: If specified, will set the default tags for all resources deployed by this module where supported.

Type: map(string)

Default: {}

deploy_connectivity_resources

Description: If set to true, will enable the "Connectivity" landing zone settings and add "Connectivity" resources into the current Subscription context.

Type: bool

Default: false

deploy_core_landing_zones

Description: If set to true, module will deploy the core Enterprise-scale Management Group hierarchy, including "out of the box" policies and roles.

Type: bool

Default: true

deploy_corp_landing_zones

Description: If set to true, module will deploy the "Corp" Management Group, including "out of the box" policies and roles.

Type: bool

Default: false

deploy_demo_landing_zones

Description: If set to true, module will deploy the demo "Landing Zone" Management Groups ("Corp", "Online", and "SAP") into the core Enterprise-scale Management Group hierarchy.

Type: bool

Default: false

deploy_diagnostics_for_mg

Description: If set to true, will deploy Diagnostic Settings for management groups

Type: bool

Default: false

deploy_identity_resources

Description: If set to true, will enable the "Identity" landing zone settings.

Type: bool

Default: false

deploy_management_resources

Description: If set to true, will enable the "Management" landing zone settings and add "Management" resources into the current Subscription context.

Type: bool

Default: false

deploy_online_landing_zones

Description: If set to true, module will deploy the "Online" Management Group, including "out of the box" policies and roles.

Type: bool

Default: false

deploy_sap_landing_zones

Description: If set to true, module will deploy the "SAP" Management Group, including "out of the box" policies and roles.

Type: bool

Default: false

destroy_duration_delay

Description: Used to tune terraform deploy when faced with errors caused by API caching or eventual consistency. Sets a custom delay period after destruction of the specified resource type.

Type:

object({
    azurerm_management_group      = optional(string, "0s")
    azurerm_policy_assignment     = optional(string, "0s")
    azurerm_policy_definition     = optional(string, "0s")
    azurerm_policy_set_definition = optional(string, "0s")
    azurerm_role_assignment       = optional(string, "0s")
    azurerm_role_definition       = optional(string, "0s")
  })

Default: {}

disable_base_module_tags

Description: If set to true, will remove the base module tags applied to all resources deployed by the module which support tags.

Type: bool

Default: false

disable_telemetry

Description: If set to true, will disable telemetry for the module. See https://aka.ms/alz-terraform-module-telemetry.

Type: bool

Default: false

library_path

Description: If specified, sets the path to a custom library folder for archetype artefacts.

Type: string

Default: ""

policy_non_compliance_message_default

Description: If set overrides the default non-compliance message used for policy assignments.

Type: string

Default: "This resource {enforcementMode} be compliant with the assigned policy."

policy_non_compliance_message_default_enabled

Description: If set to true, will enable the use of the default custom non-compliance messages for policy assignments if they are not provided.

Type: bool

Default: true

policy_non_compliance_message_enabled

Description: If set to false, will disable non-compliance messages altogether.

Type: bool

Default: true

policy_non_compliance_message_enforced_replacement

Description: If set overrides the non-compliance replacement used for enforced policy assignments.

Type: string

Default: "must"

policy_non_compliance_message_enforcement_placeholder

Description: If set overrides the non-compliance message placeholder used in message templates.

Type: string

Default: "{enforcementMode}"

policy_non_compliance_message_not_enforced_replacement

Description: If set overrides the non-compliance replacement used for unenforced policy assignments.

Type: string

Default: "should"

policy_non_compliance_message_not_supported_definitions

Description: If set, overrides the list of built-in policy definition that do not support non-compliance messages.

Type: list(string)

Default:

[
  "/providers/Microsoft.Authorization/policyDefinitions/1c6e92c9-99f0-4e55-9cf2-0c234dc48f99",
  "/providers/Microsoft.Authorization/policyDefinitions/1a5b4dca-0b6f-4cf5-907c-56316bc1bf3d",
  "/providers/Microsoft.Authorization/policyDefinitions/95edb821-ddaf-4404-9732-666045e056b4"
]

resource_custom_timeouts

Description: Optional - Used to tune terraform deploy when faced with errors caused by API limits.

For each supported resource type, there is a child object that specifies the create, update, and delete timeouts.
Each of these arguments takes a string representation of a duration, such as "60m" for 60 minutes, "10s" for ten seconds, or "2h" for two hours.
If a timeout is not specified, the default value for the resource will be used.

e.g.

resource_custom_timeouts = {
  azurerm_private_dns_zone = {
    create = "1h"
    update = "1h30m"
    delete = "30m"
    read   = "30s"
  }
}

Type:

object({
    azurerm_private_dns_zone = optional(object({
      create = optional(string, null)
      update = optional(string, null)
      read   = optional(string, null)
      delete = optional(string, null)
    }), {})
    azurerm_private_dns_zone_virtual_network_link = optional(object({
      create = optional(string, null)
      update = optional(string, null)
      read   = optional(string, null)
      delete = optional(string, null)
    }), {})
  })

Default: {}

root_id

Description: If specified, will set a custom Name (ID) value for the Enterprise-scale "root" Management Group, and append this to the ID for all core Enterprise-scale Management Groups.

Type: string

Default: "es"

root_name

Description: If specified, will set a custom Display Name value for the Enterprise-scale "root" Management Group.

Type: string

Default: "Enterprise-Scale"

strict_subscription_association

Description: If set to true, subscriptions associated to management groups will be exclusively set by the module and any added by another process will be removed. If set to false, the module will will only enforce association of the specified subscriptions and those added to management groups by other processes will not be removed. Default is false as this works better with subscription vending.

Type: bool

Default: false

subscription_id_connectivity

Description: If specified, identifies the Platform subscription for "Connectivity" for resource deployment and correct placement in the Management Group hierarchy.

Type: string

Default: ""

subscription_id_identity

Description: If specified, identifies the Platform subscription for "Identity" for resource deployment and correct placement in the Management Group hierarchy.

Type: string

Default: ""

subscription_id_management

Description: If specified, identifies the Platform subscription for "Management" for resource deployment and correct placement in the Management Group hierarchy.

Type: string

Default: ""

subscription_id_overrides

Description: If specified, will be used to assign subscription_ids to the default Enterprise-scale Management Groups.

Type: map(list(string))

Default: {}

template_file_variables

Description: If specified, provides the ability to define custom template variables used when reading in template files from the built-in and custom library_path.

Type: any

Default: {}

Resources

The following resources are used by this module:

• azapi_resource.diag_settings (resource)
• azurerm_automation_account.management (resource)
• azurerm_dns_zone.connectivity (resource)
• azurerm_express_route_gateway.virtual_wan (resource)
• azurerm_firewall.connectivity (resource)
• azurerm_firewall.virtual_wan (resource)
• azurerm_firewall_policy.connectivity (resource)
• azurerm_firewall_policy.virtual_wan (resource)
• azurerm_log_analytics_linked_service.management (resource)
• azurerm_log_analytics_solution.management (resource)
• azurerm_log_analytics_workspace.management (resource)
• azurerm_management_group.level_1 (resource)
• azurerm_management_group.level_2 (resource)
• azurerm_management_group.level_3 (resource)
• azurerm_management_group.level_4 (resource)
• azurerm_management_group.level_5 (resource)
• azurerm_management_group.level_6 (resource)
• azurerm_management_group_policy_assignment.enterprise_scale (resource)
• azurerm_management_group_subscription_association.enterprise_scale (resource)
• azurerm_network_ddos_protection_plan.connectivity (resource)
• azurerm_policy_definition.enterprise_scale (resource)
• azurerm_policy_set_definition.enterprise_scale (resource)
• azurerm_private_dns_zone.connectivity (resource)
• azurerm_private_dns_zone_virtual_network_link.connectivity (resource)
• azurerm_public_ip.connectivity (resource)
• azurerm_resource_group.connectivity (resource)
• azurerm_resource_group.management (resource)
• azurerm_resource_group.virtual_wan (resource)
• azurerm_role_assignment.enterprise_scale (resource)
• azurerm_role_assignment.policy_assignment (resource)
• azurerm_role_definition.enterprise_scale (resource)
• azurerm_subnet.connectivity (resource)
• azurerm_subscription_template_deployment.telemetry_connectivity (resource)
• azurerm_subscription_template_deployment.telemetry_core (resource)
• azurerm_subscription_template_deployment.telemetry_identity (resource)
• azurerm_subscription_template_deployment.telemetry_management (resource)
• azurerm_virtual_hub.virtual_wan (resource)
• azurerm_virtual_hub_connection.virtual_wan (resource)
• azurerm_virtual_hub_routing_intent.virtual_wan (resource)
• azurerm_virtual_network.connectivity (resource)
• azurerm_virtual_network_gateway.connectivity (resource)
• azurerm_virtual_network_peering.connectivity (resource)
• azurerm_virtual_wan.virtual_wan (resource)
• azurerm_vpn_gateway.virtual_wan (resource)
• random_id.telem (resource)
• time_sleep.after_azurerm_management_group (resource)
• time_sleep.after_azurerm_policy_assignment (resource)
• time_sleep.after_azurerm_policy_definition (resource)
• time_sleep.after_azurerm_policy_set_definition (resource)
• time_sleep.after_azurerm_role_assignment (resource)
• time_sleep.after_azurerm_role_definition (resource)
• azurerm_policy_definition.external_lookup (data source)
• azurerm_policy_set_definition.external_lookup (data source)

Outputs

The following outputs are exported:

azurerm_automation_account

Description: Returns the configuration data for all Automation Accounts created by this module.

azurerm_dns_zone

Description: Returns the configuration data for all DNS Zones created by this module.

azurerm_express_route_gateway

Description: Returns the configuration data for all (Virtual WAN) ExpressRoute Gateways created by this module.

azurerm_firewall

Description: Returns the configuration data for all Azure Firewalls created by this module.

azurerm_firewall_policy

Description: Returns the configuration data for all Azure Firewall Policies created by this module.

azurerm_log_analytics_linked_service

Description: Returns the configuration data for all Log Analytics linked services created by this module.

azurerm_log_analytics_solution

Description: Returns the configuration data for all Log Analytics solutions created by this module.

azurerm_log_analytics_workspace

Description: Returns the configuration data for all Log Analytics workspaces created by this module.

azurerm_management_group

Description: Returns the configuration data for all Management Groups created by this module.

azurerm_management_group_policy_assignment

Description: Returns the configuration data for all Management Group Policy Assignments created by this module.

azurerm_management_group_subscription_association

Description: Returns the configuration data for all Management Group Subscription Associations created by this module.

azurerm_network_ddos_protection_plan

Description: Returns the configuration data for all DDoS Protection Plans created by this module.

azurerm_policy_definition

Description: Returns the configuration data for all Policy Definitions created by this module.

azurerm_policy_set_definition

Description: Returns the configuration data for all Policy Set Definitions created by this module.

azurerm_private_dns_zone

Description: Returns the configuration data for all Private DNS Zones created by this module.

azurerm_private_dns_zone_virtual_network_link

Description: Returns the configuration data for all Private DNS Zone network links created by this module.

azurerm_public_ip

Description: Returns the configuration data for all Public IPs created by this module.

azurerm_resource_group

Description: Returns the configuration data for all Resource Groups created by this module.

azurerm_role_assignment

Description: Returns the configuration data for all Role Assignments created by this module.

azurerm_role_definition

Description: Returns the configuration data for all Role Definitions created by this module.

azurerm_subnet

Description: Returns the configuration data for all Subnets created by this module.

azurerm_virtual_hub

Description: Returns the configuration data for all Virtual Hubs created by this module.

azurerm_virtual_hub_connection

Description: Returns the configuration data for all Virtual Hub Connections created by this module.

azurerm_virtual_hub_routing_intent

Description: Returns the configuration data for all Virtual Hub Routing Intents created by this module.

azurerm_virtual_network

Description: Returns the configuration data for all Virtual Networks created by this module.

azurerm_virtual_network_gateway

Description: Returns the configuration data for all Virtual Network Gateways created by this module.

azurerm_virtual_network_peering

Description: Returns the configuration data for all Virtual Network Peerings created by this module.

azurerm_virtual_wan

Description: Returns the configuration data for all Virtual WANs created by this module.

azurerm_vpn_gateway

Description: Returns the configuration data for all (Virtual WAN) VPN Gateways created by this module.

Telemetry

NOTE: The following statement is applicable from release v2.0.0 onwards

When you deploy one or more modules using the Azure landing zones Terraform module, Microsoft can identify the installation of said module/s with the deployed Azure resources. Microsoft can correlate these resources used to support the software. Microsoft collects this information to provide the best experiences with their products and to operate their business. The telemetry is collected through customer usage attribution. The data is collected and governed by Microsoft's privacy policies.

If you don't wish to send usage data to Microsoft, details on how to turn it off can be found here.

License

• MIT License

Contributing

• Contributing
  • Raising an Issue
  • Feature Requests
  • Contributing to Code
  • Contributing to Documentation

terraform-azurerm-caf-enterprise-scale's Issues

Add issue and pull request templates

To simplify contribution from Open Source community, add .github/ISSUE_TEMPLATE.md and .github/PULL_REQUEST_TEMPLATE.md files to simplify the process of raising Issues and PRs against this project.

Managementgroup displayname example differ

Managementgroup display name example in the deployment examples with "myorg-x" differ from Enterprise-Scale examples with "Management Group prefix".

When using any of the Enterprise-Scale references from: https://github.com/Azure/Enterprise-Scale the "Management Group prefix" is also appended as part of each child Management Group display name without any spaces etc.

Streamlining the examples would make it easier to grasp and move between these solutions.

Policy definitions not available in AzureGov

In Azure commercial I can see the policies needed e.g.

$ az cloud set -n AzureCloud
Switched active cloud to 'AzureCloud'.

$ az policy definition show --name "9d2b61b4-1d14-4a63-be30-d4498e7ad2cf"
{
  "description": "This policy deploys the Log Analytics agent to Linux Azure Arc machines if the agent isn't installed.",
  "displayName": "[Preview]: Deploy Log Analytics agent to Linux Azure Arc machines",
  "id": "/providers/Microsoft.Authorization/policyDefinitions/9d2b61b4-1d14-4a63-be30-d4498e7ad2cf",
...

But AzureGov....

$ az cloud set -n AzureUSGovernment
Switched active cloud to 'AzureUSGovernment'.

$ az policy definition show --name "9d2b61b4-1d14-4a63-be30-d4498e7ad2cf"
The policy definition '9d2b61b4-1d14-4a63-be30-d4498e7ad2cf' could not be found.

I also tried to find them by displayName in case the IDs (name) was different for some reason. No luck

This is a continuation of a discussion that started in #23 (comment)

Hardcoded location value in policy definition

policy_definition_es_deploy_asc_standard.json contains hardcoded 'northeurope' value on "location" at line 126. Should this be a variable?

Error assigning custom Azure policy to Archetype

When attempting to create and assign a custom Azure Policy to an Archetype we are getting the following error:

Error: reading Policy Definition "Tag-Enforcement-Policy": policy.DefinitionsClient#GetBuiltIn: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="PolicyDefinitionNotFound" Message="The policy definition 'Tag-Enforcement-Policy' could not be found."

  on .terraform/modules/enterprise_scale/locals.policy_assignments.tf line 141, in data "azurerm_policy_definition" "external_lookup":
 141: data "azurerm_policy_definition" "external_lookup" {

Our belief is that the module is only checking for existing Azure Policies and is not checking for the custom policies that can be created as part of running this module. We have attempted to copy the example code located in the code base at modules/archetypes/lib/policy_definitions/policy_definition_es_deny_appgw_without_waf.json and modules/archetypes/lib/policy_assignments/policy_assignment_es_deny_appgw_without_waf.tmpl.json

Our custom policy in lib/policy_definitions/policy_definition_require_tags.json contains:

{
  "name": "Tag-Enforcement-Policy",
  "type": "Microsoft.Authorization/policyDefinitions",
  "apiVersion": "2019-09-01",
  "properties": {
    "displayName": "Require a tag on resources",
    "policyType": "Custom",
    "mode": "Indexed",
    "description": "Enforces existence of a tag. Does not apply to resource groups.",
    "metadata": {
      "category": "Tags",
      "createdBy": "b769b5da-f41d-4ca2-a1ad-d16a67f9f7ba",
      "createdOn": "2020-11-05T21:45:23.6465795Z",
      "updatedBy": null,
      "updatedOn": null
    },
    "parameters": {
      "tagName": {
        "type": "String",
        "metadata": {
          "displayName": "Tag Name",
          "description": "Name of the tag, such as 'environment'"
        }
      }
    },
    "policyRule": {
      "if": {
        "field": "[concat('tags[', parameters('tagName'), ']')]",
        "exists": "false"
      },
      "then": {
        "effect": "deny"
      }
    }
  }
}

Our Policy Assignment in lib/policy_assignments/policy_assignment_require_tags.json contains:

{
  "name": "Tag-Enforcement",
  "type": "Microsoft.Authorization/policyAssignments",
  "apiVersion": "2019-09-01",
  "properties": {
    "description": "Enforces existence of a tag. Does not apply to resource groups.",
    "displayName": "Tag-Enforcement",
    "notScopes": [],
    "parameters": {
      "tagName": {
        "value": "name"
      }
    },
    "policyDefinitionId": "${root_scope_resource_id}/providers/Microsoft.Authorization/policyDefinitions/Tag-Enforcement-Policy",
    "scope": "${current_scope_resource_id}"
  },
  "location": "${default_location}",
  "identity": {
    "type": "SystemAssigned"
  }
}

[Azure Policies] Support for Non-compliance messages

Requested feature

Incorporation of non-compliance messages in CAF Entreprise Scale Framework

Documentation

https://docs.microsoft.com/en-gb/azure/governance/policy/concepts/assignment-structure#non-compliance-messages

Example

In the policy assignment, please note nonComplianceMessages attribute

{
    "properties": {
        "displayName": "Enforce resource naming rules",
        "description": "Force resource names to begin with DeptA and end with -LC",
        "metadata": {
            "assignedBy": "Cloud Center of Excellence"
        },
        "enforcementMode": "DoNotEnforce",
        "notScopes": [],
        "policyDefinitionId": "/subscriptions/{mySubscriptionID}/providers/Microsoft.Authorization/policyDefinitions/ResourceNaming",
        "nonComplianceMessages": [
            {
                "message": "Resource names must start with 'DeptA' and end with '-LC'."
            }
        ],
        "parameters": {
            "prefix": {
                "value": "DeptA"
            },
            "suffix": {
                "value": "-LC"
            }
        }
    }
}

Custom variable substitution in the archetype_extension_org.json, possible?

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

Is it possible to add some kind of variable substitution on archetype_extension_org.json?
We have a use-case where we are want to specify the workspace-id for the Deploy-Nsg-FlowLogs-to-LA policy, and unless there's a way to pass that variable from Terraform to the Policy, we are pretty much stuck with the hardcoded value...

Basically, I'm looking to know if its possible to use something in the form of:

        "Deploy-Nsg-FlowLogs-to-LA": {
          "workspace": "${workspace_id}"
        }

Our archetype_extension_org.json with the Deploy-Nsg-FlowLogs-to-LA with a "hardcoded" workspace id...

{
  "extend_es_root": {
    "policy_assignments": [
      "ISO-27001-2013",
      "Deny-PublicEndpoints",
      "Deny-Storage-minTLS",
      "Deny-Resource-Locations",
      "Deny-RSG-Locations"
    ],
    "policy_definitions": [
      "Deny-SSH-From-Internet"
    ],
    "policy_set_definitions": [],
    "role_definitions": [],
    "archetype_config": {
      "parameters": {
        "Deny-Resource-Locations": {
          "listOfAllowedLocations": [
            "westeurope",
            "northeurope"
          ]
        },
        "Deny-RSG-Locations": {
          "listOfAllowedLocations": [
            "westeurope",
            "northeurope"
          ]
        },
        "Deploy-Nsg-FlowLogs-to-LA": {
          "workspace": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx"
        }
      },
      "access_control": {}
    }
  }
}

Document the benefits and use of template_file_variables

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

Is your feature request related to a problem?

I'm trying to use the module as effectively as it can be, but the lack of more extensive documentation is at least holding me back to understand all the configuration options in depth. template_file_variables is an input option that I haven't yet understood why and where to use. Please enhance the wiki documentation on this part.

Describe the solution you'd like
Humbly asking enhancing the wiki documentation for the use of template_file_variables. Use cases and why and how should we use this?

Additional context

Policy definitions and Policy assignment tries to deploy multiple time in the same scope

Hello, we try to deploy caf enterprise scale to our new tenant. I read the documentation and did the same as it is described in the Default configuration.

Main.tf :

provider "azurerm" {
  subscription_id = var.subscription_id
  client_id       = var.client_id
  client_secret   = var.client_secret
  tenant_id       = var.tenant_id
  features {}
}

data "azurerm_client_config" "current" {}

module "enterprise_scale" {
  source  = "Azure/caf-enterprise-scale/azurerm"
  version = "0.3.0"

  root_parent_id = data.azurerm_client_config.current.tenant_id
  root_id        = "MyOrganizationid"
  root_name      = "MyOrganization name"

}
module "management_group_archetypes" {
  for_each = local.es_landing_zones_map
  source   = "./modules/archetypes"

  root_id                 = "${local.provider_path.management_groups}${local.root_id}"
  scope_id                = each.key
  archetype_id            = each.value.archetype_config.archetype_id
  parameters              = each.value.archetype_config.parameters
  enforcement_mode        = try(module.management_resources.configuration.archetype_config_overrides[basename(each.key)].enforcement_mode, null)
  access_control          = each.value.archetype_config.access_control
  library_path            = local.library_path
  template_file_variables = local.template_file_variables
  default_location        = local.default_location
}
module "management_resources" {
  source = "./modules/management"

  enabled         = local.deploy_management_resources
  root_id         = local.root_id
  subscription_id = local.subscription_id_management
  settings        = local.configure_management_resources.settings

  location = coalesce(local.configure_management_resources.location, local.default_location)
  tags     = coalesce(local.configure_management_resources.tags, local.default_tags)

  resource_prefix                              = try(local.configure_management_resources.advanced.resource_prefix, local.empty_string)
  resource_suffix                              = try(local.configure_management_resources.advanced.resource_suffix, local.empty_string)
  existing_resource_group_name                 = try(local.configure_management_resources.advanced.existing_resource_group_name, local.empty_string)
  existing_log_analytics_workspace_resource_id = try(local.configure_management_resources.advanced.existing_log_analytics_workspace_resource_id, local.empty_string)
  existing_automation_account_resource_id      = try(local.configure_management_resources.advanced.existing_automation_account_resource_id, local.empty_string)
  link_log_analytics_to_automation_account     = try(local.configure_management_resources.advanced.link_log_analytics_to_automation_account, true)
  custom_settings_by_resource_type             = try(local.configure_management_resources.advanced.custom_settings_by_resource_type, local.empty_map)
}

deployment always failed with such errors:

Error: A resource with the ID "/providers/Microsoft.Management/managementGroups/nten-landing-zones/providers/Microsoft.Authorization/policyAssignments/Deny-Priv-Escalation-AKS" already exists - to be managed via Terraform this resource needs to be imported into the State. Please see the resource documentation for "azurerm_policy_assignment" for more information.

  on .terraform\modules\enterprise_scale\resources.policy_assignments.tf line 1, in resource "azurerm_policy_assignment" "enterprise_scale":     
   1: resource "azurerm_policy_assignment" "enterprise_scale" {

Error: A resource with the ID "/providers/Microsoft.Management/managementGroups/nten-landing-zones/providers/Microsoft.Authorization/policyAssignments/Deny-Priv-Containers-AKS" already exists - to be managed via Terraform this resource needs to be imported into the State. Please see the resource documentation for "azurerm_policy_assignment" for more information.

  on .terraform\modules\enterprise_scale\resources.policy_assignments.tf line 1, in resource "azurerm_policy_assignment" "enterprise_scale":     
   1: resource "azurerm_policy_assignment" "enterprise_scale" {

Error: A resource with the ID "/providers/Microsoft.Management/managementGroups/nten-landing-zones/providers/Microsoft.Authorization/policyAssignments/Deny-http-Ingress-AKS" already exists - to be managed via Terraform this resource needs to be imported into the State. Please see the resource documentation for "azurerm_policy_assignment" for more information.

  on .terraform\modules\enterprise_scale\resources.policy_assignments.tf line 1, in resource "azurerm_policy_assignment" "enterprise_scale":     
   1: resource "azurerm_policy_assignment" "enterprise_scale" {

These policy assignments have been deployed. It is tried to deploy them once again in the same deployment.

Add CONTRIBUTING.md file

Add a CONTRIBUTING.md file in the repository root (or /docs or /.github folders) to explain the contribution policy for the project.

404 url

Hi,

Dead link on this one.

https://github.com/Azure/terraform-azurerm-caf-enterprise-scale/tree/main/modules/terraform-azurerm-caf-enterprise-scale-archetypes/lib

root_parent_id validation for Management Group ID

Hi.
I am trying to use an existing Management Group as root_parent_id.
This group has an id that contains an underscore, causing the validation to fail.

According to variables.tf: "The root_parent_id value must be a valid GUID, or Management Group ID."

According to Azure Portal, the id can contain: ASCII letter, digit, -, _, (, ), .
However, that is not matched by the regex in variables.tf, which only allows for letters, numbers and hyphen.

Is this an intended limitation?

/Niklas

Improve documentation for example deployments

Improve documentation for example deployments #63

User story

As a new user of the module
I want clearer documentation of what the module deploys
So that I can get a better understanding of what will be deployed into my environment.

User acceptance criteria

Given i am a new user of the module
When visiting the example pages of terraform-caf-enterprise-scale
Then i want to find a copy-paste code example and a snapshot of the expected result in my environment.

Definition of done

• Complete Task 1
• Complete Task 2
• Complete Task 3
• Complete Task 4
• Complete Task 5

Tasks to complete

• Task 1: Update the Deploy Default Configuration example on the wiki

• Task 2: Update the Deploy Demo Landing Zone Archetypes and Deploy Custom Landing Zone Archetypes example pages on wiki

• Task 3: Create the Deploy Using Module Nesting example page and add it to the wiki

• Task 4: Update each individual variable page on the Module Variables main page, in the context of Overview, Default value, Validation and Usage section.

• Task 5: Update the Module Variables main page.

Role definition assignableScopes bug

The code contains a bug whereby the assignableScopes value in the library template will always be ignored due to incorrect logic in the ./resources.role_definitions.tf file.

This results in the assignableScopes value always being set to the current scope only for all Role Definitions.

ES-Deploy-Nsg-FlowLogs remediation task fails

When assigning the ES-Deploy-Nsg-FlowLogs policy to the Azure platform, the policy evaluation accurately reports non-compliant status for NSGs without flow logs configured.

The policy definition for policy_definition_es_deploy_nsg_flowlogs.json has a deployIfNotExists effect and configured to use a system assigned managed identity.

However, the remediation task fails since there is no name for the Microsoft.Network/networkWatchers/flowLogs resource in the policy definition template.

Here is the actual error message that can be found in Azure Activity logs for the deployIfNotExists Policy action:

The 'ifNotExists' target resource type 'Microsoft.Network/networkWatchers/flowLogs' and name '' are not valid in policy assignment '/providers/Microsoft.Management/managementGroups/cu/providers/Microsoft.Authorization/policyAssignments/CU-Deploy-NSG-FlowLogs' and definition '/providers/Microsoft.Management/managementGroups/cu/providers/Microsoft.Authorization/policyDefinitions/ES-Deploy-Nsg-FlowLogs' when evaluating a resource of type 'microsoft.network/networksecuritygroups'.

The built-in policy to deploy NSG flow logs that is similar to this does include a name property for the flow log (see line 62 in the file below)

https://github.com/Azure/azure-policy/blob/master/built-in-policies/policyDefinitions/Network/NetworkSecurityGroup_FlowLog_Deploy.json

Once you add the following line to the policy_definition_es_deploy_nsg_flowlogs.json file, the remediation task works as expected.

"name": "[if(empty(coalesce(field('Microsoft.Network/networkSecurityGroups/flowLogs[*].id'))), 'null/null', concat(split(first(field('Microsoft.Network/networkSecurityGroups/flowLogs[*].id')), '/')[8], '/', split(first(field('Microsoft.Network/networkSecurityGroups/flowLogs[*].id')), '/')[10]))]",

Feature Request: add a way to override name of the resource group created by the management resources.

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

Is your feature request related to a problem?
No

Describe the solution you'd like
I am not sure if I can set this up, but I would like to change the name of the management resource group created by the module to be able to fit our naming convention.

The current behaviour is a resource group with the name -mgmt.

It would be awesome to have it fit in with the azurecaf_name and azurecaf_naming_convention modules.

Additional context

Failure sending request: StatusCode=429 -- Original Error: context deadline exceeded

Problem statement

Since 2020-04-22, the test framework has been unable to complete successfully despite the module working on individual deployments.

Observations

• Some Unit pipeline jobs complete terraform plan without any problems, but others take up to 4x longer to complete, or fail to complete before Terraform terminates the operation with Failure sending request: StatusCode=429 -- Original Error: context deadline exceeded.
• Re-running the failed Unit pipeline jobs will eventually end in success.
• Running the E2E pipeline jobs has the same issue as the Unit pipeline jobs. Assuming the job gets past the terraform plan step, terraform apply is unable to complete, and terraform destroy also then fails.
  • terraform apply terminates with error: Failure sending request: StatusCode=429 -- Original Error: context deadline exceeded
  • terraform destroy terminates with error: Failure sending request: StatusCode=429 -- Original Error: context deadline exceeded
• The module logic is working and can be deployed successfully, as long as the 429 errors aren't observed

There's clearly a common pattern linked to Failure sending request: StatusCode=429 -- Original Error: context deadline exceeded, suggesting API throttling.

This requires further investigation as this wasn't previously an issue.

Although this is primarily a problem with our test framework, we have also observed this when running individual deployments from a workstation, mostly impacting 'terraform plan' and terraform destroy, but also when running terraform apply with a high setting for -parallelism.

Similar issues

These issues also exhibit similar behaviors with the azurerm provider

• Request throttling while deploying
• Request throttling while refreshing state
• Failure sending request: StatusCode=429

Call to customers

If you are seeing similar behaviour, please 👍 this Issue and add your comments below!

"ES-Deny-VMIPForwarding" does not deny IP Forwarding

Hi

We can't get "ES-Deny-VMIPForwarding" policy assignment to deny VM ip forwarding.

The policy assigment uses a build in definition, so it might be something wrong upstream.

{
  "name": "ES-Deny-VMIPForwarding",
  "type": "Microsoft.Authorization/policyAssignments",
  "apiVersion": "2019-09-01",
  "properties": {
    "description": "Deny IP Forwarding on Virtual Machines.",
    "displayName": "ES-Deny-IP-Forwarding-On-VM",
    "notScopes": [],
    "parameters": {
      "effect": {
        "value": "Disabled"
      }
    },
    "policyDefinitionId": "/providers/Microsoft.Authorization/policyDefinitions/bd352bd5-2853-4985-bf0d-73806b4a5744",
    "scope": "${current_scope_resource_id}"
  },
  "sku": {
    "name": "A0",
    "tier": "Free"
  },
  "location": "${default_location}",
  "identity": {
    "type": "None"
  }
}

The policy assignment has set "Disabled" as its default value in the parameters section. The only option i can see in the azure portal is AuditIfNotExists or Disabled. i would expect Deny to be a option here? I have tried it with both, but i have yet to try and set "Deny" through the main.tf file.

In the complience reason details blade we get this error. So it has identified that the VM is breaking the policy, but no remediation tasks are available and i still get to turn on and off the VM IP Forwarding.

Reason for non-compliance
No related resources match the effect details in the policy definition. (Error code: AssessmentNotFound)
Existence condition
Type
Microsoft.Security/assessments
Name
c3b51c94-588b-426b-a892-***

Are we missing something?
Any suggestions are appreciated.

Log Analytics Subscription Id missing from Policy Parameter ID

Policies that enforce Diagnostic Logs to be configured to send to Log Analytics appear to not be adding in the subscription ID or the unique GUID for the workspace name:

Example of the parameter in an assigned policy: Deploy-Diagnostics-PublicIP

/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/main-mgmt/providers/Microsoft.OperationalInsights/workspaces/main-la-00000000-0000-0000-0000-000000000000

Error upgrading from v0.1.2 to v0.2.0

Wanted to post this in case folks run into this error when attempting to upgrade from v0.1.2 to v0.2.0

│ Error: deleting Policy Definition "Deploy-Diagnostics-PublicIP": policy.DefinitionsClient#DeleteAtManagementGroup: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="InvalidDeletePolicyDefinitionRequest" Message="The policy definition 'Deploy-Diagnostics-PublicIP' cannot be deleted. It is referenced by the policy set definition '/providers/Microsoft.Management/managementgroups/cu/providers/Microsoft.Authorization/policySetDefinitions/Deploy-Diag-LogAnalytics'. Please remove this policy definition from all policy set definitions that reference it."

Release notes indicate that the Deploy-Diagnostics-PublicIP custom policy definition has been removed in favor of a built-in policy.

If you had v0.1.2 running in your environment, this policy was included in the Policy Initiative (policySet) Deploy-Diag-LogAnalytics. This policy is no longer included in the policy set but the terraform apply command attempts to delete the policy before removing it from the policy set.

To work around this error, you'll manually remove the policy from the policy set. Go to the Azure Portal, navigate to Policy definitions and search for the policy definition named Deploy Diagnostic Settings to Azure Services. Edit the initiative and remove the Deploy Diagnostic Settings for Public IP addresses to Log Analytics workspace policy then re-run the terraform plan and terraform apply commands.

Variable validation for archetype module fails

When creating a custom landing zone hierarchy, validation fails if _ is present in the map key name. Underscores are supported in a management group ID in ARM, so should be supported by this module as well.

Example hierarchy that doesn't work:

custom_landing_zones = {
  (var.root_id) = {
      display_name               = var.root_name
      parent_management_group_id = data.azurerm_client_config.current.tenant_id
      subscription_ids           = []
      archetype_config = {
      archetype_id   = "es_root"
      parameters     = {}
      access_control = {}
      }
  }

  "${var.root_id}_landing_zones" = {
      display_name               = "Landing Zones"
      parent_management_group_id = var.root_id
      subscription_ids           = []
      archetype_config = {
      archetype_id   = "es_landing_zones"
      parameters     = {}
      access_control = {}
      }
  }
}

Terraform plan output:

$ terraform plan

Error: Invalid value for variable

  on .terraform/modules/enterprise_scale/main.tf line 12, in module "management_group_archetypes":
  12:   scope_id                = each.key

The scope_id value must be a valid Subscription or Management Group ID.

This was checked by the validation rule at
.terraform/modules/enterprise_scale/modules/archetypes/variables.tf:21,3-13.

Regex validation:

Provide the ability to extend built-in archetype definitions

Problem statement

The module currently provides a set of built in archetype definitions which are used to define which policies and roles to associate at the assigned scope.

For example, the es_landing_zones archetype definition specifies that the following Policy Assignments should be created at any scope where this archetype is referenced:

• Deny-IP-Forwarding
• Deny-RDP-From-Internet
• Deny-Storage-http
• Deny-Subnet-Without-Nsg
• Deploy-AKS-Policy
• Deploy-SQL-DB-Auditing
• Deploy-VM-Backup
• Deploy-SQL-Security
• Deny-Priv-Escalation-AKS
• Deny-Priv-Containers-AKS
• Deny-http-Ingress-AKS

To change these settings, the module allows you to override this using one of two methods:

1. Copy the archetype definition file into your custom library (as specified using the library_path variable) and make your amendments to this file (leaving the existing archetype_id value so this is used in preference to the built-in definition).
2. Create a new archetype definition file in your custom library (as specified using the library_path variable) with a new archetype_id value, update to include the required configuration, and then associate this archetype definition at the desired scope using the archetype_id field within either archetype_config_overrides or custom_landing_zones variables as required.

In both of the above scenarios, there will be a maintenance overhead if you want to stay up to date with changes being published in new module releases.

Without proper maintenance, custom archetype definitions can cause failed deployments when moving to a new module version in some scenarios. An example of this is when an existing custom Policy Set Definition is updated to include additional new custom Policy Definitions. Although the module upgrade will pull in the updated Policy Set Definition, without proper maintenance of the dependencies you can get an error where the newly required custom Policy Definitions haven't been deployed by the module.

User story

To provide an improved user experience, we would like to introduce the following capability:

As a module consumer
I want to be able to extend an existing [built-in] archetype definition with my additional configuration settings without having to create a complete copy
So that I can benefit from the ability to customise my environment, whilst still receiving all updates with new module releases.

Additional context

Taking the es_landing_zones archetype definition as an example.

The built-in definition contains the following configuration:

{
    "es_landing_zones": {
        "policy_assignments": [
            "Deny-IP-Forwarding",
            "Deny-RDP-From-Internet",
            "Deny-Storage-http",
            "Deny-Subnet-Without-Nsg",
            "Deploy-AKS-Policy",
            "Deploy-SQL-DB-Auditing",
            "Deploy-VM-Backup",
            "Deploy-SQL-Security",
            "Deny-Priv-Escalation-AKS",
            "Deny-Priv-Containers-AKS",
            "Deny-http-Ingress-AKS"
        ],
        "policy_definitions": [],
        "policy_set_definitions": [],
        "role_definitions": [],
        "archetype_config": {
            "parameters": {},
            "access_control": {}
        }
    }
}

To add new settings to this, the desired outcome would be to provide an input which can be used to extend this configuration, based on the above configuration but with a method to identify the template as an expansion to the above template.

To add the Deny-Resource-Locations Policy Assignment to the es_landing_zones archetype definition with a default parameter stating the listOfAllowedLocations, the desired approach would be to simply add a template to the custom library indicating the desired additions, such as the following example:

{
    "extend_es_landing_zones": {
        "policy_assignments": [
            "Deny-Resource-Locations"
        ],
        "policy_definitions": [],
        "policy_set_definitions": [],
        "role_definitions": [],
        "archetype_config": {
            "parameters": {
                "Deny-Resource-Locations": {
                    "listOfAllowedLocations": [
                        "eastus",
                        "westus"
                    ]
                }
            },
            "access_control": {}
        }
    }
}

Stretch goal

In addition to the above, it would also be useful to remove specific items from a built-in archetype definition if not required.

Using the previous example, to remove the Deny-RDP-From-Internet Policy Assignment from the es_landing_zones archetype definition, the desired approach would be to simply add a template to the custom library indicating the desired exclusions, such as the following example:

{
    "exclude_es_landing_zones": {
        "policy_assignments": [
            "Deny-RDP-From-Internet"
        ],
        "policy_definitions": [],
        "policy_set_definitions": [],
        "role_definitions": [],
        "archetype_config": {
            "parameters": {},
            "access_control": {}
        }
    }
}

Core Management Group Hierachy

Hi,

Is it possible to be able to adapt the core "mandatory" Management Group hierarchy?

e.g. Push then down to the next level.
I would like a prod\no-prod group under the root and have the Enterprise Scale hierarchy under the prod one.

e.g. Some may not be required. Maybe an on\off toggle?

Thanks
Dan

Missing documentation - and comprehension

Hi Kevin,

thanks for the great piece of software you created with this module!

Having the need to deploy CAF Landing Zones with Terraform in my current project I was happy, when a colleague pointed me to this module.
I read the 'Getting started', deployed the default confuguration - and succeed within the first try.

I know quite much about Terraform, Azure CAF, Landing Zones, Management Groups and Policies.
But to be honest: I hardly have a clue, how this module is working 'under the hood'.
So I'm missing a lot of documentation.

As a non-developer, I would like to understand things like

• How can I customize this module?
• Where is the entry point?
• Where are configurable values?
• How to use the /test folder?
• ...

I would like to support with contributing documentation. So just let me know, whether this is requested and if yes, how to start.

Cheers,
Jens

Add a way to manage excluded scopes in the policy assignment

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

Add a way to manage excluded scopes in the policy assignment

This module works great. The only missing feature on our side is that there is no way to manage the excluded scopes in the policy assignment. It is always an empty array in the definition JSON file ("notScopes": []).

Adding a parameter to the archetype definition that would be passed to the assignment OR having a specific module variable mapping assignments to excluded scopes would fit our needs.

Thanks

Ignoring subscriptions added to sandboxes management groups

As per the recommendation here (https://docs.microsoft.com/en-us/azure/cloud-adoption-framework/ready/enterprise-scale/management-group-and-subscription-organization#define-a-management-group-hierarchy) I've set the default subscription management group to Sandboxes, and then manually moved the Visual Studio subscriptions within the Azure account to that management group.

Subsequently, when I run a plan, Terraform is looking to remove those subscriptions from the Sandboxes management group

  # module.cip_enterprise_scale.azurerm_management_group.level_2["/providers/Microsoft.Management/managementGroups/cip-sandboxes"] will be updated in-place
  ~ resource "azurerm_management_group" "level_2" {
        id                         = "/providers/Microsoft.Management/managementGroups/cip-sandboxes"
        name                       = "cip-sandboxes"
      ~ subscription_ids           = [
          - "4551fd90-6a3f-4bcc-9002-xxxxxxxxxxxxx",
          - "78a034e4-11b5-4f96-9543-xxxxxxxxxxxxx",
          - "90dce378-333f-407e-8e3b-xxxxxxxxxxxxx",
          - "f1dd6316-e7e6-453f-91da-xxxxxxxxxxxxx",
            # (2 unchanged elements hidden)
        ]
        # (3 unchanged attributes hidden)
    }

Two issues here;
Firstly, I don't know what would happen to those subscriptions should I apply this plan, so I'm rather apprehensive to do so.

Secondly, I know I could just add them to the Terraform subscription_id_overrides definition, but as an organisation with developers who could at any point create their own new Visual Studio subscription, this is not maintainable.

Ideally, it would be great to be able to continue to use subscription_id_overrides, but only on an adding basis. Ignore / don't remove any subscriptions added outside of Terraform.

I'm thinking adding lifecycle.ignore_changes to level 1 and 2, similar to below, in resources.management_groups.tf should fix this?

resource "azurerm_management_group" "level_1" {
  for_each = local.azurerm_management_group_level_1

  name                       = each.value.id
  display_name               = each.value.display_name
  parent_management_group_id = "${local.provider_path.management_groups}${each.value.parent_management_group_id}"
  subscription_ids           = each.value.subscription_ids

  lifecycle {
    ignore_changes = [ subscription_ids ]
  }
}

Allow role configuration for Policy Assignments with Managed Identity

Currently, this module auto-generates Role Assignments for any Policy Assignment when a Managed Identity is required to support policies using Modify or DeployIfNotExists effects.

The module uses the roleDefinitionIds field from the assigned Policy Definition(s) to determine which role(s) need to be assigned to the system-assigned managed identity created automatically for these Policy Assignments.

In some cases, there may be multiple built-in roleDefinitionIds specified to ensure least privilege access is achieved without using custom roles. This results in multiple Role Assignments being created for a single Policy Assignment, which can be wasteful of the current 2,000 Role Assignments per scope platform limit.

In some cases, there are also built-in Policy Set Definitions containing Policy Definitions where a required role (such as Contributor) renders all other roles unnecessary.

For example:

To improve this situation, it would be useful to be able to assign a roleDefinitionIds over-ride value which can be set in preference to the dynamically generated values.

Try to assig an policy set, will not work

Hi,

I am trying to assign a policy set to my root group. But it complains that my definition does not exist.
Does not assigning policy sets work?

{
"es_root": {
"policy_assignments": [
"ES-Allowed-Locations",
"ES-Deploy-ForwardActLogs"

"policy_set_definitions": [
"ES-Deny-Public-Endpoints-for-PaaS-Services",
"ES-Deploy-Diagnostics-LogAnalytics",

Error: reading Policy Definition "ES-Deploy-Diagnostics-LogAnalytics": policy.DefinitionsClient#GetBuiltIn: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="PolicyDefinitionNotFound" Message="The policy definition 'ES-Deploy-Diagnostics-LogAnalytics' could not be found."

There are two different examples of policy assignments that point to the same policy set.

• ES-Deploy-ForwardDiagLog
• ES-Deploy-ForwardActLogs
  Can not see that there is any difference between these?

Both of these do not like that I try to assign them to my root group.

BR
Magnus

Add example page for built-in archetype definitions expansion

Create new example page describing how to extend or exclude an existing [built-in] archetype definition with additional configuration settings.

Related to issue: #72

Policy remidiation

Hi,
Is there a milestone to assign roles for managed identities for policy auto-remediation.
Or should principal ID of managed identities be an output of the module?

Cannot create role definition and assign to mgmn group

We are getting problem to create new roles.

We are trying to add to new role definitions.
One for netops and secops.

We are trying to assign to root.

Ex of netops:
{
"name": "88888888-8888-8888-8888-888888888888",
"type": "Microsoft.Authorization/roleDefinitions",
"apiVersion": "2018-01-01-preview",
"properties": {
"roleName": "ES-NetOps",
"description": "Enterprise-scale custom Role Definition. Grants access for network operations",
"type": "customRole",
"permissions": [
{
"actions": [
"/read",
"Microsoft.Network/vpnGateways/",
"Microsoft.Network/routeTables/write",
"Microsoft.Network/vpnSites/*"
],
"notActions": [],
"dataActions": [],
"notDataActions": []
}
],
"assignableScopes": [
"/"
]
}
}

In archetype_definition_es_root:
"role_definitions": [
"ES-NetOps",
"ES-SecOps"
]

First time the create of roles:

Error message:
Error: authorization.RoleDefinitionsClient#Get: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="RoleDefinitionDoesNotExist" Message="The specified role definition with ID '**' does not exist."

Re-Run(without updates to code):
Error: A resource with the ID "/providers/Microsoft.Authorization/roleDefinitions/**/providers/Microsoft.Management/managementGroups/es" already exists - to be managed via Terraform this resource needs to be imported into the State. Please see the resource documentation for "azurerm_role_definition" for more information.

Reflection of error:
Could it have something to do with it uuidv5?
#7

Do you have any suggestions on what we do wrong?

Add example page for manual overrides of Role Definitions associated with Policy Assignments using Managed Identity

Create new example page describing how to override Role Definitions.

Related to issue: #48

Missing Role Assignments for Policies using DeployIfNotExists or Modify

Module version 0.1.0

When deploying Policy Assignments using DeployIfNotExists or Modify effects, the module is able to automatically create a Role Assignment for the Managed Identity associated with the Policy Assignment.

For built-in policies, this works as expected.

In the 0.1.0 update we appear to have introduced a bug whereby Role Assisgnments for built-in roles are not created as expected.

Copy/paste error - policy_definition_es_deploy_hub.json

Hi, the displayname and description of "azfw" object in the policy_definition_es_deploy_hub.json is wrong. Probably a copy/paste error.

terraform-azurerm-caf-enterprise-scale/modules/archetypes/lib/policy_definitions/policy_definition_es_deploy_hub.json

Line 44 in fe4ad5b

"description": "Object describing ExpressRoute gateway"

Best regards!

Management module overrides set parameters for policies

Problem statement

Tested with:
Terraform version: 0.14.10 and 0.15.3
AzureRM provider version: latest (2.58.0)
Module version: 0.2.0 and 0.3.0

Some ES policies, that are applied to the root with manually specified parameters, want to set the parameters to the default value originating from the management module, even if we don't deploy the management resources from the ES module.
It doesn't matter if we apply our parameters in the PolicyAssignment, ArchetypeDefinition, or archetype_config_overrides, it still choose the settings from the management module.

A workaround that almost work all way, is to configure configure_management_resources with a mockup where you can configure the es module with this property:

configure_management_resources = {
    settings = {
      log_analytics = {
        enabled = true
        config = {
          retention_in_days                           = 30
          enable_monitoring_for_arc                   = true
          enable_monitoring_for_vm                    = true
          enable_monitoring_for_vmss                  = true
          enable_solution_for_agent_health_assessment = true
          enable_solution_for_anti_malware            = true
          enable_solution_for_azure_activity          = true
          enable_solution_for_change_tracking         = true
          enable_solution_for_service_map             = true
          enable_solution_for_sql_assessment          = true
          enable_solution_for_updates                 = true
          enable_solution_for_vm_insights             = true
          enable_sentinel                             = true
        }
      }
      security_center = {
        enabled = true
        config = {
          email_security_contact             = "[email protected]"
          enable_defender_for_acr            = true
          enable_defender_for_app_services   = true
          enable_defender_for_arm            = true
          enable_defender_for_dns            = true
          enable_defender_for_key_vault      = true
          enable_defender_for_kubernetes     = true
          enable_defender_for_servers        = true
          enable_defender_for_sql_servers    = true
          enable_defender_for_sql_server_vms = true
          enable_defender_for_storage        = true
        }
      }
    }

    location = "westeurope"
    tags     = null
    advanced = {
      existing_resource_group_name                 = "custom-rg"
      existing_log_analytics_workspace_resource_id = "/subscriptions/11111111-1111-1111-1111-111111111111/resourceGroups/custom-rg/providers/Microsoft.OperationalInsights/workspaces/custom-la"
      existing_automation_account_resource_id      = "/subscriptions/11111111-1111-1111-1111-111111111111/resourceGroups/custom-rg/providers/Microsoft.OperationalInsights/workspaces/custom-aa"
      custom_settings_by_resource_type = {
        azurerm_automation_account = {
          name = "custom-aa"
        }
        azurerm_log_analytics_workspace = {
          name = "custom-la"
        }
      }
    }
  }

But there is still 1 hardcoded parameter in the management module for the policy Deploy-ASC-Defender which is:
ascExportResourceGroupName = "${local.root_id}-asc-export"

This blocks an upgrade to the new versions, or creation of custom named resource group for that policy.

Problem reproduction

Create a ES setup with the following config:

module "enterprise_scale" {
  source  = "Azure/caf-enterprise-scale/azurerm"
  version = "0.3.0"

  root_parent_id = "e30cb998-45a0-4111-b2fd-092fed41f457"
  #library_path     = "${path.root}/lib"
  default_location = "westeurope"
  root_id          = "es-test"
  root_name        = "es-test"
}

Run a terrafrom plan with the configuration and it displays the default values.
Example from the Deploy-ASC-Defender policyAssignment:

 # module.enterprise_scale.azurerm_policy_assignment.enterprise_scale["/providers/Microsoft.Management/managementGroups/es-test/providers/Microsoft.Authorization/policyAssignments/Deploy-ASC-Defender"] will be created
  + resource "azurerm_policy_assignment" "enterprise_scale" {
      + description          = "Deploy-ASC-Defender."
      + display_name         = "Deploy-ASC-Defender"
      + enforcement_mode     = false
      + id                   = (known after apply)
      + location             = "westeurope"
      + metadata             = (known after apply)
      + name                 = "Deploy-ASC-Defender"
      + not_scopes           = []
      + parameters           = jsonencode(
            {
              + ascExportResourceGroupLocation      = {
                  + value = "westeurope"
                }
              + ascExportResourceGroupName          = {
                  + value = "es-test-asc-export"
                }
              + emailSecurityContact                = {
                  + value = "security_contact@replace_me"
                }
              + logAnalytics                        = {
                  + value = "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/es-test-mgmt/providers/Microsoft.OperationalInsights/workspaces/es-test-la"
                }

Adjust the ES module config with a archetype_config_overrides so the module config looks like this:

module "enterprise_scale" {
  source  = "Azure/caf-enterprise-scale/azurerm"
  version = "0.3.0"

  root_parent_id = "e30cb998-45a0-4111-b2fd-092fed41f457"
  #library_path     = "${path.root}/lib"
  default_location = "westeurope"
  root_id          = "es-test"
  root_name        = "es-test"

  archetype_config_overrides = {
    root = {
      archetype_id = "es_root"
      parameters = {
        Deploy-ASC-Defender = {
          emailSecurityContact                = "[email protected]"
          logAnalytics                        = "/subscriptions/11111111-1111-1111-1111-111111111111/resourceGroups/custom-rg/providers/Microsoft.OperationalInsights/workspaces/custom-la",
          ascExportResourceGroupName          = "custom-rg"
          ascExportResourceGroupLocation      = "westeurope"
          pricingTierContainerRegistry        = "Standard"
          pricingTierAppServices              = "Standard"
          pricingTierArm                      = "Standard"
          pricingTierDns                      = "Standard"
          pricingTierKeyVaults                = "Standard"
          pricingTierKubernetesService        = "Standard"
          pricingTierVMs                      = "Standard"
          pricingTierSqlServers               = "Standard"
          pricingTierSqlServerVirtualMachines = "Standard"
          pricingTierStorageAccounts          = "Standard"
        }
      }
      access_control = {}
    }
  }
}

Run a terrafrom plan with the new configuration and it still displays the default values, instead of the ones that should have been overridden of the archetype_config_overrides config in root.
Example from the Deploy-ASC-Defender policyAssignment after the override:

 # module.enterprise_scale.azurerm_policy_assignment.enterprise_scale["/providers/Microsoft.Management/managementGroups/es-test/providers/Microsoft.Authorization/policyAssignments/Deploy-ASC-Defender"] will be created
  + resource "azurerm_policy_assignment" "enterprise_scale" {
      + description          = "Deploy-ASC-Defender."
      + display_name         = "Deploy-ASC-Defender"
      + enforcement_mode     = false
      + id                   = (known after apply)
      + location             = "westeurope"
      + metadata             = (known after apply)
      + name                 = "Deploy-ASC-Defender"
      + not_scopes           = []
      + parameters           = jsonencode(
            {
              + ascExportResourceGroupLocation      = {
                  + value = "westeurope"
                }
              + ascExportResourceGroupName          = {
                  + value = "es-test-asc-export"
                }
              + emailSecurityContact                = {
                  + value = "security_contact@replace_me"
                }
              + logAnalytics                        = {
                  + value = "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/es-test-mgmt/providers/Microsoft.OperationalInsights/workspaces/es-test-la"
                }

Which is the same result and means that the archetype_config_overrides in the es module configuration is ignored.

Expected behavior

If ES variable deploy_management_resources is set to false (or not set), the archetype_config_overrides from the management module should not be applied.

We should be able to use the ES policies used in the management module and set the parameters for the policies with either PolicyAssignment, ArchetypeDefinition, or archetype_config_overrides, without the need to configure the management module.

• Deploy-ASC-Defender
• Deploy-LX-Arc-Monitoring
• Deploy-VM-Monitoring
• Deploy-VMSS-Monitoring
• Deploy-WS-Arc-Monitoring
• Deploy-AzActivity-Log
• Deploy-Resource-Diag

Log analytics retention period always set to 30

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Versions

terraform: 0.15.1

azure provider: 2.56.0

module: 0.3.1

Description

Describe the bug
When changing the value of input variable configure_management_resources.log_analytics.config.retention_in_days, the module will correctly configure the Deploy-Log-Analytics Policy Assignment, but the Log Analytics workspace remains at the default value of 30 days.

Steps to Reproduce

1. Configure a default deployment using the code in the example below, noting that the value of input variable configure_management_resources.log_analytics.config.retention_in_days has been set to 60
2. Run the deployment
3. Validate the retention settings on the Deploy-Log-Analytics Policy Assignment
4. Validate the retention settings on the Log Analytics workspace

Code Sample

data "azurerm_client_config" "current" {
}

module "enterprise_scale" {
  source  = "Azure/caf-enterprise-scale/azurerm"
  version = "0.3.1"

  root_parent_id = data.azurerm_client_config.current.tenant_id
  root_id        = "sn"
  root_name      = "Log Analytics Test"

  }

  subscription_id_overrides = {
    management = [data.azurerm_client_config.current.subscription_id]
  }

  deploy_management_resources    = true
  configure_management_resources = local.configure_management_resources
  subscription_id_management     = data.azurerm_client_config.current.subscription_id

}

locals {
  configure_management_resources = {
    settings = {
      log_analytics = {
        enabled = true
        config = {
          retention_in_days                           = 60
          enable_monitoring_for_arc                   = true
          enable_monitoring_for_vm                    = true
          enable_monitoring_for_vmss                  = true
          enable_solution_for_agent_health_assessment = true
          enable_solution_for_anti_malware            = true
          enable_solution_for_azure_activity          = true
          enable_solution_for_change_tracking         = true
          enable_solution_for_service_map             = true
          enable_solution_for_sql_assessment          = true
          enable_solution_for_updates                 = true
          enable_solution_for_vm_insights             = true
          enable_sentinel                             = true
        }
      }
      security_center = {
        enabled = true
        config = {
          email_security_contact             = "security_contact@replace_me"
          enable_defender_for_acr            = true
          enable_defender_for_app_services   = true
          enable_defender_for_arm            = true
          enable_defender_for_dns            = true
          enable_defender_for_key_vault      = true
          enable_defender_for_kubernetes     = true
          enable_defender_for_servers        = true
          enable_defender_for_sql_servers    = true
          enable_defender_for_sql_server_vms = true
          enable_defender_for_storage        = true
        }
      }
    }
    location = null
    tags     = null
    advanced = null
  }
}

Screenshots

Correctly configured retentionInDays parameter value on the Deploy-Log-Analytics Policy Assignment:

Incorrectly configured Data Retention (Days) setting on the Log Analytics workspace:

Additional context

Add capability for "Management and monitoring" resources

To provide feature parity with the native Enterprise-scale implementation options, this feature request covers the requirement to provide "Management and monitoring" resources as per the Wingtip implementation option.

Upgrade from 0.2.0 > 0.3.0 breaking on Automation Account linking

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Versions

terraform: 0.14.8

azure provider: >= 2.46.1

module: 0.3.0

Description

When upgrading from 0.2.0 to 0.3.0, I am presented with the following error:

Error: creating Linked Service 'main-la/Automation' (Resource Group "main-mgmt"): operationalinsights.LinkedServicesClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: Code="BadRequest" Message="Automation account not found"

  on .terraform/modules/custom_root_id/resources.management.tf line 75, in resource "azurerm_log_analytics_linked_service" "enterprise_scale":
  75: resource "azurerm_log_analytics_linked_service" "enterprise_scale" {

The deployment fails when trying to delete the automation account and fails to unlink it from the workspace....preventing the new automation account from getting linked to the workspace.

1. Deploy module version 0.2.0
2. Deploy module version 0.3.0

Example code here

Cannot deploy module without core landing zones

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Versions

terraform: 15.4

azure provider: >=2.41.0

module: 0.3.1

Description

Describe the bug

I cannot seem to deploy the module using just custom_landing_zones config block and /lib folder for custom archetype definitions. Terraform plan throws an error saying that policy (set) definition could not be found.

I'm under impression that

Steps to Reproduce

1. Make sure there are no previous policy definitions deployed by this module before trying to reproduce
2. Set deploy_core_landing_zones = false
3. Create /lib folder
4. Create a file "archetype_definition_custom_root.json" and paste in the skeleton/empty archetype definition
5. Add for example all the contents of built-in es_root archetype
6. Rename the archetype from "default_empty": {... --> "es_custom": {...
7. Create one custom landing zone (with any naming) below the Tenant Root Group
8. Use es_custom archetype as archetype_id in the custom landing zone

Terraform plan should result in error stating that the policy definitions are not found and refers to the locals.policy_assignments.tf file in the "external_lookup" section(s) at line(s) 90 and/or 152.

Screenshots

Additional context
Use case: deploy totally custom management group hierarchy for example in case of Company Group + subsidiaries. Like to have all the subsidiaries and the main group at the same management group level and then build custom structure under those companies.

Also would like to define custom archetypes to use with the custom landing zones by defining them in the lib folder and to copy all the existing policy definitions from the es_root to my "custom_root" archetype definition.

I'd like to understand if this use case is achievable with this module. As MSP I'd like to see the module logic to perform separately from the customer specific configurations so that I can have all the customers using your module but with customer specific custom management groups structures and custom archetypes and not to deply the core landing zones at all.

Custom landing zone variable validation only checks first key

The validation of the custom_landing_zone keys only checks the value of the first key. This makes it possible to set invalid key names (which fails later).

Example inputs:

# Passes validation (and fails during archetype variable validation)
custom_landing_zones = {
  (var.root_id) = {
    display_name               = var.root_name
    parent_management_group_id = data.azurerm_client_config.current.tenant_id
    subscription_ids           = []
    archetype_config = {
      archetype_id   = "es_root"
      parameters     = {}
      access_control = {}
    }
  }

  test_key = {
    display_name               = "Test"
    parent_management_group_id = var.root_id
    subscription_ids           = []
    archetype_config = {
      archetype_id   = "es_landing_zones"
      parameters     = {}
      access_control = {}
    }
  }
}

# Does not pass validation 
custom_landing_zones = {
  test_root = {
    display_name               = var.root_name
    parent_management_group_id = data.azurerm_client_config.current.tenant_id
    subscription_ids           = []
    archetype_config = {
      archetype_id   = "es_root"
      parameters     = {}
      access_control = {}
    }
  }
}

Error: reading Policy Set Definition "1f3afdf9-d0c9-4c3d-847f-89da613e70a8"

Hi,

No custom inputs but erroring on a particular initiative:

"Enable Monitoring in Azure Security Center"
"/providers/Microsoft.Authorization/policySetDefinitions/1f3afdf9-d0c9-4c3d-847f-89da613e70a8"

Thanks
Dan

ES-Deny-Subnets-Without-NSG

The following policy does not exclude subnets that do not allow NSGs to be attached to it.

ES-Deny-Subnets-Without-NSG

So far, I know GatewaySubnet, AzureFirewallSubnet, and AzureFirewallManagementSubnet do not support NSG. There may be others...

Proposing the policy rule's "if" condition look like this:

"if": {
        "allOf": [
          {
            "field": "type",
            "equals": "Microsoft.Network/virtualNetworks/subnets"
          },
          {
            "field": "name",
            "notEquals": "AzureFirewallSubnet"
          },
          {
            "field": "name",
            "notEquals": "AzureFirewallManagementSubnet"
          },
          {
            "field": "name",
            "notEquals": "GatewaySubnet"
          },
          {
            "field": "Microsoft.Network/virtualNetworks/subnets/networkSecurityGroup.id",
            "exists": "false"
          }
        ]
      },

Invalid index error

I am trying to deploy the customized module and keep getting following error message:

`
64: archetype_definition = local.archetype_definitions[local.archetype_id]
|----------------
| local.archetype_definitions is object with 13 attributes
| local.archetype_id is "customer_online"

The given key does not identify an element in this collection value.
The custom landing zone declaration looks like this:
#------------------------------------------------------#
customer-web-prod = {
display_name = "Prod Web Applications"
parent_management_group_id = "cirr-landing-zones"
subscription_ids = []
archetype_config = {
archetype_id = "customer_online"
parameters = {}
access_control = {}
}
}
`

Can anyone help me solve this?

Thank you!

The given value is not suitable for child module variable "archetype_config_overrides"

Problem statement

When defining settings for multiple core landing zones within the module variable "archetype_config_overrides", if these are not of an identical object type, we observe the error:

The given value is not suitable for child module variable "archetype_config_overrides"

Terraform version: 0.15.0
AzureRM provider version: 2.56.0
Module version: 0.3.0

Steps to reproduce

The following code sample will re-produce this issue.

data "azurerm_client_config" "current" {
}

module "enterprise_scale" {
  source  = "Azure/caf-enterprise-scale/azurerm"
  version = "0.3.0"

  root_parent_id = data.azurerm_client_config.current.tenant_id
  root_id        = "es"
  root_name      = "ES Test Deployment"
  library_path   = "${path.root}/lib"

  archetype_config_overrides = {
    root = {
      archetype_id = "es_root"
      parameters = {
        Deploy-SQL-Auditing = {
          retentionDays                = "10"
          storageAccountsResourceGroup = ""
        }
        Deploy-HITRUST-HIPAA = {
          CertificateThumbprints                                        = ""
          DeployDiagnosticSettingsforNetworkSecurityGroupsrgName        = "${local.custom_root_id}-rg"
          DeployDiagnosticSettingsforNetworkSecurityGroupsstoragePrefix = local.custom_root_id
          installedApplicationsOnWindowsVM                              = ""
          listOfLocations = [
            "eastus"
          ]
        }
      }
      access_control = {}
    }
    landing-zones = {
      archetype_id = "es_landing_zones"
      parameters = {
        Deny-Priv-Containers-AKS = {
          effect = "audit"
          excludedNamespaces = [
            "kube-system",
            "gatekeeper-system",
            "azure-arc",
          ]
          namespaces = []
        }
      }
      access_control = {}
    }
  }

}

Removing either root or landing-zones object from archetype_config_overrides resolves the issue.

Error message

╷
│ Error: Invalid value for module argument
│
│   on enterprise_scale.tf line 15, in module "enterprise_scale":
│   15:   archetype_config_overrides = {
│   16:     root = {
│   17:       archetype_id = "es_root"
│   18:       parameters = {
│   19:         Deploy-SQL-Auditing = {
│   20:           retentionDays                = "10"
│   21:           storageAccountsResourceGroup = ""
│   22:         }
│   23:         Deploy-HITRUST-HIPAA = {
│   24:           CertificateThumbprints                                        = ""
│   25:           DeployDiagnosticSettingsforNetworkSecurityGroupsrgName        = "${local.custom_root_id}-rg"
│   26:           DeployDiagnosticSettingsforNetworkSecurityGroupsstoragePrefix = local.custom_root_id
│   27:           installedApplicationsOnWindowsVM                              = ""
│   28:         }
│   29:       }
│   30:       access_control = {}
│   31:     }
│   32:     landing-zones = {
│   33:       archetype_id = "es_landing_zones"
│   34:       parameters = {
│   35:         Deny-Priv-Containers-AKS = {
│   36:           effect = "audit"
│   37:           excludedNamespaces = [
│   38:             "kube-system",
│   39:             "gatekeeper-system",
│   40:             "azure-arc",
│   41:           ]
│   42:           namespaces = []
│   43:         }
│   44:       }
│   45:       access_control = {}
│   46:     }
│   47:   }
│
│ The given value is not suitable for child module variable "archetype_config_overrides" defined at .terraform\modules\enterprise_scale\variables.tf:143,1-38: cannot find a common base type for all elements.
╵

When deploying management resources, log analytics workspace is deployed in the wrong subscription

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Versions

• terraform: 0.14.10
• azure provider: hashicorp/azurerm v2.58.0
• module: 0.3.1

Description

As the title says, when deploying the management resources, the log analytics workspace (and maybe the security center aswell) may be deployed in the wrong subscription depending on which is client's active subscription (az account show).

We noticed this when trying to deploy management resources for the first time, and what actually happened is that the module/terraform was using the azure's current active subscription instead of the one defined in the subscription_id_management section - you can get the active subscription by using the following command: az account show.

No matter what value we set in the subscription_id_management, the terraform always ended deploying the log analytics workspace under the subscription the current client's context was set to.

The only workaround to ensure that the resources were deployed in the correct subscription was to set the current active subscription to the same configured in the subscription_id_management (az account set --subscription="<subscription_id_management>")

Even after using the above workaround, you still need to keep the same active subscription, otherwise terraform will try deploy resources to the current active subscription.

Steps to Reproduce

1. Ensure you have at least two subscriptions to work with (they should be on the same tenant).
2. Set subscription_id_management to the subscription that you want to deploy your management resources.
3. Set your active subscription to a different subscription az account set --subscription="some-other-subscription"
4. Terraform plan & apply and the deploy will fail
5. Search through the azure portal and lookup where the log analytics workspace was deployed and confirm that it was deployed in the account's active subscription instead of the one defined in the subscription_id_management

Error while assigning a Built-in Policy Initiative using SystemAssigned Identity

Our client would like the PCI v3.2.1:2018 Policy Initiative (id: /providers/Microsoft.Authorization/policySetDefinitions/496eeda9-8f2f-4d5e-8dfd-204f0a92ed41) assigned to their subscriptions, however when attempting this we get the following error:

Error: Duplicate object key

  on .terraform/modules/enterprise_scale/locals.policy_assignments.tf line 74, in locals:
  72:   azurerm_policy_set_definition_external_lookup = {
  73:     for policy_set_definition_id in local.policy_assignments_with_managed_identity_using_external_policy_set_definition :
  74:     policy_set_definition_id => {
  75:       name                  = basename(policy_set_definition_id)
  76:       management_group_name = try(regex(local.regex_extract_provider_scope, policy_set_definition_id), null)
  77:     }
  78:   }

Two different items produced the key
"/providers/Microsoft.Authorization/policySetDefinitions/496eeda9-8f2f-4d5e-8dfd-204f0a92ed41"
in this 'for' expression. If duplicates are expected, use the ellipsis (...)
after the value expression to enable grouping by key.

Policy Assignment document looks like this:

{
  "name": "PCI-Benchmark",
  "type": "Microsoft.Authorization/policyAssignments",
  "apiVersion": "2019-09-01",
  "properties": {
    "description": "Enables PCI benchmark initative.",
    "displayName": "PCI-Benchmark",
    "notScopes": [],
    "parameters": {},
    "policyDefinitionId": "/providers/Microsoft.Authorization/policySetDefinitions/496eeda9-8f2f-4d5e-8dfd-204f0a92ed41",
    "scope": "${current_scope_resource_id}",
    "enforcementMode": false
  },
  "location": "${default_location}",
  "identity": {
    "type": "SystemAssigned"
  }
}

Not quite sure what I'm doing wrong here, any help is appreciated.

When deploying management resources, Security Center resources are not being deployed

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Versions

• terraform: 0.14.10
• azure provider: hashicorp/azurerm v2.58.0
• module: 0.3.1

Description

When reconfiguring the configured_management_resources section to enable security_center (after a successful plan and apply with only log_analytics enabled), terraform will not detect any changes and no security center resources will be deployed.

This is the first deployed management resources:

• notice the log_analytics=true and security_center=false

configure_management_resources = {
    settings = {
      log_analytics = {
        enabled = true
        config = {
          retention_in_days                           = 60
          enable_monitoring_for_arc                   = true
          enable_monitoring_for_vm                    = true
          enable_monitoring_for_vmss                  = true
          enable_solution_for_agent_health_assessment = true
          enable_solution_for_anti_malware            = false
          enable_solution_for_azure_activity          = true
          enable_solution_for_change_tracking         = true
          enable_solution_for_service_map             = true
          enable_solution_for_sql_assessment          = false
          enable_solution_for_updates                 = true
          enable_solution_for_vm_insights             = true
          enable_sentinel                             = false
        }
      }
      security_center = {
        enabled = false
        config = {
          email_security_contact             = "[email protected]"
          enable_defender_for_acr            = false
          enable_defender_for_app_services   = false
          enable_defender_for_arm            = false
          enable_defender_for_dns            = false
          enable_defender_for_key_vault      = true
          enable_defender_for_kubernetes     = false
          enable_defender_for_servers        = false
          enable_defender_for_sql_servers    = false
          enable_defender_for_sql_server_vms = false
          enable_defender_for_storage        = false
        }
      }
    }

    location = "westeurope"
    tags     = null
    advanced = null
  }

After changing the configuration to also deploy the security center, the following results in "no changes" by Terraform.

• notice the log_analytics=true and security_center=true

configure_management_resources = {
    settings = {
      log_analytics = {
        enabled = true
        config = {
          retention_in_days                           = 60
          enable_monitoring_for_arc                   = true
          enable_monitoring_for_vm                    = true
          enable_monitoring_for_vmss                  = true
          enable_solution_for_agent_health_assessment = true
          enable_solution_for_anti_malware            = false
          enable_solution_for_azure_activity          = true
          enable_solution_for_change_tracking         = true
          enable_solution_for_service_map             = true
          enable_solution_for_sql_assessment          = false
          enable_solution_for_updates                 = true
          enable_solution_for_vm_insights             = true
          enable_sentinel                             = false
        }
      }
      security_center = {
        enabled = true
        config = {
          email_security_contact             = "[email protected]"
          enable_defender_for_acr            = false
          enable_defender_for_app_services   = false
          enable_defender_for_arm            = false
          enable_defender_for_dns            = false
          enable_defender_for_key_vault      = true
          enable_defender_for_kubernetes     = false
          enable_defender_for_servers        = false
          enable_defender_for_sql_servers    = false
          enable_defender_for_sql_server_vms = false
          enable_defender_for_storage        = false
        }
      }
    }

    location = "westeurope"
    tags     = null
    advanced = null
  }

Steps to Reproduce

1. Create the management resources for the first time, with log_analytics enabled and security_center disabled.
2. The plan and apply should go smoothly and only the log_analytics resources should be created as expected.
3. Now reconfigure the configured_management_resources section and set security_center.enabled = true, followed by terraform plan
4. Terraform will report that infrastructure is up-to-date, but it should have planned changes to deploy the security_center resources.

module.enterprise_scale.azurerm_role_assignment.enterprise_scale["<redacted>"] must be replaced

Problem statement

When making changes to a single Policy Assignment, all Role Assignments associated with Managed Identities used by Policy Assignments with deployIfNotExists or modify effects are destroyed and then re-created even when no change is necessary.

Terraform version: 0.15.0
AzureRM provider version: 2.56.0
Module version: 0.3.0

Steps to reproduce

Start with a simple deployment of Enterprise-scale:

data "azurerm_client_config" "current" {
}

module "enterprise_scale" {
  source  = "Azure/caf-enterprise-scale/azurerm"
  version = "0.3.0"

  root_parent_id = data.azurerm_client_config.current.tenant_id
  root_id        = "es"
  root_name      = "ES Test Deployment"

}

Update the configuration to change settings on a single Policy Assignment, add a new Policy Assignment, or remove an existing Policy Assignment.

In the following example, we set an override to change the effect parameter value for the Deny-Priv-Containers-AKS Policy Assignment.

data "azurerm_client_config" "current" {
}

module "enterprise_scale" {
  source  = "Azure/caf-enterprise-scale/azurerm"
  version = "0.3.0"

  root_parent_id = data.azurerm_client_config.current.tenant_id
  root_id        = "es"
  root_name      = "ES Test Deployment"

  archetype_config_overrides = {
    landing-zones = {
      archetype_id = "es_landing_zones"
      parameters = {
        Deny-Priv-Containers-AKS = {
          effect = "audit"
        }
      }
      access_control = {}
    }
  }

}

Note that the planned changes include both the expected update to the changed Policy Assignment and the re-creation of all Role Assignments associated with Policy Assignments...

Planned changes...

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # module.enterprise_scale.azurerm_policy_assignment.enterprise_scale["/providers/Microsoft.Management/managementGroups/sn-landing-zones/providers/Microsoft.Authorization/policyAssignments/Deny-Priv-Containers-AKS"] must be replaced
-/+ resource "azurerm_policy_assignment" "enterprise_scale" {
      ~ id                   = "/providers/Microsoft.Management/managementGroups/sn-landing-zones/providers/Microsoft.Authorization/policyAssignments/Deny-Priv-Containers-AKS" -> (known after apply)
      ~ metadata             = jsonencode(
            {
              - createdBy = "daa804f2-68e3-4826-8914-ca62f6eff0ce"
              - createdOn = "2021-05-07T11:20:37.0553423Z"
              - updatedBy = null
              - updatedOn = null
            }
        ) -> (known after apply)
        name                 = "Deny-Priv-Containers-AKS"
      ~ parameters           = jsonencode(
          ~ {
              ~ effect = {
                  ~ value = "deny" -> "audit"
                }
            } # forces replacement
        )
        # (7 unchanged attributes hidden)

      ~ identity {
          + principal_id = (known after apply)
          + tenant_id    = (known after apply)
            # (1 unchanged attribute hidden)
        }
    }

  # module.enterprise_scale.azurerm_role_assignment.enterprise_scale["/providers/Microsoft.Management/managementGroups/sn-landing-zones/providers/Microsoft.Authorization/roleAssignments/13c199bb-84e0-5bcf-afe4-5f80b47ef1cf"] must be replaced
-/+ resource "azurerm_role_assignment" "enterprise_scale" {
      ~ id                               = "/providers/Microsoft.Management/managementGroups/sn-landing-zones/providers/Microsoft.Authorization/roleAssignments/13c199bb-84e0-5bcf-afe4-5f80b47ef1cf" -> (known after apply)
        name                             = "13c199bb-84e0-5bcf-afe4-5f80b47ef1cf"
      ~ principal_id                     = "b79de19e-812e-49ce-bedd-b9d6cc314977" -> (known after apply) # forces replacement
      ~ principal_type                   = "ServicePrincipal" -> (known after apply)
      ~ role_definition_name             = "SQL DB Contributor" -> (known after apply)
      + skip_service_principal_aad_check = (known after apply)
        # (2 unchanged attributes hidden)
    }

  <redacted for brevity>

  # module.enterprise_scale.azurerm_role_assignment.enterprise_scale["/providers/Microsoft.Management/managementGroups/sn/providers/Microsoft.Authorization/roleAssignments/f92a085a-bd45-5d98-8da5-a214c1e6d9cd"] must be replaced
-/+ resource "azurerm_role_assignment" "enterprise_scale" {
      ~ id                               = "/providers/Microsoft.Management/managementGroups/sn/providers/Microsoft.Authorization/roleAssignments/f92a085a-bd45-5d98-8da5-a214c1e6d9cd" -> (known after apply)       
        name                             = "f92a085a-bd45-5d98-8da5-a214c1e6d9cd"
      ~ principal_id                     = "d0933104-f0c7-446b-8b78-bd884c6afcc6" -> (known after apply) # forces replacement
      ~ principal_type                   = "ServicePrincipal" -> (known after apply)
      ~ role_definition_name             = "Contributor" -> (known after apply)
      + skip_service_principal_aad_check = (known after apply)
        # (2 unchanged attributes hidden)
    }

Plan: 19 to add, 0 to change, 19 to destroy.

Having problem with template validation on ES-Deploy-LogAnalytics

I try to deploy an LOA with the policy ES-Deploy-LogAnalytics.

But i get an validation problem for the deployment part.

I had to delete:
"name": "Automation",
"type": "linkedServices",
...
..

After conversations with MS, we have realized that this policy may not exist. But we will script LOA with terraform because we need to have control of its state.

BR
Magnus

"apiVersion": "2017-03-15-preview",
"location": "[parameters('workspaceRegion')]",
"name": "[parameters('workspaceName')]",
"type": "Microsoft.OperationalInsights/workspaces",
"properties": {
"sku": {
"name": "pernode"
},
"enableLogAccessUsingOnlyResourcePermissions": true
},
"resources": [
{
"name": "Automation",
"type": "linkedServices",
"apiVersion": "2015-11-01-preview",
"dependsOn": [
"[resourceId('Microsoft.OperationalInsights/workspaces/', parameters('workspaceName'))]",
"[resourceId('Microsoft.Automation/automationAccounts/', parameters('AutomationAccountName'))]"
],
"properties": {
"resourceId": "[concat(subscription().id, '/resourceGroups/', parameters('rgName'), '/providers/Microsoft.Automation/automationAccounts/', parameters('AutomationAccountName'))]"
}
}
]

I would be happy to see an example - policy & access_control

Hello Kevin,
first of all many thanks for your effort with the repro :-)

Would you have an example for me, I can find the right values in the portal.azure.... :

 parameters = {
     policy_assignment_id = {
     param_name_1 = param_value_1

    }
     }
    access_control = {
    role_definition_name = {
    "member_1_object_id"
    }

I would also be happy to write the wiki article following this?