two service connections but reads backend data incorrectly

[mericstam]

two service connections but reads backend data incorrectly

[mericstam]

taking a look at this.

[mericstam]

@theo-albers

[mericstam]

Hi @theo-albers, I did have an issue in my setup and thought it was related, but my issue was I had two types of service connections. when I switched them to same type (Service Principal) it worked.. this is my setup:

 - task: TerraformTaskV4@4
      displayName: 'terraform init'
      inputs:
        provider: 'azurerm'
        command: 'init'
        backendServiceArm: 'spMSDN2'
        backendAzureRmResourceGroupName: 'terraform'
        backendAzureRmStorageAccountName: 'solstatestore2'
        backendAzureRmContainerName: 'state'
        backendAzureRmKey: 'state.tfstate'
 
- task: TerraformTaskV4@4
      name: plan
      displayName: 'terraform plan'
      inputs:
        commandOptions: '-out=tfplan -detailed-exitcode'
        provider: 'azurerm'
        command: 'plan'
        environmentServiceNameAzureRM: 'TerraformSP'

parts of debug log

for the init
....
##[debug]found: '/opt/hostedtoolcache/terraform/1.5.7/x64/terraform'
##[debug]which '/opt/hostedtoolcache/terraform/1.5.7/x64/terraform'
##[debug]found: '/opt/hostedtoolcache/terraform/1.5.7/x64/terraform'
##[debug]/opt/hostedtoolcache/terraform/1.5.7/x64/terraform arg: init
##[debug]backendServiceArm=5cd4e6b5-xxxx-479e-beed-xxxxxxxxx
##[debug]backendServiceArm=5cd4e6b5-xxxx-479e-beed-xxxxxxxxx
##[debug]5cd4e6b5-xxxx-479e-beed-xxxxxxxxx auth scheme = ServicePrincipal
##[debug]Setting up backend for authorization scheme: serviceprincipal.
##[debug]backendAzureRmStorageAccountName=solstatestore2
##[debug]backendAzureRmContainerName=state
##[debug]backendAzureRmKey=state.tfstate
##[debug]backendAzureRmResourceGroupName=terraform
.....





for the plan
....
##[debug]environmentServiceNameAzureRM=1c2ad8ed-xxxx-4d31-91b1-xxxxxxxxx
##[debug]1c2ad8ed-xxxx-4d31-91b1-xxxxxxxxx auth scheme = ServicePrincipal
##[debug]Setting up provider for authorization scheme: serviceprincipal.
...
...

##[debug]1c2ad8ed-xxxx-4d31-91b1-xxxxxxxxx auth param serviceprincipalid = ***
##[debug]1c2ad8ed-xxxx-4d31-91b1-xxxxxxxxx auth param serviceprincipalkey = ***
##[debug]Finished up provider for authorization scheme: serviceprincipal.
##[debug]exec tool: /opt/hostedtoolcache/terraform/1.5.7/x64/terraform
##[debug]arguments:
##[debug]   plan
##[debug]   -out=tfplan
##[debug]   -var
##[debug]   test=test
##[debug]   -detailed-exitcode

As a test can you hardcode the names of your service connections so you can rule out any variable configuration?

[theo-albers]

Hi @theo-albers, I did have an issue in my setup and thought it was related, but my issue was I had two types of service connections. when I switched them to same type (Service Principal) it worked.. this is my setup:

 - task: TerraformTaskV4@4
      displayName: 'terraform init'
      inputs:
        provider: 'azurerm'
        command: 'init'
        backendServiceArm: 'spMSDN2'
        backendAzureRmResourceGroupName: 'terraform'
        backendAzureRmStorageAccountName: 'solstatestore2'
        backendAzureRmContainerName: 'state'
        backendAzureRmKey: 'state.tfstate'
 
- task: TerraformTaskV4@4
      name: plan
      displayName: 'terraform plan'
      inputs:
        commandOptions: '-out=tfplan -detailed-exitcode'
        provider: 'azurerm'
        command: 'plan'
        environmentServiceNameAzureRM: 'TerraformSP'

parts of debug log

for the init
....
##[debug]found: '/opt/hostedtoolcache/terraform/1.5.7/x64/terraform'
##[debug]which '/opt/hostedtoolcache/terraform/1.5.7/x64/terraform'
##[debug]found: '/opt/hostedtoolcache/terraform/1.5.7/x64/terraform'
##[debug]/opt/hostedtoolcache/terraform/1.5.7/x64/terraform arg: init
##[debug]backendServiceArm=5cd4e6b5-xxxx-479e-beed-xxxxxxxxx
##[debug]backendServiceArm=5cd4e6b5-xxxx-479e-beed-xxxxxxxxx
##[debug]5cd4e6b5-xxxx-479e-beed-xxxxxxxxx auth scheme = ServicePrincipal
##[debug]Setting up backend for authorization scheme: serviceprincipal.
##[debug]backendAzureRmStorageAccountName=solstatestore2
##[debug]backendAzureRmContainerName=state
##[debug]backendAzureRmKey=state.tfstate
##[debug]backendAzureRmResourceGroupName=terraform
.....





for the plan
....
##[debug]environmentServiceNameAzureRM=1c2ad8ed-xxxx-4d31-91b1-xxxxxxxxx
##[debug]1c2ad8ed-xxxx-4d31-91b1-xxxxxxxxx auth scheme = ServicePrincipal
##[debug]Setting up provider for authorization scheme: serviceprincipal.
...
...

##[debug]1c2ad8ed-xxxx-4d31-91b1-xxxxxxxxx auth param serviceprincipalid = ***
##[debug]1c2ad8ed-xxxx-4d31-91b1-xxxxxxxxx auth param serviceprincipalkey = ***
##[debug]Finished up provider for authorization scheme: serviceprincipal.
##[debug]exec tool: /opt/hostedtoolcache/terraform/1.5.7/x64/terraform
##[debug]arguments:
##[debug]   plan
##[debug]   -out=tfplan
##[debug]   -var
##[debug]   test=test
##[debug]   -detailed-exitcode

As a test can you hardcode the names of your service connections so you can rule out any variable configuration?

I already verified the variable replacement via a temp debug pipeline. All variables were replaced properly and the service connections worked fine as well using an Azure Powershell task as test. Both service connections are Azure Resource Manager automatic connections with only the subscription filled in.

[theo-albers]

I have a Powershell script where I call terraform directly, locally. I added a secret to both App Registrations the service connections point to. When I do this, my Powershell script calling terraform init and plan works fine. For init I set the background SP and for plan the deployment SP. This confirms to me the issue is related to the pipeline task.

terraform init `
  -var-file="$($TerraformVariablesFile)" `
  -backend-config="tenant_id=$($WorkspaceCredentials.TenantId)" `
  -backend-config="subscription_id=$($WorkspaceCredentials.SubscriptionId)" `
  -backend-config="client_id=$($WorkspaceCredentials.ClientId)" `
  -backend-config="client_secret=$($WorkspaceCredentials.ClientSecret)" `
  -backend-config="resource_group_name=$($TerraformResourceGroupName)" `
  -backend-config="storage_account_name=$($TerraformStorageAccountName)" `
  -backend-config="container_name=$($TerraformContainerName)" `
  -backend-config="key=$($TerraformStateFile)" 
  
$env:ARM_TENANT_ID       = $DeploymentCredentials.TenantId
$env:ARM_SUBSCRIPTION_ID = $DeploymentCredentials.SubscriptionId
$env:ARM_CLIENT_ID       = $DeploymentCredentials.ClientId
$env:ARM_CLIENT_SECRET   = $DeploymentCredentials.ClientSecret                  
terraform plan -var-file="$($TerraformVariablesFile)" -out="$($TerraformPlanFile)" -detailed-exitcode *>&1 | Out-Host

[mericstam]

I cannot replicate the issue you are seeing with the same id for both backend and environment service connections.

I will see if I can replicate your setup better. you have two subscriptions with the same tenant, correct?

br
Manuel

[theo-albers]

I cannot replicate the issue you are seeing with the same id for both backend and environment service connections.

I will see if I can replicate your setup better. you have two subscriptions with the same tenant, correct?

br Manuel

Correct. The setup:

• Azure DevOps is tenant A
• Terraform backend service connection
  • tenant B, subscription A
• Deployment service connection
  • tenant B, subscription B
• I'm invited as guest in tenant B, belonging to tenant A

I don't see in this repo where the environment principal is resolved. I only see that it uses the property to detect the authentication schema. I don't see it actually uses the environment service connection to resolve the client/secret, but maybe I'm wrong. I also see no sign in log for this service connection in Azure DevOps under the service connection.

We are using a yaml pipeline, not a classic pipeline. Maybe your tests assume variables to be set by the pipeline infrastructure and maybe that's not the case with the yaml pipeline. Could there be a difference between classic and yaml pipeline?

[mericstam]

Hi, Unfortunately I can not replicate your setup as I do not have a way to create two subs in my tenants. (one is Visual studio subscription and the other I do not have high enough rights to create a new subscription)

I also use YAML pipelines. the two differences I see in my setup: 1. I have two tenants with one subscription in each. 2. I don't use variables for service connection names.

backendServiceArm: 'spMSDN2' and environmentServiceNameAzureRM: 'TerraformSP'

[theo-albers]

For me it's no longer an issue. I took the easy way out and I am targeting a single subscription.

[rruenroeng]

Hi there!

I was seeing a similar problem on a separate thread. I would really like to target a separate subscription. It is important for my organization to have our sensitive information (the tf state) on a different subscription that we can really limit access to.
I've taken the easy way out as a stopgap, but this is still an issue I'd love to see resolved.

[theo-albers]

Hi, Unfortunately I can not replicate your setup as I do not have a way to create two subs in my tenants. (one is Visual studio subscription and the other I do not have high enough rights to create a new subscription)

I also use YAML pipelines. the two differences I see in my setup: 1. I have two tenants with one subscription in each. 2. I don't use variables for service connection names.

backendServiceArm: 'spMSDN2' and environmentServiceNameAzureRM: 'TerraformSP'

When I look at the source code in this repo I don't see that the subscription and tenant id is being set based on environmentServiceNameAzureRM. Can you at least verify this, just by debugging the task? Where does it take the connection information from environmentServiceNameAzureRM?

[theo-albers]

We can confirm that the issue occurs when you have two subscriptions in a single tenant. Tenant A with subscription A for state and subscription B for deployment fails. It works when you have tenant A with subscription A for state and tenant B with subscription B for deployment.

We have a pipeline with the scenario that fails and one with the scenario that succeeds. Both pipelines use the same tasks. The failing pipeline succeeds for the stage where state and deployment use the same subscription.

[mericstam]

Hi, Unfortunately I can not replicate your setup as I do not have a way to create two subs in my tenants. (one is Visual studio subscription and the other I do not have high enough rights to create a new subscription)
I also use YAML pipelines. the two differences I see in my setup: 1. I have two tenants with one subscription in each. 2. I don't use variables for service connection names.
backendServiceArm: 'spMSDN2' and environmentServiceNameAzureRM: 'TerraformSP'

When I look at the source code in this repo I don't see that the subscription and tenant id is being set based on environmentServiceNameAzureRM. Can you at least verify this, just by debugging the task? Where does it take the connection information from environmentServiceNameAzureRM?

it is done here: https://github.com/microsoft/azure-pipelines-terraform/blob/ff6cc825dbaf72c902d7e198ef8ee7a9b604e267/Tasks/TerraformTask/TerraformTaskV4/src/azure-terraform-command-handler.ts#L62C42-L62C82

[theo-albers]

Yes, when I follow the flow, it should indeed work as intended. Strange....

How do you debug pipeline tasks? When I read this https://learn.microsoft.com/en-us/azure/devops/extend/develop/add-build-task?view=azure-devops I guess you are basically left with the mocha tests.

[mericstam]

Hi, it is pretty complex to debug but possible, however I have not been successful. I use console.log() a lot in the not public dev version of the task.

from the MS Learn docs
https://learn.microsoft.com/en-us/azure/devops/extend/develop/add-build-task?view=azure-devops#run-the-task

docs from azure-pipelines-tasks
https://github.com/microsoft/azure-pipelines-tasks/blob/master/docs/debugging.md#debugging-typescript-tasks-in-vs-code

not sure below works for bulid extensions, but should work for UI stuff.
using hot reload
https://github.com/microsoft/azure-devops-extension-hot-reload-and-debug