Az module cmdlets fail when ACME-PS module loaded

[sergethedev17]

Az module cmdlets fail when ACME-PS module loaded

[glatzert]

Also I implemented a small fix for the type-conversion (e.g string to state and string to identifier), which might also impact this issue and will be available in the next version somewhen this week.

[glatzert]

I've seen this error before, when I first introduced the automatic type converters, which were not very happy about being used with Powershell-classes. Can you try to explicitly load the ACME-PS module by calling import-module as a workaround? I don't have a runbook available to do the necessary tests currently.

[sergethedev17]

I tried explicitly loading the ACME-PS Module (eg Import-Module 'ACME-PS' -Force). This did not help.

There are 2 other relevant issues:

1. Removing the ACME-PS module (eg Remove-Module 'ACME-PS') after it is used does not immediately resolve the issue.
2. Commenting out all ACME-xxx cmdlets does not immediately resolve the issue.

Steps to reproduce:

1. Execute unmodified Runbook script, as per my first comment.
2. Edit script and comment out the $state = New-ACMEState -Path $stateDir line.
3. Publish updated script.
4. Execute updated script.

Excepted behaviour:
Script throws exception at step 1.
Script does not throw exception at step 4.

Actual behaviour:
Script throws exception at both steps 1 and 4.

Note however that above cannot be always reproduced. Sometimes it is necessary to follow the Steps to reproduce two or more times.

[glatzert]

My knowledge about Runbooks is very limited, nevertheless I can tell you, that the error occurs during module load of ACME-PS (that's the only location the type xml file is used).
Removing and adding the module again might do harm here, since ACME-PS makes heavy usage of powershell classes, which are - to me knowledge - not completely cleaned up during removal of a module. So if you once loaded the module in your powershell session, you should neither remove it nor import it again with using -force - that both will make an possibly undefined state for the classes.

[fmartin1987]

Hi, I was wondering if this issue is still pending because I am going exactly through the same problems...

I have published a runbook in an Azure automation account in which two different parts are being executed:

1. First one, it connects to LetsEncrypt and obtains a challenge token and its data content (Got from here).

    $state = New-ACMEState -Path $env:TEMP
    $serviceName = 'LetsEncrypt'

    # Fetch the service directory and save it in the state
    Get-ACMEServiceDirectory $state -ServiceName $serviceName -PassThru;

    # Get the first anti-replay nonce
    New-ACMENonce $state;

    # Create an account key. The state will make sure it's stored.
    New-ACMEAccountKey $state -PassThru;

    # Register the account key with the acme service. The account key will automatically be read from the state
    New-ACMEAccount $state -EmailAddresses $EmailAddress -AcceptTOS;

    # Load an state object to have service directory and account keys available
    $state = Get-ACMEState -Path $env:TEMP;

    # It might be neccessary to acquire a new nonce, so we'll just do it for the sake of the example.
    New-ACMENonce $state -PassThru;

    # Create the identifier for the DNS name
    $identifier = New-ACMEIdentifier $domain;

    # Create the order object at the ACME service.
    $order = New-ACMEOrder $state -Identifiers $identifier;

    # Fetch the authorizations for that order
    $authZ = Get-ACMEAuthorization -State $state -Order $order;

    # Select a challenge to fullfill
    $challenge = Get-ACMEChallenge $state $authZ "http-01";

    # Inspect the challenge data
    $challenge.Data;

2. Then, another function tries to write that token file (with that content) in a storage account

    try
    {
        # Create new file
        New-Item -ItemType File -Name "$token"

        # Insert data into it
        Set-Content -Path "$token" -Value "$tokenData" -NoNewline;

        # Set blob path
        $blobName = ".well-known/acme-challenge/" + "$token"

        # Get key to storage account
        $acctKey = (Get-AzStorageAccountKey -Name $storageName -ResourceGroupName $STResourceGroupName).Value[0]
        
        # Map to the reports BLOB context
        $storageContext = New-AzStorageContext -StorageAccountName $storageName -StorageAccountKey $acctKey
        
        # Copy the file to the storage account
        Set-AzStorageBlobContent -File "$token" -Container "public" -BlobType "Block" -Blob $blobName -Context $storageContext -Force -Verbose
    }
    catch {
        Write-Error -Message $_.Exception
        throw $_.Exception
    }
}

Later more stuff should be done but that is out of the scope. The thing is:

• If I execute only the first part (the one that makes usage of ACME-PS) the powershell script ends ok.

• If I skip the first part and I invoke the write_challenge_token function (uses Az.Storage) (with hardcoded strings for both parameters (simulating that letsencrypt would have return them)), it also works fine.

• However, if I execute both parts and I try to call the write function passing it the token and tokenData properly obtained from Letsencrypt, then powershell complains about the modules importation:

Token       : HHuku4LO6C5QdLd7EPsAnwqgdvA_mLRdIpSBdgdee-M
Filename    : HHuku4LO6C5QdLd7EPsAnwqgdvA_mLRdIpSBdgdee-M
RelativeUrl : /.well-known/acme-challenge/HHuku4LO6C5QdLd7EPsAnwqgdvA_mLRdIpSBdgdee-M
AbsoluteUrl : uatapidockers.dev.bestinver.es/.well-known/acme-challenge/HHuku4LO6C5QdLd7EPsAnwqgdvA_mLRdIpSBdgdee-M
Content     : HHuku4LO6C5QdLd7EPsAnwqgdvA_mLRdIpSBdgdee-M.LtDDLv4YxLmEHubnL3RMJjT4w4UMTenG-OYoiOX_Ckw
Get-AzStorageAccount : The 'Get-AzStorageAccount' command was found in the module 'Az.Storage', but the module could 
not be loaded. For more information, run 'Import-Module Az.Storage'.
At line:86 char:19
+ $storageAccount = Get-AzStorageAccount -ResourceGroupName $STResource ...
+                   ~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Get-AzStorageAccount:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CouldNotAutoloadMatchingModule
 
Set-AzureStorageBlobContent : The 'Set-AzureStorageBlobContent' command was found in the module 'Azure.Storage', but 
the module could not be loaded. For more information, run 'Import-Module Azure.Storage'.
At line:93 char:1
+ Set-AzureStorageBlobContent -File $fileName -Container "public" -Cont ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Set-AzureStorageBlobContent:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CouldNotAutoloadMatchingModule

Please, any help would be appreciated.
Thanks in advance,
Best Regards,
Fernando.

[glatzert]

I really have litte idea about runbooks, but it seems you'll need to check if your modules are loaded (Get-Module) and if not load them.

Reloading a module (e.g. ipmo ACME-PS -force) will make problems with ACME-PS, if you do it in the same powershell session.

[natelowry]

I'm also running into this after upgrading from ACMESharp. I'm running the powershell script from some .NET code, but haven't run into issues like this in the past. How can I help debug?

[glatzert]

Can you outline how you initialize and use the script from your integrated shell?
How's module loading done? How's the lifetime of the shell instance?
Do you reload the module at any time?
What exactly is the error message?

[andyghc]

Just a note, I had all sorts of modules issues with my automation account and ACME-PS. I did get it to work by starting fresh with a new automation account, and importing the relevant az. modules. In my script I also manually import modules for use:

Import-Module Az.Accounts
Import-Module Az.Automation
Import-Module Az.Storage
Import-Module ACME-PS

Seems to work okay for me.

[natelowry]

I actually took a different approach for LE which I got working, so didn't end up figure out what was going on. Sorry I can't be more helpful, but I'll try.

Can you outline how you initialize and use the script from your integrated shell?

It's a .NET 4.6.1 exe running as a Windows service. Loaded like so:

var runspace = RunspaceFactory.CreateRunspace();
runspace.Open();
var pipeline = runspace.CreatePipeline();
pipeline.Commands.Add(new Command("foo/bar.ps1");
pipeline.Invoke();

How's module loading done? How's the lifetime of the shell instance?

I tried a few different ways with installing it for AllUsers or CurrentUser ahead of time and as a part of the script. With and without AllowClobber

Do you reload the module at any time?

I think so and it sounds like maybe that's the problem?

What exactly is the error message?

My error message was pretty much the same as above (sry I don't have the exact text)

[GitMaggie]

I had a similar case with this issue and now it's resolved .Hope it helps .
Root Cause:

• Import fails for Az.Storage module after ACME-PS module is imported. The reason behind such behavior is the environment in which Automation service runs powershell scripts is not publicly available PowerShell. Automation jobs run in a special secure environment.
  Resolution :
• To use it in Automaiton runbook, we need import Az.Storage module explicitly . The import statement should be before importing/running cmdlet from ACME-PS module.

Import-Module 'Az.Storage'
"Module Az.Storage is loaded"
...
$state = New-ACMEState -Path $PWD.Path

[fmartin1987]

I agree with @GitMaggie and as @glatzert suggested too, this (at least in my case) was a modules importation issue.

After load Az.Storage explicitly, it worked fine.

Thanks to all involved.

[glatzert]

Since I still do not have such an account, I'll not be able to verify that behaviour.
Nevertheless I added some information to the runbook example (via 2928282).

In the case anyone wants to add something, either use this issue or feel free to provide a PR.