Missing OS Disk Size on many VMs

[azure]

Missing OS Disk Size on many VMs

[hovsepm]

From @martinsawicki on June 3, 2017 1:7

1. Can you pls check if: IVirtualMachine.Inner.StorageProfile.OsDisk.DiskSizeGB is null or 0 as well? If so, then Azure is not returning this info to the SDK.
2. When and how were these VMs created? Or specifically: do they use the Managed disks feature or the old Unmanaged?
3. Which version of the SDK are you using?

[hovsepm]

From @brharr on June 6, 2017 2:3

Martin,

1. I did what you suggested and unfortunately the value comes back as null.
2. All of these VMs were created within the last month and all of them are using Managed Disks
3. As for the SDK, the Fluent version is 1.0.0

One other things to point out is that this is only true for the OS Disk. All of the Data Disks come through with all of the correct information.

Any ideas or thoughts?

Thanks,
Brian

From: Martin Sawicki [mailto:[email protected]]

1. Can you pls check if: IVirtualMachine.Inner.StorageProfile.OsDisk.DiskSizeGB is null or 0 as well? If so, then Azure is not returning this info to the SDK.
2. When and how were these VMs created? Or specifically: do they use the Managed disks feature or the old Unmanaged?
3. Which version of the SDK are you using?

[hovsepm]

From @martinsawicki on June 6, 2017 4:41

@brharr The fact you're getting nulls even from the inners indicates Azure is not returning this info in the context your code is calling for it. The fact you're using managed disks is also very relevant.

Now, are these VMs in a Running state at the time you're querying them, or some other state? (Managed disk size would not be available through this API if the VM is not running).

Also, if these are managed disks, then the preferred way to get information about them is actually via the Managed Disks API, not so much the VM (which was originally designed on the service-side for the unmanaged scenario, and there have been service-side issues with getting Managed disk info using that API).

The info you get from the Managed Disks API on the other hand does not depend on the state of the VM. You'd just use the IVirtualMachine.OSDiskId to get the ID of the disk resource to query for and then call the Disks API for it. For example:

var osDisk = azure.Disks.GetById(virtualMachine.OSDiskId);

To see more sample/test code that does something related, see: https://github.com/Azure/azure-sdk-for-net/blob/Fluent/src/ResourceManagement/Azure.Fluent/Fluent.Tests/Compute/VirtualMachineManagedDiskOperationsTests.cs
(or the many other Compute-related tests in the parent folder)

Pls let me know if this winds up helping.

[hovsepm]

From @brharr on June 6, 2017 14:32

Martin,

First of all, all of the VMs are Stopped to keep my costs down as I develop this application. So that does answer one of the questions. However, your suggestion to just pull the Disk and get the information that way caused a completely new problem. When I run the following command:

var osDisk = azure.Disks.GetById(vm.OSDiskId);

I get the following exception which doesn’t make any sense considering that the resourceGroupName parameter is not a requirement in the function:

Unhandled Exception: Microsoft.Rest.ValidationException: 'resourceGroupName' cannot be null.
at Microsoft.Azure.Management.Compute.Fluent.DisksOperations.d__7.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Azure.Management.Compute.Fluent.DisksOperationsExtensions.d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Azure.Management.Compute.Fluent.DisksImpl.d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Azure.Management.ResourceManager.Fluent.Core.GroupableResources5.<GetByResourceGroupAsync>d__9.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Microsoft.Azure.Management.ResourceManager.Fluent.Core.GroupableResources5.d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Azure.Management.ResourceManager.Fluent.Core.GroupableResources`5.GetById(String id)
at AzureIaaSAudit_ConsoleCore.Program.CreateVMEntity(IVirtualMachine vm, Azure azure) in C:\Users\brharr\OneDrive - Microsoft\Development\AzureIaaSAudit-ConsoleCore\Program.cs:line 255
at AzureIaaSAudit_ConsoleCore.Program.Main(String[] args) in C:\Users\brharr\OneDrive - Microsoft\Development\AzureIaaSAudit-ConsoleCore\Program.cs:line 128

So now I am in an even worse state than I was before.

Thanks,
Brian

From: Martin Sawicki [mailto:[email protected]]
Subject: Re: [Azure/azure-sdk-for-net] Missing OS Disk Size on many VMs (#3318)


The fact you're getting nulls even from they inners indicates Azure is not returning this info in the context your code is calling for it. The fact you're using managed disks is also very relevant.

Now, are these VMs in a Running state at the time you're querying them, or some other state? (Managed disk size would not be available through this API if the VM is not running).

Also, if these are managed disks, then the preferred way to get information about them is actually via the Managed Disks API, not so much the VM (which was originally designed on the service-side for the unmanaged scenario, and there have been service-side issues with getting Managed disk info using that API).

The info you get from the Managed Disks API on the other hand does not depend on the state of the VM. You'd just use the IVirtualMachine.OSDiskId to get the ID of the disk resource to query for and then call the Disks API for it. For example:

var osDisk = azure.Disks.GetById(virtualMachine.OSDiskId);

To see more sample/test code that does something related, see: https://github.com/Azure/azure-sdk-for-net/blob/Fluent/src/ResourceManagement/Azure.Fluent/Fluent.Tests/Compute/VirtualMachineManagedDiskOperationsTests.cs<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2Fazure-sdk-for-net%2Fblob%2FFluent%2Fsrc%2FResourceManagement%2FAzure.Fluent%2FFluent.Tests%2FCompute%2FVirtualMachineManagedDiskOperationsTests.cs&data=02%7C01%7Cbrharr%40microsoft.com%7C7143b025758c48538d3108d4ac966aec%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636323209420840554&sdata=rmLiHag9Adj0ftFpdvalSxRPRr%2BhfkCYVCrka4Pt05A%3D&reserved=0>

Pls let me know if this winds up helping.

[hovsepm]

From @martinsawicki on June 6, 2017 15:2

can you pls say what your vm.OSDiskId actually returns in that case? (the actual value)

[hovsepm]

From @martinsawicki on June 6, 2017 16:53

(you can replace subscription ID or other confidential parts of the string with X's - i just need to see the format of the ID. The error you're seeing hints at a problem with the structure of that ID. We have a number of tests testing that without issues, so it'd be interesting to understand if there are circumstances that trigger some misbehavior there)

[hovsepm]

From @brharr on June 6, 2017 18:43

So I added in a Console.Writeln statement to my code to get the OSDiskId and apparently it is only failing on a single OS Disk. Here is ID of the one that is causing the Exception:

/subscriptions/XXXXXXXX-XXXX-4b05-b32c-e96edf011ed4/resourceGroups/AzureIaaS-Audit/providers/Microsoft.Compute/disks/vm-linux-non-lb_OsDisk_1_de95028bdbe94fd6b1675f5ca0fdf4ef

From: Martin Sawicki [mailto:[email protected]]

(you can replace subscription ID or other confidential parts of the string with X's - i just need to see the format of the ID. The error you're seeing hints at a problem with the structure of that ID. We have a number of tests testing that without issues, so it'd be interesting to understand if there are circumstances that trigger some misbehavior there)

[anuchandy]

This is a long discussion hence summarizing it here for easy triaging -

User want to retrieve OS Disk size of bunch of VMs. VMs OS disks are managed disks. It seems when retrieving those virtual machines CRP is not including the actual OS Disk size in the VM response payload, instead service set null value for this property.

The solution we suggested is - to use managed disk APIs to retrieve the same OS disk resource, the id of the OS disk resource can be retrieved from virtualMachine.OSDiskId. This way user was able to retrieve disk size of all OS disks except ONE.

For the failing ONE, the exception user getting is thrown from SDK validations logic not by service: 'resourceGroupName' cannot be null.
It looks like an error in parsing the id that result in null resource group name. The failing id is /subscriptions/XXXXXXXX-XXXX-4b05-b32c-e96edf011ed4/resourceGroups/AzureIaaS-Audit/providers/Microsoft.Compute/disks/vm-linux-non-lb_OsDisk_1_de95028bdbe94fd6b1675f5ca0fdf4ef

We need to see whats going wrong when parsing this ID using .NET SDK utils for ResourceId.