Doesn't work when trying to acquire a lease with Azure function core tool

[azure]

Doesn't work when trying to acquire a lease with Azure function core tool

[XiaoningLiu]

@kylepope thanks for the detailed investigation. Sorry for the late response, I'm quite busy these weeks.

We are working on a new architecture of Azurite, lease is an important part in the implementation. For current Azurite implementation, @edwin-huber any quick thoughs?

[kylepope]

@XiaoningLiu Thanks for the response -- that's news to me but good to hear! Just curious -- any clues what a ballpark timeline on a preliminary implementation of the new architecture? The whole lease handing described in the docs is definitely complicated so I'm sure it'll take time to get that right.

A few more details on things I looked into with my use-case described above:

• I believe MD5 hash error is due to the fact that the GetBlob implementation will calculate the hash from the the raw blob from disk to the response body. I thought I might be able to remove new lease properties I added to the persisted blob prior to hash generation (and add them to response headers instead for example), but I think this would require a bigger change and I'm not sure what the ramifications might be.
• I was specifically looking in general in the code for how some pieces of blob data are converted from raw blob properties to response headers, like lease state (which should be a "x-ms-lease-state" header in a get blob response), but it looks like we're not doing that in a general way right now.
• I also tried to adding a function to lib/core/env.js to generate a new ID for lease information to perhaps store that separately (e.g. prefix with a "D" character, following what was done for block ID and snapshot ID) and then making the assignment to request.id specifically for lease operations. However, this doesn't end up working because some of that information needs to be saved as part of the blob I think (I ran into more different errors). I'm sure I'd need to make changes to other pieces of the implementation, but I'd need to figure out which.

Anyway, I'm pretty much out of free cycles at the moment. That said, I'd love to hear more about your future plans.

[kylepope]

I've been encountering this problem while trying to get Azure Durable Functions to work on a mac using Azurite for the storage emulation. The specific problem here is due to the fact Azurite validation isn't quite right. If you look at lib/validation/blob/LeaseId.js, there are two places where a isUUID is used. This check isn't to spec since the definition of a Guid is a little more flexible than what the validation is checking. I'd recommend loosening the restriction, though I'm not sure what is most appropriate here (thoughts?).

While getting around this issue in my own local testing, I noticed a second problem with how leases work. The Durable Functions extension seems to go through the following flow in my testing:

1. Get lease properties via a HEAD /devstoreaccount1/durablefunctionshub-leases/default/durablefunctionshub-control-02
2. Acquire a lease on the resource via a PUT /devstoreaccount1/durablefunctionshub-leases/default/durablefunctionshub-control-02?comp=lease
3. Perform a actual update to the resource via a PUT /devstoreaccount1/durablefunctionshub-leases/default/durablefunctionshub-control-02 with a body like { "PartitionId": "durablefunctionshub-control-02", "Owner": "<my-name>", "Token": "df7b79142c0b4b219d259bc67b19f641", "Epoch": 2 }

However, what happens here is that the acquired lease ID, expiration and other information written to the blob store in step 2 is overwritten with information provided in step 3. The reason for that is that step 3 routes to the _createOrUpdateBlob function in lib/core/blob/StoreManager.js, which will blow away previous contents and update them with new ones.

I toyed with the idea to propagate the lease properties to the newly created/saved object, but ran into separate issues because the MD5 hash of the blob's value no longer matches what was saved by call 3 (I think). I may not be understanding how this stuff works, but it seems like we should instead be saving lease information to a separate document/place in the store so it remains persisted.

Anyone have thoughts on this?

[XiaoningLiu]

@kylepope Thanks! For the new architecture, we have an initial design specification. In the new design, we have several layers, the protocol layer will auto generated by API swagger definition, and persistence layer will be manually maintained. Lease information needs to be persisted.

There are still several months before we can publish the new architecture of Azurite, that means for critical issues or bugs, we should fix in current Azurite architecture. As you can see in the Azurite issues, there are plenty of items to be fixed in current architecture, we are kind of lacking resources to fix them in a short time. Contributions are very welcome.

It's very difficulty for an emulator implements 100% features or align with the real Azure Storage services. For example, the API versions supported. But we can define a matrix make the differences more clear.

For the lease issue, I don't understand about the md5 issue. The MD5 hash is the blob's content md5? As far as I know, the blob content will be persisted into disk, and the properties will be persisted into database object. The hash generation shouldn't be impacted by the blob properties in the database.

[XiaoningLiu]

Hi Can you try latest released 3.3.0-preview? We did some refactor for lease implementation for blobs and containers. If the issue still happens for 3.3.0-preview, please share your environments, example code, and steps for the reproduce. @blueww is helping with Azure Function comptiability invesitgation with Azurite, and we'd like more information.

[edwin-huber]

Azure Functions Core Tools now working with latest release.
Closing this issue.