Reading an append blob to a string takes long and then returns HTTP 412

[azure]

Reading an append blob to a string takes long and then returns HTTP 412

[XiaoningLiu]

@petmat You are right about "trying to read an append blob that is constantly getting more lines in it."

blobURL.download() will try to download a blob with a HTTP Get request into a stream. When stream unexpected ends due to such as network broken, a retry will resume the stream read from the broken point with a new HTTP Get request.

The second HTTP request will use conditional header IfMatch with the blob's ETag returned in first request to make sure the blob doesn't change when the 2nd retry happens. Otherwise, a 412 conditional header doesn't match error will be returned. This strict strategy is used to avoid data integrity issues, such as the blob maybe totally over written by someone others. However, this strategy seems avoiding you from reading a constantly updated log file when a retry happens.

While I don't think it's bug, but we need to make this scenario work for you. There are 2 solutions, please have a try:

1> snapshot the append blob first, and read from the snapshot blob
2> set maxRetryRequests to 0 in blobURL.download() options. And download a small range (for example 4MB) every time. This means no conditional retry will happen, so no 412 error either. But the returned stream may not complete when there are network issues. Set a small range will help avoid it. Check the stream length.

[petmat]

I went with solution number one and I got the snapshot of the blob downloaded with the following code:

// ...
const blobURL = BlobURL.fromContainerURL(containerURL, blobName);
console.log('Downloading blob...');
const snapshotResponse = await blobURL.createSnapshot(Aborter.none);
const snapshotURL = blobURL.withSnapshot(snapshotResponse.snapshot);
const response = await snapshotURL.download(Aborter.none, 0);
console.log('Reading response to string...', snapshotURL.blobContext.length);
const body = await streamToString(response.readableStreamBody);
// ...

Downloading the blob did take some time though. (couple of minutes) But then again the size of the blob is ~140 MB. I didn't explore the solution number two because it seemed a bit contrived. But I'm happy with this 👍 Next I'm just going to implement a strategy to decide how often a new snapshot will be created and possibly removing old ones.

Thank you!