Giter Site home page Giter Site logo

Comments (6)

alfpark avatar alfpark commented on July 18, 2024 2

There could be a lot of causes for this problem, but most likely the uncommitted block list on storage is in a state that will no longer work with the script. You will need to do one of the following: (1) perform a GetBlockList (with blocklisttype=uncommitted) to retrieve the uncommitted block list, commit the block list, then delete the blob, (2) create a dummy blob (can be of length zero) with the same blob name in the same container and transfer it using a non-blocked transfer, (3) wait 7 days for the uncommitted block list to garbage collect.

You can do either of (1) or (2) using the azure-storage python package, or you can perform (2) easily with a GUI-based tool like Azure Storage Explorer (windows-only unfortunately).

from azure-batch-samples.

theodorDiaconu avatar theodorDiaconu commented on July 18, 2024 1

Bump I have the same exact issue. For a 120mb file.

from azure-batch-samples.

alfpark avatar alfpark commented on July 18, 2024

Can you provide more context and/or output from the execution?

from azure-batch-samples.

chanhou avatar chanhou commented on July 18, 2024

I am transfer a tsv file to the blob and i disable the match md5 and using sas key which provided by the cup.

screenshot from 2015-09-04 11 14 42

from azure-batch-samples.

chanhou avatar chanhou commented on July 18, 2024

Thanks for your reply.

Again I have another question.

The problem you said already fixed but other problem arise,

that is it always said that connection aborted, error(110,'connection timed out') when the progress around 96%, no matter I change the chunksizebytes and num of workers.

How should I do for this problem?

from azure-batch-samples.

alfpark avatar alfpark commented on July 18, 2024

Unfortunately, that's an issue that will be hard to diagnose since that will depend on a variety of factors, including your connection quality and bandwidth, among other things. I would try to keep the chunksizebytes at default and decreasing the number of workers in the range of 2 to 8 to see if that alleviates your problem. Then you can increase that number with something that's a good tradeoff between speed and consistency in transfer.

I am closing this issue now since the discussion is no longer related to the original issue.

from azure-batch-samples.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.