Comments (8)
Thank you Tres!
from python-storage.
@william-silversmith The headers needed to do this task are buried several layers deep. I'm wondering if a better solution than trying to guess which ones to map onto the blob's properties, we might be better off designing for your usecase by supplying a response_callback
argument to the Blob.download_*
methods:
- The callback would take the blob instance and a
requests.Response
object as its arguments. - For non-chunked downloads, the callback would be called once; chunked downloads would call it once per chunk.
An example callback implementation for your usecase might be something like:
def _update_blob_content_encoding(blob, response):
if blob.content_encoding is None: # latch
blob.content_encoding = response.headers.get('Content-Encoding')
from python-storage.
If we don't use a callback, then the relevant headers to start with would seem to be:
Cache-Control
->Blob.cache_control
Content-Disposition
->Blob.content_disposition
Content-Encoding
->Blob.content_encoding
Content-Length
->Blob.size
Content-Type
->Blob.content_type
Etag
->Blob.etag
X-Goog-Generation
:Blob.generation
X-Goog-Hash
:Blob.crc32c
X-Goog-Metageneration
:Blob.metageneration
X-Goog-Storage-Class
:Blob.storage_class
from python-storage.
I almost never need more exotic headers than the ones you listed. If it's easier to only expose those headers, that's fine with me. If you want to go for a more general solution, I am fine with a (synchronous) callback. Thanks again!
from python-storage.
Any word on when this might drop? Thanks for your help!
from python-storage.
Hi Tres, I've made some changes to my own branch of this repo and they seem to work. I tried making a PR but the button was greyed out. Here's a link to the changes in case y'all are interested. In the mean time, I'll direct my repos to use the forked version.
from python-storage.
Unfortunately, it seems PyPI won't let me use a direct reference to github. It would be extremely convenient if some version of this could get merged.
from python-storage.
Figured out the PR submission issue.
from python-storage.
Related Issues (20)
- tests.system.test_bucket: test_blob_exists_hierarchy failed HOT 1
- tests.system.test_bucket: test_bucket_list_blobs_hierarchy_root_level failed
- tests.system.test_bucket: test_bucket_list_blobs_hierarchy_first_level failed
- tests.system.test_bucket: test_bucket_list_blobs_hierarchy_second_level failed
- tests.system.test_bucket: test_bucket_list_blobs_hierarchy_third_level failed
- tests.system.test_bucket: test_bucket_list_blobs_hierarchy_w_include_trailing_delimiter failed
- tests.system.test_bucket: test_bucket_list_blobs failed HOT 1
- tests.system.test_bucket: test_bucket_list_blobs_w_user_project failed
- tests.system.test_bucket: test_bucket_list_blobs_paginated failed HOT 1
- tests.system.test_bucket: test_bucket_list_blobs_paginated_w_offset failed HOT 1
- tests.system.test_transfer_manager: test_upload_many_skip_if_exists failed HOT 2
- tests.system.test_transfer_manager: test_upload_many_from_filenames_with_attributes failed HOT 1
- tests.system.test_transfer_manager: test_download_many failed HOT 1
- tests.system.test_transfer_manager: test_download_many_with_threads_and_file_objs failed HOT 1
- tests.system.test_bucket: test_ubla_set_unset_preserves_acls failed HOT 1
- tests.system.test_kms_integration: test_bucket_w_default_kms_key_name failed HOT 1
- Make it possible to update / add to the user-agent for an existing client object
- Micropi cannot install the package HOT 1
- OSError occurred while downloading files using transfer_manager.download_many_to_path HOT 6
- Warning: a recent release failed
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from python-storage.