Giter Site home page Giter Site logo

qingstor / qingstor-sdk-python Goto Github PK

View Code? Open in Web Editor NEW
14.0 11.0 12.0 351 KB

The official QingStor SDK for the Python programming language.

Home Page: https://qingstor-sdk-python.readthedocs.io/en/latest/

License: Apache License 2.0

Makefile 0.79% Python 99.21%
qingstor-sdk qingstor yunify qingcloud

qingstor-sdk-python's Introduction

QingStor SDK for Python

Build Status API Reference License Join the chat

English | 中文

The official QingStor SDK for the Python programming language.

Before you start using the SDK, make sure you have a basic understanding of the concepts of QingStor object storage (such as Zone, Service, Bucket, Object, etc.).

This SDK try to keep a one-to-one correspondence with the methods on the QingCloud QingStor object storage documentation. For details of each method, please refer to the corresponding chapter in the link.

Quick Start

Now you are ready to code. You can read the detailed guides in the list below to have a clear understanding.

Checkout our releases and change log for information about the latest features, bug fixes and new ideas.

Reference Documentations

Contributing

Please see Contributing Guidelines of this project before submitting patches.

LICENSE

The Apache License (Version 2.0, January 2004).

qingstor-sdk-python's People

Contributors

dependabot[bot] avatar owarai avatar prnyself avatar sightseer avatar wangmq0719 avatar xuanwo avatar yu31 avatar zhengchaochao avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

qingstor-sdk-python's Issues

Add timeout support for request send

Since there is no timeout parameter in put_object, get_object, delete_object, head_object, options_object, it is trend to lead a forever waiting for response if the response message is lost when an net error happens, this pull request is to add timeout parameter in put_object, get_object, delete_object, head_object, options_object methods.
The timeout parameter provides the user an way to control how long to wait, when an net error occurs, which might lead program to wait for response forever, the default value of timeout is set to None, and one can easily ignore it, if the user does not want to have a timeout be set.

Ref: #34

分段上传如何合并的问题

第一、我参照该文件 qingstor-sdk-python/docs/example/multipart_upload.md ,想要使用qingstor的文件分段上传功能,当文件分段上传完成后,该文档最后的部分给出了fin_resp,这是一个从未出现的参数

第二,参考上下问的描述,我使用了如下方法尝试合并文件
with open(filepath, 'rb' ) as f:
for i in range(parts_count):
start = time.time()
part_size = min(chunk_size, file_size - i * chunk_size)
data = f.read(part_size)
part_resp = bucket_srv.upload_multipart(object_key,upload_id=upload_id,part_number=str(i), body=data)
end = time.time()
elasped = end -start
print(end-start)
print(elasped*(parts_count-i))
if part_resp.status_code != 201:
print("Upload part of object(name:{}) to bucket({}) failed with given message:{}".format(object_key,bucket_name,str(part_resp.content,'utf-8')))
abort_resp = bucket_srv.abort_multipart_upload(object_key,upload_id)
print("Abort multi upload executed: 204 expected, actually:%d", abort_resp.status_code)
else:
print("part %d uploaded." %i)
list_resp = bucket_srv.list_multipart(object_key=object_key, upload_id=upload_id)
final_resp = bucket_srv.complete_multipart_upload(object_key=object_key,upload_id=upload_id)

但得到了下面的错误
File "qstor_upload.py", line 57, in
final_resp = bucket_srv.complete_multipart_upload(object_key=object_key,upload_id=upload_id)
File "/usr/local/python3/lib/python3.8/site-packages/qingstor_sdk-2.6.0-py3.8.egg/qingstor/sdk/service/bucket.py", line 1210, in complete_multipart_upload
File "/usr/local/python3/lib/python3.8/site-packages/qingstor_sdk-2.6.0-py3.8.egg/qingstor/sdk/service/bucket.py", line 1197, in complete_multipart_upload_request
File "/usr/local/python3/lib/python3.8/site-packages/qingstor_sdk-2.6.0-py3.8.egg/qingstor/sdk/service/bucket.py", line 1233, in complete_multipart_upload_validate
TypeError: 'NoneType' object is not iterable

your signature is not canonical signature stated in the latest document

you can refer to my implementation in python:
https://github.com/yiakwy/Siganture-Authentication-Package/blob/master/SDK/sign5py3k.py

in client side:

Request.py

(1) in Authorization part, it should start with "Signature" so that other programmes can know at which part of authorization process you are in.

(2) You did not specify which encoding algorithms you are using. This should be a serious bug in the future. Optionally, you should specify which headers you want to use to encode for a signature. 403 response will tell users how to use headers.

(3) signature template should be :
signature_string = "(request-target): {method} {path}\n{ret}" (notice to ascii space)

and path should be included in signature_string template. The path does not include parameters otherwise for some url, the result of which will exceed 128 bytes.

config.py

the best practice should be:

a global_config.py for system. Then use dir to open a user's local config and replace the default values with ones defined by a user. I noticed that you use yaml. But in python, .py is just enough because it is a dynamic language. Using .py, you can implement "Lazy loading mode" very easily.

Architecture

As a SDK, I suggest the architecture as following:

Config & authentication

Global_config.py

Auth.py

  1. SignatureAuthentication(Authentication)

Qingstor.py

class QingstorAuthentication(SignatureAuthentication):
      def __init__(app_key, app_secret, permi=(), timeout=60)
class Application:
     def make_req(method, url, params, headers, timeout)
           # use httpConnection, Reqeust, Oauth whatever method you like ,,, haha
 class QingstorApplication(Application):
     # business logics
     def get_xx
     def get_yy
     def get_zz
  1. Constants should be defined in __init__.py; you can also import other files in __init__.py

  2. Do to abuse to use get_xx method, if you just want to get attribute internally , access it directly or use _get_xx method instead.

error.py

  1. I suggest to define a handle_error method in "exception.py" or "error.py"
def handler_errors(func)

    logger =  loggerAdapter(_log, func.__name__)

    def wrapper(*args, **kw):
         try:
             ret = func(*args, **kw)
         except xx:
         except yy:
         except zz:
             logger.WHAT[ex]()
             # return None or raise an error as you wish
    return wrapper

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.