labd/commercetools-python-sdk

Commercetools Python SDK

Python SDK for Commercetools

This is an unofficial Python SDK for the Commercetools platform. It only supports Python 3.6+ and uses type annotation for an improved development experience.

The API is generated using the commercetools api RAML file and uses marshmallow for the serialization and deserialization.

Installation

pip install commercetools

Example

from commercetools.platform import Client

client = Client(
    client_id="<your-client-id>",
    client_secret="<your-client-secret>",
    scope=["<scopes>"],
    url="https://api.europe-west1.gcp.commercetools.com",
    token_url="https://auth.europe-west1.gcp.commercetools.com",
)

product = (
    client
    .with_project_key("<your-project-key>")
    .products()
    .with_id("00633d11-c5bb-434e-b132-73f7e130b4e3")
    .get())
print(product)

The client can also be configured by setting the following environment variables:

export CTP_CLIENT_SECRET="<client secret>"
export CTP_CLIENT_ID="<client id>"
export CTP_AUTH_URL="https://api.europe-west1.gcp.commercetools.com"
export CTP_API_URL="https://auth.europe-west1.gcp.commercetools.com"
export CTP_SCOPES="<comma seperated list of scopes>"

And then constructing a client without arguments:

from commercetools.platform import Client

client = Client()

product = (
    client
    .with_project_key("<your-project-key>")
    .products()
    .with_id("00633d11-c5bb-434e-b132-73f7e130b4e3")
    .get())

print(product)

Releasing

To release this package first (pip) install bump2version and update the CHANGES file. Then update the version (either major/minor/patch depending on the change)

bumpversion --tag <major,minor,patch>

bumpversion is naive because it string replaces, so do a sanity check it didn't accidentally update a Pypi dependency. If not, push the code:

git push --follow-tags

We use GitHub actions so make sure the build succeeds and then go to the tags tab (https://github.com/labd/commercetools-python-sdk/tags).

Click the dots to trigger a release action. Copy the changelog items in the release description and enter the release version. This will upload the release to PyPi.

commercetools-python-sdk's Issues

New commercetools.platform.Client not compatible with pytest mocker

When using the (as docs described) Client imported from from commercetools.platform import Client the pytest fixture commercetools_api doesn't work.

It looks like no http requests are mocked and actual requests are triggered (instead of using the mocker), causing the invalid token exception.

Using the "old" from commercetools.import Client fixes the problem (although it uses a slighly different API).

Tested with commercetools version 14.

BUG broken auth missing token

commercetools==5.0.0

got valid creds (works with curl)
creds are present

always dies *** oauthlib.oauth2.rfc6749.errors.MissingTokenError: (missing_token) Missing access token parameter.

seems related to requests/requests-oauthlib#324

same happens with 4.1.0 so i believe its the interface to the oauthlib that has changed

Homepage URL is incorrect

In setup.py:

url="https://github.co/labd/commercetools-python-sdk"

should be

url="https://github.com/labd/commercetools-python-sdk"

Docs not updating since switching from Sem-ver

Could be related to the switch, could not be. But the changelog scared me a bit when I was reading it earlier. https://commercetools-python-sdk.readthedocs.io/en/latest/ 😛

400 InvalidField errors give marshmallow error instead of raising CommercetoolsError

Example response:

{"statusCode":400,"message":"The value \'49136\' is not valid for field \'sapCode\'. ","errors":[{"code":"InvalidField","message":"The value \'49136\' is not valid for field \'sapCode\'. "}]}

The reason is because in _schemas/_error.py line 502 an invalid schema is used:

marshmallow.fields.Nested( nested="commercetools.schemas.None.anySchema", unknown=marshmallow.EXCLUDE, allow_none=True, ),
This happens a few more times in nested lists with unknown types.

Issue with scope value

Problem
When I try to create a client using Python SDK, I get a warning in scope value. If I follow the warning and fix it, I get error in scope value
Warning: Scope has changed from “manage_products:” to “manage_products:{project_key} view_products:{project_key}.
Error: Permissions exceeded. Only the following permissions can be requested: manage_products

Using the below code to connect:

client = Client(
    client_id="my-client-id",
    client_secret="my-client-secret",
    # scope=["manage_products:{project-key}"],
    scope=["manage_products:{project-key}", "view_products:{project-key}"],
    url="https://api.{region}.gcp.commercetools.com",
    token_url="https://auth.{region}.gcp.commercetools.com",
)

client = Client()

product = (
    client
    .with_project_key("project_key")
    .products()
    .with_id({guid_Id})
    .get())

Feature request: Autoreload testing server

F.e. using werkzeug https://palletsprojects.com/p/werkzeug/

graphql usage

Hi,
I could not find how to properly build GraphQLRequest object. Trying to build it like
body = GraphQLRequest( query="""query ReturnProductsIds($limit: Int!, $offset: Int!) { products( where: "masterData(published=true)" sort: "id asc" offset: $offset limit: $limit ) { results { id } } }""", variables=GraphQLVariablesMap([('limit', 500), ('offset', 0)]) )
and it doesn't work. After object serialization variables become empty dict which causes error response from commercetools API
Appreciate if you provide an example.

New Version release

Hello guys, we notice that the version release was bumped but not released yet, commit, there is any plan to release the new version?

Thanks

Missing CartCreated schema

Hi! While calling Client.messages.query I'm noticing that the CartCreated schema is missing:

Traceback (most recent call last):
  File "messages.py", line 41, in get_messages
    tptest = sdk_client.messages.query()
  File "/usr/local/lib/python3.7/site-packages/commercetools/services/messages.py", line 63, in query
    schema_cls=MessagePagedQueryResponseSchema,
  File "/usr/local/lib/python3.7/site-packages/commercetools/client.py", line 122, in _get
    return schema_cls().load(response.json())
  File "/usr/local/lib/python3.7/site-packages/marshmallow/schema.py", line 728, in load
    data, many=many, partial=partial, unknown=unknown, postprocess=True
  File "/usr/local/lib/python3.7/site-packages/marshmallow/schema.py", line 866, in _do_load
    unknown=unknown,
  File "/usr/local/lib/python3.7/site-packages/marshmallow/schema.py", line 674, in _deserialize
    index=index,
  File "/usr/local/lib/python3.7/site-packages/marshmallow/schema.py", line 496, in _call_and_store
    value = getter_func(data)
  File "/usr/local/lib/python3.7/site-packages/marshmallow/schema.py", line 667, in <lambda>
    val, field_name, data, **d_kwargs
  File "/usr/local/lib/python3.7/site-packages/marshmallow/fields.py", line 356, in deserialize
    output = self._deserialize(value, attr, data, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/marshmallow/fields.py", line 728, in _deserialize
    result.append(self.inner.deserialize(each, **kwargs))
  File "/usr/local/lib/python3.7/site-packages/marshmallow/fields.py", line 356, in deserialize
    output = self._deserialize(value, attr, data, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/commercetools/helpers.py", line 222, in _deserialize
    return self._load(value, data, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/commercetools/helpers.py", line 209, in _load
    f"Could not find discriminator schema {discriminator_value} for field '{self.name}' ({value})"
ValueError: Could not find discriminator schema CartCreated for field 'results' (redacted)

Deprecation warnings form marshmallow

I get warnings from marshmallow/fields.py:184: in __init__:

WARNING from "foo.py" in test_foobar():
".../commercetools/platform/models/_schemas/category.py RemovedInMarshmallow4Warning: The 'missing' argument to fields is deprecated. Use 'load_default' instead."

There are many marshmallow.fields.String(...) with the deprecated missing argument.

The deprecation is from 2021, see: https://github.com/marshmallow-code/marshmallow/blob/dev/CHANGELOG.rst#3130-2021-07-21

Testing: Add constraintAttribute validation

The testing backend currently allows having attribute values that don't match the constraint of that attribute, e.g. having two different values for two variants of the same product for an attribute that has the SAME_FOR_ALL constraint.

The real commercetools API gives a nice error message like:

commercetools.exceptions.CommercetoolsError: The value '"Some Other value"' is not valid for field 'anAttributeWithASameForAllConstraint'. Allowed values are: "Some value". The value of the attribute must be the same across all variants.

It would be nice if we had similar validation in the testing backend. In quick half-pseudocode something like:

   def validate_constrained_attribute(
        variants: List[models.ProductVariantDraft], attr: ProductAttribute
    ):
        constraint_type = attr.constraint
        if constraint_type == models.AttributeConstraintEnum.SAME_FOR_ALL:
            flat_attributes = [
                item
                for sublist in [var.attributes for var in variants]
                for item in sublist
            ]
            # check if all values for the attribute across all variants are the same:
            values = [a.value for a in flat_attributes if a.name == attr.name]
            if not values:
                return

            unique_values = some form of set depending on attribute type
            if len(unique_values) != 1:
                raise ValidationError(
                    "Some complete error message"
                )
            else:
                return
        elif constraint_type == models.AttributeConstraintEnum.UNIQUE:
            flat_attributes = [
                item
                for sublist in [var.attributes for var in variants]
                for item in sublist
            ]
            # check if all values for the attribute across all variants are different:
            values = [a.value for a in flat_attributes if a.name == attr.name]
            if not values:
                return

            unique_values = some form of set depending on attribute type
            if len(unique_values) != len(values):
                raise ValidationError(
                    "Some complete error message"
                )
            else:
                return
        elif constraint_type == models.AttributeConstraintEnum.COMBINATION_UNIQUE:
            # this will probably be a little more convoluted
        else:
            raise ValidationError(
                "Could not recognize constrained attribute type: %s", attr.constraint
            )

Unit test objects are not timezone aware

When you create objects they use datetime.datetime.now(), which is local aware, while Commercetools always returns UTC.

General commercetools errors are not serialized

Could not find discriminator schema General for field 'errors' ({'code': 'General', 'message': "Oops. This shouldn't happen."})

This happens when CT has downtime or general API errors:

'{"statusCode":500,"message":"Oops. This shouldn\'t happen.","errors":[{"code":"General","message":"Oops. This shouldn\'t happen."}]}'

querying /orders/search endpoint through always an error

I'm trying to query https://docs.commercetools.com/api/projects/order-search#search-orders endpoint by calling:

range_query = {
    "query" : {
    "and": [
        {
            "range": {
                "field": "createdAt",
                "gte": "2023-05-31T23:00:00.000Z"
            }
        },
        {
            "exact": {
                "field": "store.key",
                "value": "at"
            }
        }
    ]
},
    "sort" : None,
    "limit" : 10,
    "offset": 10
}

orderSearchRequest = OrderSearchRequest("query"=range_query)
ctClient = client.with_project_key("prod").orders().search().post(body=orderSearchRequest}))

The root cause is the deserialize of orderSearchRequest in the post request body results in a empty dict which results in '400' error code CT response with following message
b'{"code":400,"errors":["Invalid JSON at '.query': Exactly one [fullText], [exact], [prefix], [wildcard], [range] or [exists] expression needed."]}'

could you please assists what is missing here?
thanks

KeyError in message schema

With the following payload:

{
  createdAt: '2021-01-30T16:34:06.907Z', 
  id: 'cb093b59-045d-47eb-8c6e-0a7fbf15b14b', 
  notificationType: 'Message', 
  payloadNotIncluded: {
    payloadType: 'OrderCreated', 
    reason: 'Payload too large'
  }, 
  projectKey: 'some-project', 
  resource: {
    id: 'a41e26ce-8801-4795-bc93-b1507e1d925f', 
    typeId: 'order'
  }, 
  resourceUserProvidedIdentifiers: {
    orderNumber: 'ORDER00001'
  }, 
  resourceVersion: 1, 
  sequenceNumber: 1, 
  version: 1
}

The post-load of the OrderCreatedMessageSchema fails on

del data["type"]

Why is requests_mock and Werkzeug are normal dependencies?

I'm wondering why requests_mock and Werkzeug are not only dev dependencies ?

commercetools-python-sdk/setup.py

Lines 5 to 15 in d31c024

	install_requires = [
	"requests>=2.7.0",
	"pytz",
	"requests-oauthlib>=1.0.0",
	"requests_mock>=1.8.0",
	"marshmallow>=3.10.0",
	"marshmallow-enum>=1.5.1",
	"WebOb>=1.8.0",
	"Werkzeug>=0.15.2",
	"wrapt>=1.10.0",
	]

Readme Example Isn't Working

Hi,

First off, thank you for your work on this SDK.

I was playing around with the latest update from yesterday, specifically this piece of code in the README.rst:

product = (
    client
    .with_project_key("<your-project-key>")          # Error 1
    .products()                                      # Error 2
    .with_id("00633d11-c5bb-434e-b132-73f7e130b4e3") # Error 3
    .get()
)

Adding and removing lines from that code, here are the different errors I see:

1. AttributeError: 'Client' object has no attribute 'with_project_key'
2. TypeError: 'ProductService' object is not callable
3. AttributeError: 'ProductService' object has no attribute 'with_id'

However, the old client syntax still works:

product = client.products.get_by_id(id)

I'd appreciate any advice on how to fix this issue.

Python Version: 3.8.2
SDK Version: 13.0.0

Cheers
Sia

Schema field String(many=True) doesn't validate data correctly

The ProjectSchema does contains these fields:

countries = marshmallow.fields.String(many=True)
currencies = marshmallow.fields.String(many=True)
languages = marshmallow.fields.String(many=True)

When initialising this schema with this data, there are validation errors raised:

{"countries": ["AT", "NL", "CH", "BE", "GB", "DE", "IT", "LU", "ES"], "currencies": ["EUR", "GBP", "CHF"], "languages": ["en-GB", "de-DE", "fr-FR", "nl-BE", "nl-NL"] }

marshmallow.exceptions.ValidationError: {'currencies': ['Not a valid string.'], 'countries': ['Not a valid string.'], 'languages': ['Not a valid string.']}

But it works when I change the fields in the schema like this.

countries = marshmallow.fields.List(marshmallow.fields.String())
currencies = marshmallow.fields.List(marshmallow.fields.String())
languages = marshmallow.fields.List(marshmallow.fields.String())

test

Bearer token caching

While using this library from a stateless setup like lambda, does the client creation code request bearer token every time the code is invoked? Is this okay, or should I cache the bearer token somewhere to avoid requesting a new bearer token every time?

cart_discounts extraction not working if discounts are enabled on total price

{{[}}ERROR{{] ValueError: Could not find discriminator schema totalPrice for field 'target' (

{'type': 'totalPrice'}
)}}
Traceback (most recent call last):
File "/var/task/ct_extract.py", line 194, in extract
{{ query_result = entity_query(offset=query_offset, limit=result_limit, where="lastModifiedAt >= "

{from_date}" and lastModifiedAt < "{to_date}"".format(from_date=from_date, to_date=to_date))}}
File "/var/task/commercetools/services/cart_discounts.py", line 78, in query
return self._client._get(
File "/var/task/commercetools/client.py", line 40, in _get
return response_class.deserialize(response.json())
File "/var/task/commercetools/platform/models/cart_discount.py", line 293, in deserialize
return CartDiscountPagedQueryResponseSchema().load(data)
File "/var/task/marshmallow/schema.py", line 723, in load
return self._do_load(
File "/var/task/marshmallow/schema.py", line 862, in _do_load
result = self._deserialize(
File "/var/task/marshmallow/schema.py", line 665, in _deserialize
value = self._call_and_store(
File "/var/task/marshmallow/schema.py", line 493, in _call_and_store
value = getter_func(data)
File "/var/task/marshmallow/schema.py", line 658, in getter
return field_obj.deserialize(
File "/var/task/marshmallow/fields.py", line 367, in deserialize
output = self._deserialize(value, attr, data, **kwargs)
File "/var/task/marshmallow/fields.py", line 671, in _deserialize
return self._load(value, data, partial=partial)
File "/var/task/marshmallow/fields.py", line 654, in _load
valid_data = self.schema.load(value, unknown=self.unknown, partial=partial)
File "/var/task/marshmallow/schema.py", line 723, in load
return self._do_load(
File "/var/task/marshmallow/schema.py", line 862, in _do_load
result = self._deserialize(
File "/var/task/marshmallow/schema.py", line 611, in _deserialize
ret_l = [
File "/var/task/marshmallow/schema.py", line 614, in
self._deserialize(
File "/var/task/marshmallow/schema.py", line 665, in _deserialize
value = self._call_and_store(
File "/var/task/marshmallow/schema.py", line 493, in _call_and_store
value = getter_func(data)
File "/var/task/marshmallow/schema.py", line 658, in getter
return field_obj.deserialize(
File "/var/task/marshmallow/fields.py", line 367, in deserialize
output = self._deserialize(value, attr, data, **kwargs)
File "/var/task/commercetools/helpers.py", line 294, in _deserialize
return self._load(value, data, **kwargs)
File "/var/task/commercetools/helpers.py", line 280, in _load
raise ValueError(

ValueError: Could not find discriminator schema totalPrice for field 'target' ({'type': 'totalPrice'})

If the configuration of the cart discount on total price is not set for any of the cart discount records the extract works fine - I have an issue only when a cart discount record is set on total price

Resource types missing from ResourceTypeId

I'm trying to add a custom type for a cart which is in the list of customizable data types but the list of types in this project doesn't seem to include it.

Requests mock querystring converts all values to a multivalve dict

In de unit tests this doesn't work: client.product_projections.query(limit=1)

Then this error is raised: marshmallow.exceptions.ValidationError: {'limit': ['Not a valid integer.']}

That is caused by using abstract.AbstractQuerySchema().load(request.qs) which is used in "testing/product_types.py#46".
That qs property on the RequestMock request object converts a querystring ?limit=1 to a multivalue dict: {'limit': [1]}, instead of a normal integer {'limit': 1}.

Error response cannot be parsed

Tested version: 14.0.0b12

Problem
commercetools SDK raises an Exception (no CommercetoolsError) because it cannot parse an error response from commercetools;

Could not parse error response
This is due to a validation error:

{'errors': {0: {'actionIndex': ['Unknown field.'], 'action': ['Unknown field.']}}}

In the following block:

        try:
            obj = ErrorResponseSchema().loads(response.content)
        except ValidationError:
            raise Exception(f"Could not parse error response: {response.content}")

Example response:

b'{"statusCode":400,"message":"A value is required for field \'contract\'.","errors":[{"code":"RequiredField","message":"A value is required for field \'contract\'.","action":{"action":"setCustomField","name":"someCustomField","value":"24e3565c-9135-ec11-a459-6045bd8b7265"},"actionIndex":1,"field":"contract"}]}'

Single quotes are allowed for values during testing

Commercetools does not like single quotes around values. F.e. createdAt >= '2019-09-09' does not work, but does in our testing framework.

Low prio issue imo :P. PoC fix: https://github.com/labd/commercetools-python-sdk/compare/feature/bugfix-malformed-where?expand=1

predicate with date not allowed

Commercetools allows predicates like lastModifiedAt > "2021-05-01" but it gives a marshmallow validation error.

def test_orders_query_filter_single_date(commercetools_api, old_client):
    order = get_test_order()
    commercetools_api.orders.add_existing(order)
    where = [
        'createdAt >= "2019-10-15"',
    ]

    results = old_client.orders.query(where=where)
    assert results.total == 1

Probably have to handle/detect in testing/predicates.py 503 if it's a datetime and convert to a valid datetime