watson-developer-cloud/python-sdk

:snake: Client library to use the IBM Watson services in Python and available in pip as watson-developer-cloud

python-sdk's Issues

Current build of master branch is failing

From the Travis CI logs of the failures for what seems to be all supported versions of Python....

The command "openssl aes-256-cbc -K $encrypted_c3e85137836d_key -iv $encrypted_c3e85137836d_iv -in .env.enc -out .env -d" failed and exited with 1 during .

Add Get Dialog File Call in the Dialog API

The Dialog API Supports the call to Download the Dialog File(GET /v1/dialogs/{dialog_id}). Not Sure if this call was not available earlier.

I am talking about the Number 6 in here. I can have this assigned to myself if you guys dont mind!

Unicode and Language Translation

Not sure if this is an issue or not but when passing in unicode to the language detection needs to be encoded. ie. if data is unicode then it has to be encoded

txt = data.encode("utf-8", "replace")

and the encoded text passed into identify

language_translation.identify(txt)

whereas
language_translation.identify(data)
throws a unicode error.

Add Alchemy emotion call

Create examples and verify implementation of AlchemyData News

Master now contains an implementation for AlchemyData News, but this needs some testing and examples.

We should start by showing examples of basic usage in /examples/alchemy_data_news_v1.py, and verifying that the SDK works correctly for these examples. There are examples to try at http://querybuilder.alchemyapi.com/builder (under the Examples tab). A high-quota api key should be used, since these can use a lot of transactions.

Version parameter not included in Document Conversion Service

The current version of the python-sdk doesn't add the required version parameter to the URL:
https://gateway.watsonplatform.net/document-conversion/api/v1/convert_document?version=2016-02-10

The Exception being thrown looks like the following:

Traceback (most recent call last):
File "document_conversion_v1.py", line 12, in
print(json.dumps(document_conversion.convert_document(document=document, config=config), indent=2))
File "/usr/local/lib/python2.7/dist-packages/watson_developer_cloud/document_conversion_v1.py", line 36, in convert_document
return self.request(method='POST', url='/v1/convert_document', files=files, accept_json=True)
File "/usr/local/lib/python2.7/dist-packages/watson_developer_cloud/watson_developer_cloud_service.py", line 250, in request
raise WatsonException('Error: ' + error_message + ', Code: ' + str(response.status_code))
watson_developer_cloud.watson_developer_cloud_service.WatsonException: Error: You must include a version parameter., Code: 400

Add Tone Analyzer v3

Instantiating ToneAnalyzerV3Beta should have default version value

The example code suggests you can instantiate a tone analyzer by passing in just 2 arguments, as follows:

ta = ToneAnalyzerV3Beta(username='yourusername',password='yourpassword')

However, this is not right -- you have to pass in a version argument as well, thusly:

tone_analyzer = ToneAnalyzerV3Beta(version='2016-02-11', username='yourusername', password='yourpassword')

Since it doesnt seem obvious what the right version argument should be, maybe this should default to ToneAnalyzerV3Beta.latest_version?

AlchemyNewsDataV1 does not support pagination

Will currently throw the following error if you try to add the "next" query parameter to retrieve the next page.

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/watson_developer_cloud/alchemy_data_news_v1.py", line 39, in get_news_documents
    return self._alchemy_html_request(method_url='/data/GetNews', method='GET', params=params)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/watson_developer_cloud/watson_developer_cloud_service.py", line 177, in _alchemy_html_request
    accept_json=True)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/watson_developer_cloud/watson_developer_cloud_service.py", line 245, in request
    raise WatsonException('Error: ' + error_message)
WatsonException: Error: Invalid field = 'next'

I've identified the bug in the code and will place a PR soon. Adding this issue for documentation/tracking purposes.

Add more documentation

I ran pylint and got:

We probably want to focus on documentation and test cases

Update the authorization service to use the service host.

Use https://stream.watsonplatform.net/ for streaming urls or https://gateway.watsonplatform.net otherwise

Include optional parameters in AlchemyVision examples

There are a couple optional parameters to some AlchemyVision APIs that would be nice to include in the examples. There's knowledgeGraph=(0|1) for GetRankedImageFaceTags and GetRankedImageKeywords, and forceShowAll=(0|1) for GetRankedImageKeywords.

Switch Speech to Text to use WebSockets

HI @daniel-bolanos I was thinking we can provide support for WebSockets in the sdk by adding an existing library and handling the authorization using the authorization service.
What do you think?

We should:

• Find a websocket library
• Design a listener/observer pattern to model websockets

Parse VCAP_SERVICES to select the credentials based on the selected plan

Error imports in examples is incorrect

Link to responses package is a 404

From readme dependencies:
responses link goes to https://github.com/watson-developer-cloud/python-sdk/blob/master/for%20testing

Add Sample Output to Examples in Comments

Hello,
I was walking through the examples today (i.e. https://github.com/watson-developer-cloud/python-sdk/blob/master/examples/relationship_extraction_v1_beta.py ) and noticed that they don't give a lot of insight into what's going on in the example, since it's just an import/invoke.

Since all of these scripts require login credentials I think it'd be a nice addition if the output from the print statements at the end of the script could be followed by comments showing what the output would be. For example:

print(["Hi!" for i in range(0, 3)])
"""
['Hi!', 'Hi!', 'Hi!']
"""

Use multiple exception classes

When catching exceptions, users probably want to see the exception type and write code to handle it.

Example:

import sys

try:
    f = open('myfile.txt')
    s = f.readline()
    i = int(s.strip())
except IOError as e:
    print "I/O error({0}): {1}".format(e.errno, e.strerror)
except ValueError:
    print "Could not convert data to an integer."
except:
    print "Unexpected error:", sys.exc_info()[0]
    raise

How to make continuous transmission in speech to text

The transcribe function stops once it encounters a pause. How do I extend the time? https://www.ibm.com/smarterplanet/us/en/ibmwatson/developercloud/doc/speech-to-text/input.shtml#timeouts
In this link I believe I need to play with continuous transmission but I'm unsure how to fit this in this code. Can you please help

Add support for Concept Insights

move resources to the test folder

Remove question and answer

Add missing services

The first release came with just a few services. We need to add the rest of them.

• AlchemyVision
• AlchemyLanguage
• AlchemyDataNews
• Retrieve and Rank
• Document Conversion
• Concept Insights

Change Travis to not run integration tests on pull requests

Currently contributor's pull requests fail because the integration tests cannot decrypt the credentials. This should be fixed as in the Node SDK project.

Installation should not depend on pypandoc

Attempting to install without the pypandoc module installed results in the error:

FileNotFoundError: [Errno 2] No such file or directory: 'README.md'
warning: pypandoc module not found, could not convert Markdown to RST

sst and tts examples are out of date

@jsstylos:
The model voice names have changed in
https://github.com/watson-developer-cloud/python-sdk/blob/master/examples/speech_to_text_v1.py
https://github.com/watson-developer-cloud/python-sdk/blob/master/examples/text_to_speech_v1.py

Support Visual Recognition v2-beta

Simplify import statement for NLC

change

from watson_developer_cloud import NaturalLanguageClassifierV1 as NaturalLanguageClassifier

to

from watson_developer_cloud import NaturalLanguageClassifierV1

to reduce text clutter in API reference

Test Document Conversion media type detection

Document conversion issues with non JSON (ANSWER UNITS) response

If I try with a MWE with a conversion to NORMALIZED_TEXT or NORMALIZED_HTML,

with open('sample.pdf', 'rb') as document:
    watson_config = {'conversion_target': DocumentConversionV1.NORMALIZED_TEXT}
    print(json.dumps(document_conversion.convert_document(document=document, config=watson_config)))

watson_developer_cloud_service.py (line 233):

response_json = response.json()

throws an errorJSONDecodeError: Expecting value: line 1 column 1 (char 0), i.e. the response doesn't conform to JSON, as it's either text or HTML, therefore I believe it should be contemplated, as I think is the original intention of return response in line 243.

Generally broken get_news_documents method

def get_news_documents(self, start, end, max_results=10, query_fields=None, return_fields=None, time_slice=None):
    if isinstance(return_fields, list):
        return_fields = ','.join(return_fields)
    params = {'start': start,
              'end': end,
              'maxResults': max_results,
              'return': return_fields,
              'timeSlice': time_slice}
    if isinstance(query_fields, dict):
        for key in query_fields:
            params[key if key.startswith('q.') else 'q.' + key] = query_fields[key]
    return self._alchemy_html_request(method_url='/data/GetNews', method='GET', params=params)

This method incorrectly handles the query_fields parameter by assuming all query_fields are prefixed with the letter 'q'. Query fields like dedup and dedupThreshold cannot be passed into this function since they should not be prefixed with a q. This issue is also responsible for a PR I made with respect to another issue I created around the use of pagination #53

Allow voice recording for the Speech to Text service

I think we should also allow users to record audio for Speech to Text, not just use a file.

Please give me some ideas.

Can't import alchemy-related classes from sdk

After using 'pip install --upgrade watson-developer-cloud' in my virtualenv

from watson_developer_cloud import AlchemyLanguageV1

results in 'ImportError: cannot import name 'AlchemyLanguageV1'

but

from watson_developer_cloud import SpeechToTextV1 

works fine.

This is the case for both python 2.7 and python 3.5.

List the required dependencies along with their licenses

Dependency	License	url
requests	Apache 2.0	http://docs.python-requests.org/en/latest/

Dict comprehension incompatible with python 2.6

Import statement results in the following exception:

Traceback (most recent call last): File "/home/chakravr/workspaces/bluegoat-experimental/bluegoat-raas-experimental/chakravr-experiments/src/main/scripts/bluemix_scripts/run_raas_csf_experiment.py", line 15, in <module> from watson_developer_cloud import RetrieveAndRankV1, WatsonException File "/usr/lib/python2.6/site-packages/watson_developer_cloud/__init__.py", line 15, in <module> from .watson_developer_cloud_service import WatsonDeveloperCloudService File "/usr/lib/python2.6/site-packages/watson_developer_cloud/watson_developer_cloud_service.py", line 52 return {k: v for k, v in dictionary.items() if v is not None} ^ SyntaxError: invalid syntax

Syntax appears to be incompatible with python 2.6: http://stackoverflow.com/questions/1747817/create-a-dictionary-with-list-comprehension-in-python. Works fine with python 2.7

Add read-the-docs support

We can easily add read-the-docs as our documentation site for the python sdk. I already did some work in conf.py. Our documentation is in markdown so we will have to do some conversions.

[relationship-extraction] Allow developers to choose the return type ('json', 'xml')

This is not an issue, I just wanted to bring this up as an enchancement: If you see the file "relationship_extraction_v1_beta.py", you can see that "'rt': 'xml'" is hardcoded.

This means that this service will ALWAYS return "XML" as its format. However, I prefer using JSON, and other developers have their own preferences, so I propose a solution: Just let the developer choose which Data Type they prefer! We can do this by adding "rt" or "return_type" as one of the parameters passed into the "extract" function of the class.

Update the dialog example to use the GA version of the service

Steps to get credentials in readme are for Natural Language Classifier

Need to make the Natural Language Classifier an example service tile to select.

Update travis AlchemyAPI key with a higher limit

Integration tests of AlchemyAPI Data News require a high-limit API key

JSON from get_dialog in Dialog API throws an exception when trying to import back into the dialog

So I faced this issue according to the last comment on #31. I got around it by removing accept_json parameter from the request and extracting raw content from the request object.

def get_dialog(dialog_id, accept='application/wds+json'):
        #accept_json = False
        headers = {'accept' : accept}
        auth = HTTPBasicAuth(config.username, config.password)
        return request(method='GET', url='https://gateway.watsonplatform.net/dialog/api/v1/dialogs/{0}'.format(dialog_id), auth=auth, headers=headers).content

I can submit a pull request if you approve these changes.

WatsonException should show description text

I have the maximum number of NLC classifiers. When I try to allocate another one using a CURL call I see the following error

{
  "code" : 400,
  "error" : "Entitlement error",
  "description" : "This user or service instance has the maximum number of classifiers."
}

If I do the same thing with the watson_developer_cloud Python sdk, the following exception is displayed.

WatsonException: Error: Entitlement error, Code: 400

It would be really helpful if the Python sdk also displayed the description text, since offhand I don't know what an "Entitlement error" is.

It looks like this information has to be fished out of the response local variable in WatsonDeveloperCloudService.request in watson_developer_cloud_service.py.

Improve the error message for 401 responses

Currently if we don't use valid credentials we get:

watson_developer_cloud.watson_developer_cloud_service.WatsonException: Error: 
<HTML><HEAD><meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
<TITLE>Watson Error</TITLE>
</HEAD>
<BODY>
<HR>
<p>Invalid access to resource - /speech-to-text/api/watson-auth-token</p>
<p>User access not Authorized.</p>
<p>Gateway Error Code : ERCD50-LDAP-NODN-ERR</p>
<p>Unable to communicate with Watson.</p>
<p>Request URL : /speech-to-text/api/watson-auth-token</p>
<p>Error Id : 847403581</p><p>Date-Time : 2015-10-15T12:35:53-05:00</p>
</BODY>
</HTML>, Code: 401

As with the node-sdk and java-sdk we should return
"Unauthorized: Access is denied due to invalid credentials"

[text-to-speech] Add support for customizations

Add support for customizations. See: https://watson-api-explorer.mybluemix.net/apis/text-to-speech-v1
There are new endpoints like /v1/pronunciation and /v1/customizations

The python-sdk is incompatible with a Permuation of Korean Laptop / Korean Code Page and Python 3

Although it is ok with a Korean Laptop / Korean Code Page and Python 2.
Talking to my Korean colleagues I have learnt that this is quire common for pip packages, which is why they stick to Python 2. The strange thing is that the other packages they installed (django and request), were both OK.
Short term fix is a note in the readme. Long term fix is to modify the code that breaks using the Korean Python 3 permutation.

Language Translation missing methods

The Python SDK is missing methods for the following Language Translation methods:
List identifiable languages
GET /v2/identifiable_languages

Create model
POST /v2/models

Delete model
DELETE /v2/models/{model_id}

Get model details
GET /v2/models/{model_id}

API Reference for Document Conversion incorrect: 400 error because no version in request

https://www.ibm.com/smarterplanet/us/en/ibmwatson/developercloud/document-conversion/api/v1/?python#convert-document

Is inconsistent with what's actually in the SDK example.

document_conversion = DocumentConversion(username='YOUR SERVICE USERNAME',
password='YOUR SERVICE PASSWORD',
version='2016-02-09')

The SDK actually requires you to put in version date(?) for object instantiation.

WatsonException drops description

The WatsonException doesn't return the description property from the service error message. For example, with not enough training data in the Natural Language Classifier, this is returned:

raise WatsonException('Error: ' + error_message + ', Code: ' + str(response.status_code))
watson_developer_cloud.watson_developer_cloud_service.WatsonException: Error: Data too small, Code: 400

• The description is missing from this.
• Also might want to put the message, code, and description on separate lines
• Also, to match service, labels should be lower case. For example,

{
   "description": "No training data is specified. The form field 'training_data' should contain the training CSV and 'training_metadata' should contain a JSON metadata string.",
   "error": "Missing training data",
   "code": 400
}

Dialog update_profile does not work

The update_profile method for DialogV1 does not work. I have found the issue is that client_id is not passed as it should be along with the name_values according to the API explorer:

https://watson-api-explorer.mybluemix.net/apis/dialog-v1#!/Content/putAppFile

Please update this method and also if possible add some examples for the dialogV1 class.

Add integration test.