urschrei/pyzotero

Pyzotero: a Python client for the Zotero API

Pyzotero: An API Client for the Zotero API

Quickstart

1. pip install pyzotero or conda config --add channels conda-forge && conda install pyzotero
2. You'll need the ID of the personal or group library you want to access:
   • Your personal library ID is available here, in the section Your userID for use in API calls
   • For group libraries, the ID can be found by opening the group's page: https://www.zotero.org/groups/groupname, and hovering over the group settings link. The ID is the integer after /groups/
3. You'll also need^† to get an API key here
4. Are you accessing your own Zotero library? library_type is 'user'
5. Are you accessing a shared group library? library_type is 'group'.

Then:

from pyzotero import zotero
zot = zotero.Zotero(library_id, library_type, api_key)
items = zot.top(limit=5)
# we've retrieved the latest five top-level items in our library
# we can print each item's item type and ID
for item in items:
    print('Item: %s | Key: %s' % (item['data']['itemType'], item['data']['key']))

Documentation

Full documentation of available Pyzotero methods, code examples, and sample output is available on Read The Docs.

Installation

• Using pip: pip install pyzotero (it's available as a wheel, and is tested on Python 3.7 and up)
• Using Anaconda:conda config --add channels conda-forge && conda install pyzotero
• From a local clone, if you wish to install Pyzotero from a specific branch:

Example:

git clone git://github.com/urschrei/pyzotero.git
cd pyzotero
git checkout dev
pip install .

Testing

Run pytest . from the top-level directory.

Issues

The latest commits can be found on the dev branch, although new features are currently rare. If you encounter an error, please open an issue.

Pull Requests

Pull requests are welcomed. Please read the contribution guidelines. In particular, please base your PR on the dev branch.

Versioning

As of v1.0.0, Pyzotero is versioned according to Semver; version increments are performed as follows:

1. MAJOR version will increment with incompatible API changes,
2. MINOR version will increment when functionality is added in a backwards-compatible manner, and
3. PATCH version will increment with backwards-compatible bug fixes.

Citation

Pyzotero has a DOI:

You may also cite Pyzotero using CITATION.cff.
A sample citation (APA 6th edition) might look like:

Stephan Hügel, The Pyzotero Authors (2019, May 18). urschrei/pyzotero: Version v1.3.15. http://doi.org/10.5281/zenodo.2917290

License

Pyzotero is licensed under the MIT license. See license.txt for details.

† This isn't strictly true: you only need an API key for personal libraries and non-public group libraries.

pyzotero's Issues

Add support for full-text content

Retrieving is probably a greater priority than setting:

https://www.zotero.org/support/dev/web_api/v3/fulltext_content

'self' entry missing from links

With version 1.1.2, I am getting the error below during a request to 'everything'.

A little investigation shows that there is indeed no 'self' pagination entry in 'links'. I think one can produce this from self.request.url, but I wonder if there's an easier way.

From the v3 Zotero API, it does not seem there is a 'self' entry in the pagination links:

When the total number of results matched by a read request is greater than the current limit, the API
will include pagination links in the HTTP Link header. Possible values are rel=first, rel=prev, rel=next,
and rel=last. For some requests, the header may also include a rel=alternate link for the relevant
page on the Zotero website.

So, I'm not sure how this is/was supposed to work. My application (zot_bib_web) runs fine with a previous version, 0.10.2.

File "./zot.py", line 185, in retrieve_bib
items = zot.everything(zot.collection_items(collection, content=content, style=style, order=order_by, sort=sort_order))
File "/Library/Python/2.7/site-packages/pyzotero/zotero.py", line 544, in everything
while not self.links['self'] == self.links['last']:
KeyError: u'self'

comments

So it might just be me, and the end of a long day, but I'm struggling a bit with the API.

Can you perhaps include a hello world that just lists entries in a library and prints out the titles?

More specifically, I guess I'm looking for really straightforward methods like:

• library.items - list of item dictionaries, maybe with optional filter parameter
• library.item - item by id
• library.collections - list of collections

So that I can just do:

for item in library.items:
    print item.title

Or:

library.item(125429).title

add item with a parent

When attaching a file usin pyzotero API, one can specify an ID for a parent,

attachment_simple(files[, parentid])

I'd like to also allow for

create_items(items)

to have an optional parent id,

create_items(items[, parentid])

Is this already included in pyzotero? If not, is it possible within zotero API?

implementing the API request to delete tags

I am building a library to administer and query tags in a sophisticated way using pyzotero. I have a sanitize method which should change tags with upper case letters into lowercase versions. Unfortunately Zotero somehow memorizes a tag that was used once even if no item in the library has this tag. Since Zotero also does not distinguish between lowercase and uppercase tags the original uppercase tag is restored once I add its lowercase version. Thus it is not possible to change tags on the basis of capitalization and it is also not possible to remove all uppercase versions first and then add the lowercase versions.

I found out Zotero's web api does have a tag request for deleting tags from the library and not only removing them form items (https://www.zotero.org/support/dev/web_api/v3/write_requests#deleting_multiple_tags). Would you consider to implement this request type?

zlib.collection_items: only retrieving first 25 items

hey Stephan, thank you so much for pyzotero!
I wish I could fix problems myself and contribute but I am just a rookie so the best I can do is to report this issue.
So, unfortunately, I have not been able to get all my collections' items, which are 76, but rather just 25 (which is exactly the same amount of items shown per page in Zotero, do you think this has anything to do with it?)

Here's what I did:

zlib = zotero.Zotero('1677289','user','4g5SrCY6IUW1x6LPUfXrzVGe') # My own user ID / api key
items = zlib.collection_items('THA7JMA5') # My collection: https://www.zotero.org/silviaegt/items/collectionKey/THA7JMA5
itemlist = []
for item in items:
    itemlist.append(item['data']['title'])
print(len(itemlist))

And here's my complete code for further reference

Can't use flat dicts as the container for items due to multiple occurrences of e.g. 'Contributor'

And this is going to be a pain to deal with, since the API call simply returns multiple instances of the creator tag.

Updating item returns "Invalid request, probably due to unsupported parameters"

Attempting to update an attachment with update_item() returns an Invalid Request. The only metadata I'm changing are the attachment's tags.

KeyError when using Zotero.make_iter

I'm running into a KeyError: u'self' when using Zotero.make_iter, specifically on https://github.com/urschrei/pyzotero/blob/v1.1.14/pyzotero/zotero.py#L616 (for pyzotero v 1.1.14). I think the problem is related to 'self' entry missing from links · Issue #43 · urschrei/pyzotero.

Here's a Jupyter notebook demonstrating the problem:

https://gist.github.com/404573708a805861edb5

In the notebook, I also write zot.all_items as a lazy generator of all the items. From what I understand zot.everything grabs all the items on call rather than do so lazily. (Interested in your thoughts on such an approach.)

Problems related with "No JSON object could be decoded" in Python 2.7.13

Hello.

Firstly I want to say that I new with Zotero and Pyzotero so I want to know exaclty if this is my fault or not.

I'm having an error using Pyzotero when I want to retrieve information in bibtext format. If I do the following:

item = zot.item(item_key, format='bibtex')

where:

• zot: A zotero instance

• item_key: A key of the item that I want to recover from API.

The following error raises:

raise ValueError("No JSON object could be decoded")

I seems that error is produced when Pyzotero uses "requests" library. I think it could be my Python version (Python 2.7.13).

If I call to method without "format" the library works well. In a Ubuntu 14.04 machine (Using older version of Python 2) it works without problems.

Any ideas?

Thanks in advance.

all_top() raises 403 error (not authorized) from zotero server

Hi,
when I try to get all top Items, I get a http 403 error, telling me:
"UserNotAuthorized: Not authorized: (403)
URL: https://api.zotero.org/users/######/items/top?content=json&start=50
With .items() it works flawlessly for the first 50 items...
Best regards,
Jacob

zot.children(item_id) fails in python 2.7

Here is the code that reproduces the error for me:

from pyzotero import zotero
user_id='blah'
user_key='blah'
item_id='3D7R2SMR'
zot = zotero.Zotero(user_id, user_key)
zot.children(item_id)

Here is the relevant portion of the traceback

/home/idm/Envs/tozero/lib/python2.7/site-packages/pyzotero/zotero.pyc in wrapped_f(self, *args, **kwargs)
    162             from each entry, then to feedparser, then to _process_content                    
    163             """                                                                              
--> 164             orig_func = func(self, *args, **kwargs)                                          
    165             retrieved = self._retrieve_data(orig_func)                                       
    166             # get etags from the response                                                    


/home/idm/Envs/tozero/lib/python2.7/site-packages/pyzotero/zotero.pyc in children(self, item)
    227         """ Get a specific item's child items                                        
    228         """                                                                          
--> 229         query_string = '/users/{u}/items/{i}/children'.format(i = item)              
    230         return self._build_query(query_string)                                       
    231                                                                                      

KeyError: 'u' 

It appears str.format raises KeyError when it is not passed all variables as a mapping. _build_query, which is supposed to call str.format to set {u}, is never reached because the KeyError is raised first.

Item creation failing

I'm writing a script to batch additions of items, and I'm getting an invalid request error:

Traceback (most recent call last): File "csvin.py", line 65, in <module> result = server.create_items(items) File "/usr/local/lib/python2.7/dist-packages/pyzotero/zotero.py", line 551, in create_items self._error_handler(req, error) File "/usr/local/lib/python2.7/dist-packages/pyzotero/zotero.py", line 740, in _error_handler (req.get_full_url(), req.get_method(), req.get_data()) pyzotero.zotero_errors.UnsupportedParams: Invalid request, probably due to unsupported parameters: https://api.zotero.org/users/5770/items?key=[key] POST {"items": [{"numPages": "113 \u043b.", "itemType": "manuscript", "language": "", "title": "\u0420\u0443\u043a\u043e\u043f\u0438\u0441\u044c \u0413\u0430\u044f\u0437\u0430 \u0418\u0441\u0445\u0430\u043a\u044b\u0439 \u201c\u041e\u0441\u043d\u043e\u0432\u044b \u043d\u0430\u0446\u0438\u043e\u043d\u0430\u043b\u044c\u043d\u043e\u0439 \u0436\u0438\u0437\u043d\u0438 \u0438 \u043a\u0443\u043b\u044c\u0442\u0443\u0440\u044b \u0434\u043b\u044f \u043c\u0443\u0441\u0443\u043b\u044c\u043c\u0430\u043d\u201d \u0412\u043d\u0443\u0442\u0435\u043d\u043d\u0435\u0439 \u0420\u043e\u0441\u0441\u0438\u0438 \u043d\u0430\u0446\u0438\u043e\u043d\u0430\u043b\u044c\u043d\u043e\u0439 \u0430\u0432\u0442\u043e\u043d\u043e\u043c\u0438\u0438", "url": "", "type": "\u0440\u0443\u043a\u043e\u043f\u0438\u0441\u044c", "manuscriptType": "", "rights": "", "libraryCatalog": "", "shortTitle": "", "extra": "", "accessDate": "", "tags": [], "place": "", "notes": [], "date": "15 \u0438\u044e\u043d\u044f 1914 \u0433.; 25 \u0438\u044e\u043d\u044f 1914 \u0433.", "abstractNote": "", "creators": [{"lastName": "", "creatorType": "author", "firstName": ""}], "archiveLocation": "\u0444. \u0420-2461, \u043e\u043f. 1 \u0434. 1", "archive": "\u041d\u0410 \u0420\u0422", "callNumber": ""}]}

The surrounding logic just pulls some data from a CSV and fills in the item templates: https://gist.github.com/1086267

No longer any need to decode json (at least in some functions)

It looks like _retrieve_data is now returning the decoded json object instead of a string (at least for some API methods). One example is item_types(). i.e., calling Zotero.item_types() throws an exception because you're trying to json decode something that's already been decoded. Perhaps the response content-type has changed for these items, or perhaps you changed _retrieve_data to automatically decode json responses, but forgot to update the methods that call it?

Problems creating items

This is probably a newbie code error. I tried what seemed like a simple hello world, and still get a KeyError from check_items():

server = zotero.Zotero(user_id, user_key)
book = server.item_template('book')
book[u'title'] = u'Hello World'
server.check_items([book])
server.create_items([book])

This raises a KeyError:
Traceback (most recent call last):
File "dump_schema.py", line 27, in
server.check_items(book)
File "/Library/Python/2.6/site-packages/pyzotero/zotero.py", line 676, in check_items
if self.templates['item_fields'] and not
KeyError: 'item_fields'

I've tried without the 'Hello World' assignment (thus passing a blank template through, unmodified), and get the same exception. OSX 10.6.8, python 2.6, pyzotero 0.9.5.

Read APIs and the hello world from the docs seem to work fine.

easy_install assumes pyzotero 0.9.51 is newest version

I installed pyzotero as a dependency of zot4rst and it installed the 0.9.51 version what is probably not the right thing, is it?

I followed the instructions on https://bitbucket.org/egh/zot4rst and also an easy_intsall -U pyzotero assumes that 0.9.51 is the newest version.

If this is not intended a version jump to 1.0.0 or to 0.9.91 would be great.

p. s. I am running ubutuntu 64 bit system (but I don't think that this matters here).

If there are any questions feel free to send me a message :)

Creating new item fails, gives error

Running the following code keeps giving me the error that is shown. The 400 error code appears to be indicating that writing to the library is not allowed, but this is for my personal Zotero library and I have allowed write permissions. What could be causing this error?

zlib = zotero.Zotero(<ID>, 'user', <API KEY>)

newitem = zlib.item_template('book')
newitem['creators'][0]['firstName'] = 'Monty'
newitem['creators'][0]['lastName'] = 'Cantsin'
newitem['title'] = 'Maris Kundzins: A Life'
newitem['relations'] = ''

resp = zlib.create_items([newitem])

---

HTTPError Traceback (most recent call last)
/Users/[USER]/anaconda/lib/python3.5/site-packages/pyzotero/zotero.py in create_items(self, payload)
952 try:
--> 953 req.raise_for_status()
954 except requests.exceptions.HTTPError:

/Users/[USER]/anaconda/lib/python3.5/site-packages/requests/models.py in raise_for_status(self)
861 if http_error_msg:
--> 862 raise HTTPError(http_error_msg, response=self)
863

HTTPError: 400 Client Error: Bad Request for url: https://api.zotero.org/users/[ID]/items?key=[API KEY]

During handling of the above exception, another exception occurred:

UnsupportedParams Traceback (most recent call last)
in ()
4 newitem['relations'] = ''
5
----> 6 resp = zlib.create_items([newitem])

/Users/[USER]/anaconda/lib/python3.5/site-packages/pyzotero/zotero.py in create_items(self, payload)
953 req.raise_for_status()
954 except requests.exceptions.HTTPError:
--> 955 error_handler(req)
956 return self._json_processor(feedparser.parse(data))
957

/Users/rclement/anaconda/lib/python3.5/site-packages/pyzotero/zotero.py in error_handler(req)
1236 error_handler(new_req)
1237 else:
-> 1238 raise error_codes.get(req.status_code)(err_msg(req))
1239 else:
1240 raise ze.HTTPError(err_msg(req))

UnsupportedParams:
Code: 400
URL: https://api.zotero.org/users/[ID]/items?key=[API KEY]
Method: POST
Response: Uploaded data must be a JSON array

---

Basic "hello world" code broken

I ran the following code (taken from http://pyzotero.readthedocs.org/en/latest/index.html#hello-world):

from zoteroconf import user_id, user_key

# http://pyzotero.readthedocs.org/en/latest/index.html

from pyzotero import zotero
zot = zotero.Zotero(user_id, user_key)
items = zot.items()
for item in items:
    print 'Author: %s | Title: %s' % (item['creators'][0]['lastName'], item['title'])

with pyzotero 0.9.3 and got the following error:

Traceback (most recent call last):
  File "pyzotero_test.py", line 7, in <module>
    items = zot.items()
  File "/home/ubuntu/.virtualenvs/zoterotest/lib/python2.7/site-packages/pyzotero/zotero.py", line 102, in wrapped_f
    return self._process_content(parsed)
  File "/home/ubuntu/.virtualenvs/zoterotest/lib/python2.7/site-packages/pyzotero/zotero.py", line 552, in _process_content
    return self._standard_items(content)
  File "/home/ubuntu/.virtualenvs/zoterotest/lib/python2.7/site-packages/pyzotero/zotero.py", line 569, in _standard_items
    for e in retrieved.entries]
  File "/usr/lib/python2.7/json/__init__.py", line 326, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 360, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 378, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

When I run the code with a version of pyzotero that worked for me in November 2011 (0.9.1), I get the following error:

Traceback (most recent call last):
  File "pyzotero_test.py", line 9, in <module>
    print 'Author: %s | Title: %s' % (item['creators'][0]['lastName'], item['title'])
TypeError: string indices must be integers

I'm using Python 2.7.1+ on ubuntu. I get the same errors running Python 2.7 on OS X.

Thanks!

Parentheses error in readme sample code

The sample code produces an error due to misplaced parentheses.

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-6-b1231b082426> in <module>()
      9 # we can print each item's item type and ID
     10 for item in items:
---> 11     print('Item Type: %s | Key: %s') % (item['data']['itemType'], item['data']['key'])

TypeError: unsupported operand type(s) for %: 'NoneType' and 'tuple'

Replace:

print('Item: %s | Key: %s') % (item['data']['itemType'], item['data']['key'])

with

print('Item Type: %s | Key: %s' % (item['data']['itemType'], item['data']['key']))

Allow 'path' as item key in check_items()?

Attempting to update an attachment with update_item() throws an InvalidItemFields via to_check(), because 'path' is not included in the template set:

template = template | set([
'tags',
'notes',
'itemType',
'creators',
'mimeType',
'linkMode',
'note',
'charset',
'dateAdded',
'version',
'collections',
'dateModified',
'relations',
# attachment items
'parentItem',
'mtime',
'contentType',
'md5',
'filename'])

Could this be changed or are there any reasons not to include 'path' in template...like some security issues or smth?

Thanks anyway for the awesome module!

PS: I currently work around this issue by replacing to_check by a modified method that includes path in template (this allows me to update an attachment's path so the attachment links to a different file than before).

everything() returns a 403

zot.top() functions fine, but zot.everything(zot.top()) generates a 403 using a private individual library. Is it possible that the API key is being dropped in the everything call? In my script output I see URL: https://api.zotero.org/users//items/top?limit=5&content=json&start=5 with no key appended.

KeyError: 'content' when trying to add parameters to a request

Maybe user error? e.g.

zot.add_parameters(content='bib', style='mla')
collection = zot.collection(<collectionID>)

I get the error

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-60-5383e09cce1e> in <module>()
      1 zot.add_parameters(content='bib', style='mla')
----> 2 collection = zot.collection('XXXXXXX')

/usr/local/lib/python3.5/site-packages/pyzotero/zotero.py in wrapped_f(self, *args, **kwargs)
    174             processor = self.processors.get(content)
    175             # process the content correctly with a custom rule
--> 176             return processor(parsed)
    177         # it's binary, so return raw content
    178         elif fmt != 'json':

/usr/local/lib/python3.5/site-packages/pyzotero/zotero.py in _bib_processor(self, retrieved)
    714         items = []
    715         for bib in retrieved.entries:
--> 716             items.append(bib['content'][0]['value'])
    717         self.url_params = None
    718         return items

/usr/local/lib/python3.5/site-packages/feedparser.py in __getitem__(self, key)
    355             elif dict.__contains__(self, realkey):
    356                 return dict.__getitem__(self, realkey)
--> 357         return dict.__getitem__(self, key)
    358 
    359     def __contains__(self, key):

KeyError: 'content'

Zotero response format changed?

Hi Stephan,

Neil from KCL here!
Been using this fine for a while now only just noticed that it was you who created it!
Hope all is good.

Suddenly I'm getting a lot of Unicode Errors when the incoming string is sent to the element tree for parsing.

Thought you should know
I think this update may have played minor havoc with the parsing of the response...
https://groups.google.com/forum/?fromgroups#!topic/zotero-dev/NN4I5-DVOoY

Can't save HTML file attachments

Could support for arbitrary files, or at least HTML attachments be added?

Content type is contentType': u'text/html', and calling the "file" method on this attachment item produces errors that vary with python version, e.g. "UnicodeEncodeError: 'ascii' codec can't encode characters in position 11-12: ordinal not in range(128)", or a ValueError in the latest PyZotero.

Would it make sense to fail gracefully or throw a documented exception?

File "./zot.py", line 1584, in dumpFiles f.write(self.zot.file(item.key)) File "/usr/local/lib/python2.7/site-packages/pyzotero/zotero.py", line 187, in wrapped_f return retrieved.json() File "/usr/local/lib/python2.7/site-packages/requests/models.py", line 819, in json return json.loads(self.text, **kwargs) File "/usr/local/Cellar/python/2.7.10/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 338, in loads return _default_decoder.decode(s) File "/usr/local/Cellar/python/2.7.10/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 366, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/local/Cellar/python/2.7.10/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 384, in raw_decode raise ValueError("No JSON object could be decoded")

pyzotero error

Hi , I was able to use
http://pyzotero.readthedocs.org/en/latest/
with my library but when I changed library_type to 'group' and add a [search] parameter it return

File "pyzotero\zotero.py", line 125, in wrapped_f
retrieved = self._retrieve_data(func(self, *args))
TypeError: top() takes exactly 1 argument (2 given)

Any hints?

Problem with os x

Hi,

On os x, when I do from pyzotero import zotero I get :

Fatal Python error: PyThreadState_Get: no current thread
Abort trap: 6

and python quits.

Any explication ? Thanks.

Unauthenticated access to public libraries

The API supports access to public user and group libraries without including an API key. Not sure how this could best be implemented in pyzotero. Maybe by reorganizing the Zotero instance around the idea of a libraryID (group or user) rather than a userID, e.g. Zotero(libraryID, libraryType[, apiKey])

unexpected keyword argument 'payload' in update_collection and set_fulltext

Using the following (from the documentation),

c = zot.collections()
c[0]['name'] = 'Whither Digital Humanities?'
zot.update_collection(c[0])

gives,

TypeError: request() got an unexpected keyword argument 'payload'

This also happens with the new set_fulltext method. Versions: Python 3.6.0, requests 2.13.0, pyzotero 1.2.1. The cause seems to be here, using payload=json.dumps(payload)), where requests.put expects data= or json=.

I also noticed Content-Type is not being set to application/json in these methods- changing to data= may not be enough.

get number of items in library.

Hi,
it may sound stupid - but I always get only the first 50 items of my library.
In order to access all of them, i need to know the total number of items in my library in order to create an iteration loop that accesses all items ...
or how do I know how often I have to iterate with an offset of n*50?
Thanks,
Jacob

Error when trying to add attachment that already exists

If an attachment already exists in the library, the response from the API does not include a 'prefix' element, and therefore authdata['prefix'] on line 774 in zotero.py throws an exception.

Unexpected behavior for check_items (with respect to the doc)

In the doc, it is said :

Before calling this method, the use of check_items() is encouraged, in order to confirm that the item to be created contains only valid fields.

and then an example is provided

template = zot.item_template('book')
template['creators'][0]['firstName'] = 'Monty'
template['creators'][0]['lastName'] = 'Cantsin'
template['title'] = 'Maris Kundzins: A Life'
resp = zot.create_items([template])

If we modify this example to follow the recommendation of using check_items() as follow

template = zot.item_template('book')
template['creators'][0]['firstName'] = 'Monty'
template['creators'][0]['lastName'] = 'Cantsin'
template['title'] = 'Maris Kundzins: A Life'
zot.check_items([template])
resp = zot.create_items([template])

it crashes with

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/oreilly/.local/lib/python3.4/site-packages/pyzotero/zotero.py", line 806, in check_items
    to_check = set(i for i in list(item['data'].keys()))
KeyError: 'data'

It is not obvious how to use check_items() in conjunction with item_template() since these templates miss a bunch of fields that seems to be expected (fields that are there when used it in conjunction with items() for example).

Notes not retrieved from server

Hi,

I've been trying to use pyzotero to interact with the zotero server and automate a few actions. So far this has been going fine but I seem to have hit a roadblock with notes.
Checking the documentation and the code there aren't many but a few references seem to point out that notes should also be fetched from the server.

I've tried different things including the everything method but the returned list/dicts doesn't contain notes (tab next to 'tags'). I do get a note when it is associated with a file (i.e. PDF) but not when it's associated with an item (parent of PDF and Notes - don't know the exact term).

So the question is, how do I fetch these notes?

broken link from rtd 'Show on GitHub' to github

http://pyzotero.readthedocs.org/en/latest/

'Show on GitHub' and 'Edit on GitHub' link to an index.rst on github, which doesn't exist. The file is called index.txt in the repo

Python 3 support

Hi,
thank you for this nice library. What do you think about supporting python3? Would this be easily possible or nothing for a quick fix?

TypeError: __init__() takes at most 3 arguments (4 given) when running example

Hi there.

I've just installed pyzotero using

pip install pyzotero

I then attempted to run an example listed in the docs.

from pyzotero import zotero
zot = zotero.Zotero('123', 'user', 'ABC1234XYZ')
first_ten = zot.items(limit = 10)

However, on running this I'm presented with the following error.

>>> zot = zotero.Zotero('123', 'user', 'ABC1234XYZ')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: __init__() takes at most 3 arguments (4 given)

It's a little perplexing because the init function does in fact take 4 arguments if you count self.

class Zotero(object):
    """
    Zotero API methods
    A full list of methods can be found here:
    http://www.zotero.org/support/dev/server_api
    """
    def __init__(self, library_id=None, library_type=None, api_key=None):
    ...

Any ideas as to why this may be happening?

zot.update_item fails : object not JSON serializable

When I try to update an item, wheter or not I have changed its contents, i get the following error. (Ubuntu 10.04 with Python 2.6.5)
What may have happened?!
Thanks in advance for any hints,
Jacob

>>>zot.update_item(items[4])

Traceback (most recent call last):
  File "", line 1, in 
    zot.update_item(items[4])
  File "/usr/local/lib/python2.6/dist-packages/pyzotero/zotero.py", line 789, in update_item
    to_send = json.dumps(i for i in self._cleanup(payload))
  File "/usr/lib/python2.6/json/__init__.py", line 230, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/python2.6/json/encoder.py", line 367, in encode
    chunks = list(self.iterencode(o))
  File "/usr/lib/python2.6/json/encoder.py", line 317, in _iterencode
    for chunk in self._iterencode_default(o, markers):
  File "/usr/lib/python2.6/json/encoder.py", line 323, in _iterencode_default
    newobj = self.default(o)
  File "/usr/lib/python2.6/json/encoder.py", line 344, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError:  at 0x91b2dec> is not JSON serializable

installation issues

I just did a git pull to see where the project is at, and ran into a couple of issues.

First, not sure why, but pip install . installed elementtree, which is part of the Python 2.7 standard library now. Is it necessary to require this?

Second, see below on feedparser (where mybib.py is just the example code in the readme):

ImportError: No module named feedparser
$ sudo pip install feedparser
Requirement already satisfied (use --upgrade to upgrade): feedparser in /usr/local/Cellar/python/2.7.1/lib/python2.7/site-packages
Cleaning up...
$ sudo pip install --upgrade feedparser
Downloading/unpacking feedparser
  Downloading feedparser-5.0.1.tar.bz2 (204Kb): 204Kb downloaded
  Running setup.py egg_info for package feedparser

Installing collected packages: feedparser
  Found existing installation: feedparser 5.0.1
    Uninstalling feedparser:
      Successfully uninstalled feedparser
  Running setup.py install for feedparser

Successfully installed feedparser
Cleaning up...
$ python mybib.py 
Traceback (most recent call last):
  File "mybib.py", line 1, in <module>
    from pyzotero import zotero
  File "/Users/darcusb/code/pyzotero/pyzotero/zotero.py", line 18, in <module>
    import feedparser
ImportError: No module named feedparser

Pyzotero attachment functions fail on group library

Hello,

I am trying to create a HTML file attachment in my group library, using the attachment_simple and attachment_both functions in the Pyzotero wrapper for Zotero API. It works fine on the user library(my personal library) but fails when I use it on a group library, giving a UserNotAuthorised response. I have the Admin permission in the group library. Below is the code and the error response:
The below code fetches HTML content from an XML file and creates a File object out of it, which is the file attachment to be created

from pyzotero import zotero
import xml.etree.ElementTree as exml
import codecs
def addHTMLAtchmnt(itemId):
#Code to locate XML file from where HTML content is to be fetched
    item = zot.item(itemId)
    extra =str(item[0][u'extra'])
    filename = extra[extra.rfind(".")+1:] + "-atom.xml"
    doc = exml.parse(fileloc + filename)
    root = doc.getroot()

    #FetchHTML content from the XML
    htmltext = root.find('{http://www.w3.org/2005/Atom}content').text
    if htmltext != None:
        htmlfile = codecs.open('htmlattchmnt.html','w',encoding='utf-8')
        htmlfile.write(htmltext)
        htmlfile.close()
        zot.attachment_simple(['htmlattchmnt.html'], itemId)
        print 'Attachment created'

zot = zotero.Zotero(<groupId>, 'group', <apiKey>)
fileloc = "D:\\GitHub\\awol-backup\\"
#Add HTML attachment to the Parent record with ID: 2GMSK2MJ
addHTMLAtchmnt('2GMSK2MJ')

Error Response:

Traceback (most recent call last):
File "C:\Users\Pavan\workspace2\ZoteroTest\atri\zotero\test\zot_html_attachment.py", line 51, in
addHTMLAtchmnt('9U9NBFEF')
File "C:\Users\Pavan\workspace2\ZoteroTest\atri\zotero\test\zot_html_attachment.py", line 25, in addHTMLAtchmnt
zot.attachment_simple(['htmlattchmnt.html'], itemId)
File "C:\Python27\lib\site-packages\pyzotero\zotero.py", line 1031, in attachment_simple
return self._attachment(to_add, parentid)
File "C:\Python27\lib\site-packages\pyzotero\zotero.py", line 782, in _attachment
created = create_prelim(payload)
File "C:\Python27\lib\site-packages\pyzotero\zotero.py", line 692, in create_prelim
error_handler(req)
File "C:\Python27\lib\site-packages\pyzotero\zotero.py", line 1238, in error_handler
raise error_codes.get(req.status_code)(err_msg(req))
pyzotero.zotero_errors.UserNotAuthorised:
Code: 403
URL: https://api.zotero.org/users//items/9U9NBFEF/children?key=
Method: POST
Response: Forbidden

In the Error response I see that the POST URL has /users/ which I actually think should be /groups/ since am trying to create it in a group.
https://github.com/urschrei/pyzotero/blob/master/pyzotero/zotero.py#L673
Please let me know if I've missed out on anything.

Thank you!

Regards,
Pavan

error while updating_item

I get the following error

Traceback (most recent call last):
File "./updateEmptyTitles.py", line 128, in
zot.update_item(item)
File "/usr/lib/python2.6/site-packages/pyzotero/zotero.py", line 1079, in update_item
return self._json_processor(feedparser.parse(data))
File "/usr/lib/python2.6/site-packages/pyzotero/zotero.py", line 553, in _json_processor
items[key][u'etag'] = self.etags[key]
IndexError: list index out of range

while trying to update the title; it passed check_items()

items = zot.items(q='Frost')
for item in items:
if item['title'] == '':
zot.update_item(item)

the item itself

{u'DOI': u'', u'itemType': u'journalArticle', u'extra': u'', u'seriesText': u'', u'series': u'', u'abstractNote': u'Our understanding of the way in which HIV responds to highly active antiretroviral therapy (HAART) has benefited greatly from the use of mathematical models of viral dynamics and evolution. In this paper, I review the role that these models may play in the design and analysis of studies of structured treatment interruptions (STIs). STIs are being investigated in several different contexts: to reduce drug toxicities; to boost HIV-specific immune responses; and to allow reversion of drug resistance mutations in highly drug-experienced patients. I illustrate how models can help to compare the dynamics and evolution of HIV in these different scenarios, and to assess the risks and benefits of STIs.', u'archive': u'', u'title': u'Dynamics and evolution of HIV-1 during structured treatment interruptions.', u'ISSN': u'', u'archiveLocation': u'', u'etag': '5f95cc6480c65c5a914ce0121c45111f', u'journalAbbreviation': u'', u'issue': u'3', u'seriesTitle': u'', u'updated': 'Wed, 09 Apr 2014 20:08:16 PST', u'tags': [], u'accessDate': u'', u'libraryCatalog': u'', u'volume': u'4', u'callNumber': u'', u'key': u'4JR9SQ96', u'date': u'June 2002', u'pages': u'119\u2013127', u'group_id': u'835960', u'shortTitle': u'', u'language': u'English', u'rights': u'', u'url': u'http://eutils.ncbi.nlm.nih.gov/entrez/eutils/elink.fcgi?dbfrom=pubmed&id=12416446&retmode=ref&cmd=prlinks', u'publicationTitle': u'AIDS reviews', u'creators': [{u'lastName': u'Frost', u'creatorType': u'author', u'firstName': u'Simon D W'}]}

thank you very much for any help and thanks for the module

requests.exceptions.ConnectionError in follow()

follow() raises an inappropriate exception after an initial call to collection_items. The first call works, the second one (at 49 items) results in the exception below rather than StopIteration as documented.

File "./zot.py", line 258, in retrieve_bib
items += zot.follow()
File "/Library/Python/2.7/site-packages/pyzotero/zotero.py", line 128, in wrapped_f
retrieved = self._retrieve_data(func(self, *args))
File "/Library/Python/2.7/site-packages/pyzotero/zotero.py", line 264, in _retrieve_data
headers=self.default_headers())
File "/Library/Python/2.7/site-packages/requests/api.py", line 69, in get
return request('get', url, params=params, **kwargs)
File "/Library/Python/2.7/site-packages/requests/api.py", line 50, in request
response = session.request(method=method, url=url, **kwargs)
File "/Library/Python/2.7/site-packages/requests/sessions.py", line 465, in request
resp = self.send(prep, **send_kwargs)
File "/Library/Python/2.7/site-packages/requests/sessions.py", line 573, in send
r = adapter.send(request, **kwargs)
File "/Library/Python/2.7/site-packages/requests/adapters.py", line 415, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', gaierror(8, 'nodename nor servname provided, or not known'))

OAuth?

I've just written some code to use OAuth with Zotero, after a lot of trial and error. This lets an application request access to someone's library. The user clicks a button to approve access, rather than having to manually generate an API key. Once approved, the application can retrieve an API key to use with the existing code.

https://github.com/takluyver/cite2c/blob/master/cite2c/zotero_oauth.py

I came across pyzotero while looking for information on how to do this. Would you be interested in some code to help with OAuth?

Fantastic tool!

It is less of an issue or bug but rather to express my feeling. It is a very useful and helpful tool. Also, very well documented. This is one of the best python libraries I have seen.

One small suggestion: for installation instructions, it may be helpful to show a few screenshots in the process of granting permission after logging in Zotero.

JSONDecodeError on EPUB Downloads

General

Platform: Linux
Python version: 3.6.5
Pyzotero version: 1.3.1

Problem description

I get an JSONDecodeError when using Zotero.file(). Curiously, this only seems to happen with epub files, pdf and mobi work fine.

Traceback (most recent call last):
  File "/home/merlin/Documents/scholarium.at/venv/lib/python3.6/site-packages/django/core/handlers/exception.py", line 41, in inner
    response = get_response(request)
  File "/home/merlin/Documents/scholarium.at/venv/lib/python3.6/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response
    response = self._get_response(request)
  File "/home/merlin/Documents/scholarium.at/venv/lib/python3.6/site-packages/django/core/handlers/base.py", line 187, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/home/merlin/Documents/scholarium.at/venv/lib/python3.6/site-packages/django/core/handlers/base.py", line 185, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/merlin/Documents/scholarium.at/Produkte/views.py", line 343, in medien_runterladen
    medium = zot.file(getattr(obj, art))
  File "/home/merlin/Documents/scholarium.at/venv/lib/python3.6/site-packages/pyzotero/zotero.py", line 192, in wrapped_f
    return retrieved.json()
  File "/home/merlin/Documents/scholarium.at/venv/lib/python3.6/site-packages/requests/models.py", line 892, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/lib64/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python3.6/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

items retrieved with format=bibtex broken

General

Platform: Windows 7
Python version: 2.7.8
Pyzotero version: 1.2.15 apiversion 3

Problem description

• What were you trying to do?
  I am trying to write an utility to make a new zotero collection out of bbl file (which was generated by bibtex from a zotero created master bib file.

bitems = zot.everything(zot.top(format='bibtex'));
zitems = zot.everything(zot.top());

I read a bbl file, look for matching entries (using citekey) in the list of items retrieved (bibtex formated). Then I get the Zotero key of that item (when and if those two lists matched) and copy those items to the new collection. Would work, IF those lists matched (i.e. were in the same order).

But worse, the bibtex list is broken. One of the references had a \n in the middle of the abstract. This completely derail the bibtex formated list. A single item now has many corresponding bibtex items (broken, with only some fields, one just have the rest of paragraph after the \n)

Thansks,
M.

Mutating value returned by item_template() corrupts cache

Creating a new item by calling item_template() method and mutating it means that future calls to item_template() return a template which retains these mutations.

Most probable cause: the returned value is a reference to the cached value, rather than a copy.

Demonstration code:

a=zot.item_template("conferencePaper")
print a
a["title"]="aaaaaaaaaaaaaa"
b=zot.item_template("conferencePaper")
print b

Output:

{u'DOI': u'', u'ISBN': u'', u'itemType': u'conferencePaper', u'extra': u'', u'series': u'', u'conferenceName': u'', u'abstractNote': u'', u'archive': u'', u'attachments': [], u'title': u'', u'proceedingsTitle': u'', u'archiveLocation': u'', u'tags': [], u'accessDate': u'', u'libraryCatalog': u'', u'volume': u'', u'callNumber': u'', u'date': u'', u'pages': u'', u'publisher': u'', u'shortTitle': u'', u'language': u'', u'rights': u'', u'url': u'', u'notes': [], u'place': u'', u'creators': [{u'lastName': u'', u'creatorType': u'author', u'firstName': u''}]}

{u'DOI': u'', u'ISBN': u'', u'itemType': u'conferencePaper', u'extra': u'', u'series': u'', u'conferenceName': u'', u'abstractNote': u'', u'archive': u'', u'attachments': [], u'title': 'aaaaaaaaaaaaaa', u'proceedingsTitle': u'', u'archiveLocation': u'', u'tags': [], u'accessDate': u'', u'libraryCatalog': u'', u'volume': u'', u'callNumber': u'', u'date': u'', u'pages': u'', u'publisher': u'', u'shortTitle': u'', u'language': u'', u'rights': u'', u'url': u'', u'notes': [], u'place': u'', u'creators': [{u'lastName': u'', u'creatorType': u'author', u'firstName': u''}]}

Dep on feedparser 5.1.3 excludes projects using any other version of feedparser

In setup.py, you have:

install_requires=['feedparser == 5.1.3', 'pytz', 'requests'],

which means that any project using any other version of feedparser cannot use pyzotero (e.g. Plone 4 uses feedparser 5.0.1, http://dist.plone.org/release/4.3.1/versions.cfg).

Can this be changed to "<= 5.1.3", for example, to specify a minimum requirement (instead of an exact requirement)?

No specific version requirement would be ideal, but I assume you have some reason for picking 5.1.3. In which case, an "equal to or greater" a specific version would be nicer than "exactly 5.1.3".

Thank you for considering

Tags retrieve error

Hi,

I have a KeyError: 'content' when retrieving my list of tags. The error occurs in _standard_items function at line 568:

        items = [json.loads(e['content'][0]['value'])
            for e in retrieved.entries]

Here is the keys of the tag entries parsed form https://api.zotero.org/users/xxx/tags?content=json&key=xxx
['updated', 'updated_parsed', 'published_parsed', 'author', 'title', 'links', 'title_detail', 'href', 'link', 'published', 'numitems', 'author_detail', 'id']
As you can see, there's no 'content' key :)

Any ideas? Thanks!

Text encoding issues

I have a number of items in my Zotero library that have non-ASCII characters. I do not always have properly encoded results returned from pyzotero. Some examples:

When I use zot.item(item_key, content='citation', style='chicago-author-date'), this is returned when I print using UTF8 encoding: <span>(NoĂŤl 2011)</span>. However, whenever I use zot.item(item_key, content='bib', style='chicago-author-date'), and I print using UTF8 encoding, I get:

<div class="csl-entry">Noël, Marie-Pierre. 2011. “Isocrates and the Rhetoric to Alexander: Meaning and Uses of Tekmerion.” <i>Rhetorica: A Journal of the History of Rhetoric</i> 29 (3) (August): 319–335.</div>

Clearly, it is not an issue with the data Zotero's API is giving to pyzotero, since one method prints properly and the other does not. There must be an encoding difference in your methods.

File download not working properly

Have file id from getting item. When I try to download the file I get the following.

>>> zfile = zotero_cloud._client.file("EPPE3IZB")
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/local/lib/python2.7/dist-packages/pyzotero/zotero.py", line 120, in wrapped_f
    retrieved = self._retrieve_data(func(self, *args))
  File "/usr/local/lib/python2.7/dist-packages/pyzotero/zotero.py", line 257, in _retrieve_data
    error_handler(self.request)
  File "/usr/local/lib/python2.7/dist-packages/pyzotero/zotero.py", line 1274, in error_handler
    raise error_codes.get(req.status_code)(err_msg(req))
UnsupportedParams:
Code: 400
URL: https://zoterofilestorage.s3.amazonaws.com/5b6ba34667afa55c0ea91c89ba693b46?response-content-type=application%2Fpdf&AWSAccessKeyId=AKIAI4IEEOCQFLSM5GMQ&Expires=1423627505&Signature=KYrT%2BEt8%2FUWeHnaft3rscApo%2Bm4%3D
Method: GET
Response: 
InvalidArgumentOnly one auth mechanism allowed; only the X-Amz-Algorithm query parameter, Signature query string parameter or the Authorization header should be specifiedAuthorizationBearer xykjPp4Z0jeaBDAudYEqYA6x46B22F07DBB20F1FTZMDeQh2Ef8aQfqCw9npdV3CpWs+AdJAW3hvPS7FCfEPx2UL4Pmp5OvZbrDirtDYiJoBMTi6nAk=

Any idea what I'm doing wrong here?
I can take the above URL and put into the browser and the file comes right up.

Thanks

Output bug of everything(tags) function

Hi!
The sentence zot.everything(zot.tags()) return a list composed by string values for the first 100 items and lists (links,meta,tag) for the others.
From the doc, the awaited behavior would have been only a list of string.
Bye,
z-acl

Win 8.1 / Anaconda Python 3.5 / Pyzotero 1.1.16 installed by pip