e-alizadeh/zotero2readwise

A Python Library to retrieve annotations and notes from Zotero and upload them to your Readwise.

zotero2readwise's Issues

After the last release all my articles from Zottero got duplicated

As in the title. I have set the cronjob for 3 am every day. And on 20 Oct (after the new release on 19 Oct) all the articles got pushed to the Zotero second time. I tried to simply remove them, but they got pushed again. Can we somehow fix this issue?

Readwise Source type

Note sequence and tags

I've noticed that highlight sequencing is random, rather than chronological -- meaning that sometimes highlights from the same page appear out of order.

I've also noticed that tags don't seem to come across properly. So using the ".h1" tag from Readwise for a heading is not rendered properly as a tag and, hence, also not as a heading.

One cause of this seems to be that page numbers can be wrong (e.g. I have a case of highlight on p82 turning up in Readwise as p81, and thus under the wrong heading)

Error: Process completed with exit code 127

I keep getting this error when I try to run the automation. As shown in the instructions, I set up the secrets, but it seems to be looking for a "zotero_library_id" which wasn't one of the three. Did I miss something?

Researching the error suggests that either (a) something is missing or (b) the script doesn't have other permissions it needs to run. Option b seems unlikely so I'm stuck on the hypothesis that it is option a, but after starting from a fresh fork, replacing the secrets, and generally doing everything I can think of, I'm stuck.

Sync failing - Process completed with exit code 1.

Can this handle area annotations on PDFs?

The title basically says what I'm asking. I've got it working fine for text/highlight-based annotations and notes, but it doesn't work for area-based annotations, which is something I'd love to have.

Just wondering if I'm missing something or it's a known limitation.

Add documentation (docstrings)

Only sync pre-specified color(s)?

It would be convenient to have a highlight color which signifies "I want this to be synced to Readwise". In my case, most of my highlights aren't review-worthy in a generalized context outside of whatever research I'm doing. Some small amount are.

I imagine most people don't use all of the available options anyway.

change highlight url to local Zotero

great job!

I use this in obsidian workflow as you posted in workflow

I has some suggestion:

1. Readwise highlight url points to an link to Zotero local app rather than web. So we can use url in Obsidian to open Zotero pdf .
2. metadata url is zotero url, can change it to local pdf file

Ability to add a custom tag to all highlights

This feature is to add a tag like #zotero to all highlights.
This option should be disabled by default.

Integrate with Zotfile

I know that it specifically states that this is meant for using the zotero reader.

However, I read on my tablet and then use the zotfile integration to pull back into zotero. In this case, zotfile extracts the annotations from the file and adds them as notes as well as making an "*_annotated.pdf" version of the file.

Would be great if this could pull those highlights as well.

run.py -h says notes are included by default by they aren't

When running run.py -h in version 0.4.4, run.py tells users that notes are included by default. However, they are not.

The reason for this is simply that the help parameter for the corresponding argument says so:
help="Include Zotero notes | Options: 'y'/'yes' (default), 'n'/'no'",
Line 36 of run.py at 18715da

But the default parameter is set to 'n':
default="n",
Line 35 of run.py at 18715da

As I am a rather new user of Zotero2Readwise, I can only assume I would have expected a default="y" here, so that the user isn't surprised that certain data has not been exported to Readwise. But, if this is a deliberate choice, then the help string should be adapted.

Sync highlights after a given Zotero version

Redwise Author is 'api.zotero.org' instead of actual document author

Hi, there is an 'ImportError'

ImportError: cannot import name 'write_library_version' from 'zotero2readwise.helper' (/Users/username/miniconda/lib/python3.9/site-packages/zotero2readwise/helper.py)

Z2R.zt2rw Approach 2 (through python terminal) stopped working in July 2022

It goes through the normal sequence in python, exactly as before but highlights just no longer appear in Readwise. I recreated my zotero key and readwise token and still not appearing. No errors, zotero seems to be collating and pushing the data to Readwise.

Anyone else experiencing this? Did Readwise change something on their end?

Feature request: Truncate creator/author field if it exceeds 1024 characters

Hi!

Thank you for your library!

Description of error

I have tried to import my Zotero annotations to Readwise but get the following error (both as a GitHub Actions Workflow and on my local machine):

Retrieving ALL annotations from Zotero Database. 
It may take some time...

282 Zotero items are retrieved.
ZOTERO: Start formatting 282 annotations/notes...
It may take some time depending on the number of annotations...
A complete message will show up once it's done!


ZOTERO: Formatting Zotero Items is completed!!



Readwise: Push 282 Zotero annotations/notes to Readwise...
It may take some time depending on the number of highlights...
A complete message will show up once it's done!

Traceback (most recent call last):
  File "/Users/stephan/Documents/GitHub/workflows/run.py", line 64, in <module>
    zt2rw.run()
  File "/Users/stephan/Library/r-miniconda/envs/test/lib/python3.9/site-packages/zotero2readwise/zt2rw.py", line 55, in run
    self.readwise.post_zotero_annotations_to_readwise(formatted_items)
  File "/Users/stephan/Library/r-miniconda/envs/test/lib/python3.9/site-packages/zotero2readwise/readwise.py", line 149, in post_zotero_annotations_to_readwise
    self.create_highlights(rw_highlights)
  File "/Users/stephan/Library/r-miniconda/envs/test/lib/python3.9/site-packages/zotero2readwise/readwise.py", line 72, in create_highlights
    raise Zotero2ReadwiseError(
zotero2readwise.exception.Zotero2ReadwiseError: Uploading to Readwise failed with following details:
POST request Status Code=400 (Bad Request)
Error log is saved to error_log_400_failed_post_request_to_readwise.json file.

The error_log_400_failed_post_request_to_readwise.json indicates for 29 items:

{"author": ["Ensure this field has no more than 1024 characters."]}

Proposed solution

After concatenating all creators while importing a Zotero item, would it be possible to check if the string exceeds 1024 characters, and if it does, replace it with <last name of first author> et al.? Considering how prevalent a large number of authors is nowadays in some fields, I imagine many people will encounter that problem.

Probably relates to #43 and #27.

Troubleshooting sync items failing

I'm getting the same 36 annotations/notes "failed to format" error. I created new highlights and annotations in iOS, PC zotero and they sync fine. I've deleted a bunch of fields where it looked like there were too many (ie. authors). Nothing is working, same number 36. How do I find out which files are breaking? I opened JSON file but don't know how to search by document key.

Add highlights failed to be pushed to Readwise next to error logs.

Fails on notes missing attachment_url and inked annotations

I have two issues, both about to be fixed in a pull request.

#46 introduced the metadata key for "attachment_url", but this line accesses it even if it's not set. That value (and creators) should have initial values "" so that they exist in the dict.

Additionally, many of my notes are handwritten, but fail to import into readwise (which makes sense). We should ignore 'ink' type annotations.

Send case law and other types of documents to Readwise?

Would it be possible to send other types of documents other than books and articles to Readwise? For example I annotate a lot of case law and laws and reports.
I’m fine if these are categorised as articles if this means they are also sent to Readwise.

But if it’s possible to categorise them correctly and get them into Readwise that’d be wonderful! Is this a possibility?

Readwise highlight url points to an link to Zotero API instead of a web app

Zotero Annotations/Notes Failed to Format

I receive the below message after running the script provided. The can see the 2/9 that didn't fail in my Readwise account.

NOTE: 7 Zotero annotations/notes (out of 9) failed to format.
You can run save_failed_items_to_json() class method to save those items.

Here is an example of what I see in my json file:

    "meta": {},
    "data": {
        "key": "UIY4CRAT",
        "version": 1612,
        "parentItem": "DT44V28V",
        "itemType": "annotation",
        "annotationType": "highlight",
        "annotationText": "The basis of psychological targeting draws on selfregulatory goal-directed conceptualizations of personality, broadly known as cybernetic theories (e.g., Allport, 1937; Carver, 1979; DeYoung, 2015; Fleeson & Jayawickreme, 2015; Fleeson & Wilt, 2010; O. P. John et\u00a0al., 2010; McAdams & Pals, 2006; Revelle & Condon, 2015; Van Egeren, 2009). According to cybernetic theory, humans constantly compare their current states to their desired states (i.e., their goal states). If there is a mismatch between the two, then individuals adjust their behaviors and actions to close the gap. In such a system, predisposing tendencies\u2014such as personality traits\u2014allow humans to dynamically adapt to situations (Rauthmann et\u00a0 al., 2014). Thus, personality traits, beliefs, and motivational tendencies are useful for psychological targeting because they are the building blocks of a particular person\u2019s cybernetic system that allows them to better adapt to situations.",
        "annotationComment": "",
        "annotationColor": "#ffd400",
        "annotationPageLabel": "2",
        "annotationSortIndex": "00001|003198|00271",
        "annotationPosition": "{\"pageIndex\":1,\"rects\":[[322.5,500.47,555.02,511.48],[310.5,488.47,555.17,499.48],[310.5,476.47,555.59,487.48],[310.5,464.47,555.52,475.48],[310.5,452.47,555.03,463.48],[310.5,440.47,555.24,451.48],[310.5,428.47,554.83,439.48],[310.5,416.47,555.05,427.48],[310.5,404.47,554.98,415.48],[310.5,392.47,555.32,403.48],[310.5,380.47,554.86,391.48],[310.5,368.47,555.42,379.48],[310.5,356.47,555.09,367.48],[310.5,344.47,555.41,355.48],[310.5,332.47,554.79,343.48],[310.5,320.47,555.23,331.48],[310.5,308.47,555.04,319.48],[310.5,296.47,493.51,307.48]]}",
        "tags": [],
        "relations": {},
        "dateAdded": "2023-12-01T18:15:55Z",
        "dateModified": "2023-12-01T18:15:55Z"
    }
},
{

Partial push of highlights to Readwise

Currently, the library fetches all Zotero highlights/notes and pushes them to Readwise each time.

For efficiency and also in order to avoid any potential duplicated highlights due to either library changes or changes in Readwise de-duplication algorithm, the library should be able to push only latest highlights.

This is related to the issue #31.