c-okelly/org_to_anki

Python3 module to convert Txt, Org or LibreOffice files into Anki decks

org_to_anki

Python module to convert org or txt files into Anki question decks. Also offered as an Add-on for Anki Lists to Anki

Provides a command line script to upload an org or txt file to Anki.

Based on AnkiConnect. An addon for Anki that provides a HTTP api for Anki.

Contents

0. Supported file types
1. What is an org file
2. Using Org or text files
3. Using LibreOffice files
4. Installation
5. Comand line scripts
6. Running tests
7. Contributing

Supported file types

This project supports reading from both .org, .txt and odt (LibreOffice) files.

To use LibreOffice files these must first be saved as HTML files. Please read the documentation on this.

Microsoft Word (.docx) files are no longer supported. Migration path is below

Word files to LibreOffice migraiton path

Users with existing Word files can migarte them to Libre office by doing the following:

1. Install LibreOffice (Free / OpenSource Word equvilant)
2. Open the file in LibreOffice
3. Save file as ODT format

What is an org file?

An org files a basic text file, very similar to .txt files. The file type is from Org Mode for Emacs but really you don't need to know much about this.

This is all you need to know for this project.

• Asterisks ( * ) are used to to create bullet point style lists.

• .org files are series of lists

• A list item for a question starts with *

• A list item for a answer starts with **

• A comment starts with a #

Examples are provided below.

Here is some example syntax for how question should be laid out

* Question
** Answer 1
** Answer 2
* Another Question
** Another answer

More information on org-mode

Org mode is for keeping notes, maintaining TODO lists, planning projects, and authoring documents with a fast and effective plain-text system

orgmode homepage

Org and Text files

Documentation on Org and Text files

LibreOffice

Documentation on LibreOffice files

Installation

Requirements

1. Have the Anki app installed.

2. Installing the Anki plugin AnkiConnect.

   i. For macOS users of anki-connect see note for macOS users

   ii. For Windows users of anki-connect see notes for Windows users

To install this module from source on Linux or macOS:

git clone https://github.com/c-okelly/org_to_anki
cd org_to_ani
python3 setup.py install

If the org_to_anki command returns a module not found error try the using the following command:

pip3 install ./

instead of

python3 setup.py install

Command line scripts

org_to_anki

org_to_anki

• Upload the default org file to Anki

org_to_anki /path/to/your/org/file.org

• Upload the specified file to Anki

Config values

• Config in project root contains all the default values for running
• Default org file is located at ~/orgNotes/quickOrgNotes.org
• Default anki-connect url is http://127.0.0.1:8765/

Testing

A number of extra libraries are used in testing

• To run unittests

python3 setup.py nosetests

Contributing

All contributions are welcome. Please open a issue first to discuss your ideas!

org_to_anki's Issues

Support card type - Cloze correctly

All of the default types of cards are supported with the exception of Cloze

The cards has two fields => "Text" and "Extra" the second which is optional

The following three use cases should be correctly parsed into closure cards. There is going to be an issue of empty cards and parser overflow into the next card already

* When was Dublin founded {{c1::1204}}
# type=Cloze
** Some extra info

* When was Dublin founded {{c1::1204}}
# type=Cloze

* When was Dublin founded {{c1::1204}}
* Some extra info
# type=Cloze

Ensure cards types are valid and warn user when this is not the case

example
#fileType= exampleType1

If exampleType1 is not a relevant type warn the user and stop the program

Non-Technical Help Request

Hi,

I do not have a technical error or inquiry.
I have a very a basic issue that I have spent a lot of time trying to resolve.

I installed your add-on in Anki.
I created a basic test file, then copied the steps in your "Word or LibreOffice files" page.
It worked the first time around when I imported into Anki, then it stopped working when I tried again.
So I downloaded LibreOffice and converted the test file into an HTML.
However, every time I try to import something it asks me to fill out "Import Options" and "Field Mapping".
I have no idea what I am supposed to fill out as there is not really any guidance. When I fill it out, etc, it gives me flash cards with gibberish.

I have no error report
I am a Windows 10 user

Please advise.
Thank you,
Ben

Handle null return

Handle the null return when ankiConnect fails to parse the data correctly

Add support for Cloze overlapper cards

Support the creation of cloze overlapper cards as generated

Points

• Need to check the correct model exists
• Generate format from list
• Expand on more complicated use cases

Example of the json the needs to be generated in order to make the card

{
    "action": "addNote",
    "version": 6,
    "params": {
        "note": {
            "deckName": "Remote Decks",
            "modelName": "Cloze (overlapping)",
            "tags": [],
            "fields": {
                "Title": "title",
                "Full": "<div>{{c21::Item 1}}</div><div>{{c21::Item 1}}</div><div>{{c21::Item 1}}</div>",
                "Text2": "<div>Item 1</div><div>{{c2::Item 1}}</div><div>...</div>",
                "Text3": "<div>...</div><div>Item 1</div><div>{{c3::Item 1}}</div>",
                "Text1": "<div>{{c1::Item 1}}</div><div>...</div><div>...</div>",
                "Settings": "1,1,0 | n,n,n,n",
                "Original": "Item 1<div>Item 1</div><div>Item 1</div>",
                "Sources": ""
            }
        }
    }
}

does not support Arabic

you make a great addon it just need a little improvement
i hope yo will help me

Error report from the popup

utf8 canot decode 0xc8

What is your operating system

win10

Expand support for .txt files

Expand support for .txt files format

• Add e2e testing
• Update readme

Test how system handle input with empty questions

Auto cloze from a wordlist.txt

First, thanks for sharing your work, discovered Anki just a few days ago, was looking which tools exist for it already, found also your tool, looks promising, from its style I like it most, as very simple structure, simple than anki-editor for org-mode for example.

• [Ubuntu 18.04 ] Operation system
• [.org, meaning text ] Original file type

Now, I had following idea, how would it be if you could specify a wordlist.txt file with a list of words in it, one word per line, and org_to_anki would use this list to create "auto cloze" entries.

Example:
wordlist.txt contains for example the word:
Joe

Now having an input .org file as
* Joe is a great man.

And the script would convert it automatically to:

* {{c1::Joe}} is a great man.
# type=Cloze

Meaning, surrounding it with double paranthesis for cloze, always using c1:: for example, then in next line adding the type definition which is required for Cloze each time. # type=Cloze

This would allow us converting a set of sentences, quickly into Cloze questions, based on some wordlists.

Any tips and hints are welcome, thanks in advance. Keep up the great work.

Add automated testing to Anki integration

Org files parsing of logbook and lists

Following text in an org files throws an error:

   :LOGBOOK:
   CLOCK: [2019-05-01 Wed 23:36]--[2019-05-02 Thu 00:01] =>  0:25
   :END:
   1. [X] L0

Notes tag does not work for special file types

Please follow the template below to report your issue.

• Raw text
• Error report provided by Anki
• Operatin system
• Original file type

Please include the following information:

Any general information

The notes tag is not respected in special file types when it is used under non-question bullet points

Raw text of the file you tried to upload

Use this website to save your raw text and share the link here optionally => https://pastebin.com/

# fileType=organisedFile 
* Planning
  # type = notes
** Some notes
*** Some more notes
* Section 1
** Question
*** Answer

Error report from the popup

None

What is your operating system

What was the original file type

org

User should get a message asking them to confirm before adding the cards to anki?

MacOS installation succeeds, but usage fails.

Please follow the template below to report your issue.

• Raw text
• Error report provided by Anki
• Operatin system
• Original file type

Please include the following information:

After a successful install usage of org_to_anki fails with the following output:

Traceback (most recent call last):
  File "/usr/local/bin/org_to_anki", line 11, in <module>
    load_entry_point('org-to-anki==0.1', 'console_scripts', 'org_to_anki')()
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 489, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2793, in load_entry_point
    return ep.load()
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2411, in load
    return self.resolve()
  File "/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2417, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
ModuleNotFoundError: No module named 'org_to_anki'

What is your operating system

Mac OS 10.13.6
Python 3.7.3

Bug with image in question line

Example text

* Question A  [image=https://lh5.googleusercontent.com/pQzlN0OvcjbFkYhD8gki4UbM2b0U9-7LAte4HY9bIMuTjbSixUVkQXZJTbhIVn-oag0gLIkLdT-9VvhJYmFsuf1Y4DMM0qruPF7pk3OsWJ-WCPvZbnnLO7ZlF-Tv23QpQ1yuPrjI]
** Answer

Support for bullet points that are topics

About org files

Hi, there. I noticed the PR in Add org_to_anki GitHub link #8. However, I'm quite misunderstood about this tool, especially what the ".org" file is for. I have a humble suggestion, why not add a little more description in README.md file of your project, so that even new users like me can easily tell what this tool is all about. Thanks!

Support for .pdf not available as answer, rather than only .png (image)

Tried pdf documents as answer, not working, ok, acceptable, just mentioning here.

Support the upload of images via txt files

Supports tag in notes

Should support tags in notes

Improvement: Multiple fields

Say you have the following in your org file:

* Second Question
** Second Answer 
** Front hint test 

Which results in the generation of the following card (or note, always forget the anki terminology):

It would be nice if you could for instance put the following in your org file:

* Second Question
** Second Answer 
# field='Front Hint'
** Front hint test 

And have the plugin generate the following card:

Support image inline for Emacs

As discussed in #69

Support for topics document type

Clean up readme

Readme is a mess. The information need to be better organised.

Idea.

Should break build / testing / general info into a separate file

Need to have clear use cases for the users to use for syntax / word / libre office support so it is clear

Question that have the type notes should be ignored

Exports Defaults to Subdeck beneath "0. List Notes" Deck

I am on Windows exporting from org files. When I export, a correctly named subdeck is created beneath deck "0. List Notes".

I'm sure I'm missing something obvious, but how do I prevent the creation of this top level deck?

Thank you for your time and for this project. It's incredibly helpful.

What is the second script?

The README states

Provides two command line scripts. First is to upload an org or txt files to Anki. The second allow quick notetaking and storage.

But Command line scripts only lists org_to_anki, so I'm wondering about that other notetaking and storage script.

I'm still wondering whether it's worth adopting your plugin or not. Used to rst/markdown syntax, but org seems like a fair alternative.
It also seems like a vim plugin could be wrapped around org_to_anki, to upload a file selection (eg a :ToAnki mapping).

How much effort would it take to support the reverse process, Anki to org? If I could sync Anki with a git repo of .org files, it would be extremely satisfying (the git repo could then be used as part of a Zettelkasten).

Add support for uploadDeck = false

This should not upload the current deck and should print some sort of message indicating that it will be skipped.

Should normally be used for inprogress files

Support unicode encoding for foreign languages

Should be able to support foreign languages in unicode format

Should be supported from LibreOffice and from raw text files
Tests should cover both

Example

* Hello world in Chinese?
** 你好，世界

Installation on Windows?

Hi!
In installation instructions I can read:

To install this module from source on Linux or macOS:

git clone https://github.com/c-okelly/org_to_anki
cd org_to_ani
python3 setup.py install

But, how to install org_to_anki on Windows?

Thanks, your software looks incredible.

Support users adding pictures via urls that automatically download

As above

Should have a look at how anki-connect handles the url stuff. Could be a possible attack vector so not really sure about this feature tbh but it is technically an enabler for anki-remote-decks

Investigate issues that causes missing html in bs4 4.9.1

Set bs4 version from 4.8.1 to 4.9.1

3 test should fail

Fix empty question bugs and org elements parsing

This files should create just a single question and not throw errors

* 
** 
* order list
** Answer
* Planner [0/0]
** Planner [/]
   #type=notes
** Something 
   :LOGBOOK:
   CLOCK: [2019-04-19 Fri 14:27]--[2019-04-19 Fri 14:27] =>  0:00
   :END:
* 
** 

Integrate natively into Anki application

Integrate as an Anki plugin.

Will required at least the following

• update license to be compliant
• Write code for optional replacement ankiConnector
• Figure out a way to correctly packages code
• upload to ankiWeb
• Profit

Refactor the AnkiConnectWrapper submodule

Refactor module to create a better abstraction

There should be a builder use so create the different types of connector based upon the environment they are running in.

All of the methods that are used for the construction of JSON should be refactored out

All method in the actual connection class should match some sort of basic interface

Created limited support for docx files

Create limited support for docx files.

Seems to be issue converting bullet points into a reasonable format with the current available libraries so support will just be expanded to * and # style notations

[idea] assign LibreOffice Writer list styles to Anki fields

Would it be possible to introduce this feature: assign LibreOffice Writer lists styles to Anki fields?

In Writer you can use lists styles or create your own styles and using them is easy. For each line I wish I could use different list style, so that when importing in Anki each style would go to different field. This would allow to create multi-field notes in Writer.

Add-on would have to have configuration file to define which style goes to which field.

Another idea would be to use paragraph styles so that add on would match each style to different field.
The easiest way to style is to use Bold, Italic, Underline, Cross-out or text colors.

This would make work very easy because for a long list of content user could use Writer's format painter icon to quickly format multiple lines or in more advanced cases find and replace feature to format lines of text (Alt find and replace LO add-on).

How to add an image to Extra field of Cloze?

Tried, did not work here. Do you have any working example? Thanks in advance. Adding an image on a basic note is working here, but not in Extra field of cloze.

Add support for documents in bullet points format

Add support for bullet point document.

Requirements

Documents

• libreoffice -> mac

• office -> mac

• office -> mac

• libreoffice -> mac (s the required(?))

• Complete relevant documentation

• Remove support for raw docx files as this is confusing

If line starts with TODO (case sensitive) ignore line and children

If line starts with TODO it should be ignored as this will be considered a question / section in progress and not ready for upload.

Example below

* TODO Some Linux topic
** Question 1
*** Answer 1

Multi-level lists do not seem to import in Word

Are multi-level lists from .docx files supported? If so, are there any formatting or other limitations? Or some other requirements?

Currently running into an error when I try to import into Anki.

Raw text of the file you tried to upload

Word and HTML files: Anki Addon (Lists to Anki) Error Report.zip

Error report from the popup

The error was 'utf-8' codec can't decode byte 0xb7 in position 53017: invalid start byte.

Error report:
Traceback (most recent call last):
File "C:\Users\Hawley Griffin\AppData\Roaming\Anki2\addons21\1029306148_init_.py", line 52, in importNewFile
parseAndUploadOrgFile(filePath, embedded=True)
File "C:\Users\Hawley Griffin\AppData\Roaming\Anki2\addons21\1029306148\org_to_anki\main.py", line 29, in parseAndUploadOrgFile
_parseAndUpload(filePath, embedded)
File "C:\Users\Hawley Griffin\AppData\Roaming\Anki2\addons21\1029306148\org_to_anki\main.py", line 45, in parseAndUpload
deck = parseData.parse(filePath)
File "C:\Users\Hawley Griffin\AppData\Roaming\Anki2\addons21\1029306148\org_to_anki\org_parser\parseData.py", line 17, in parse
formatedData = convertBulletPointsDocument(filePath)
File "C:\Users\Hawley Griffin\AppData\Roaming\Anki2\addons21\1029306148\org_to_anki\converters\BulletPointHtmlConverter.py", line 27, in convertBulletPointsDocument
documentType = checkDocumentType(filePath)
File "C:\Users\Hawley Griffin\AppData\Roaming\Anki2\addons21\1029306148\org_to_anki\converters\BulletPointHtmlConverter.py", line 37, in checkDocumentType
soup = BeautifulSoup(htmlFile, 'html.parser')
File "lib\site-packages\bs4_init.py", line 245, in init
File "C:\Program Files\Python36\lib\codecs.py", line 700, in read
File "C:\Program Files\Python36\lib\codecs.py", line 503, in read
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb7 in position 53017: invalid start byte

What is your operating system

Windows 10

What was the original file type

Word File (.docx) converted to .html

Add Docs for adding an image via url #41

[image=url]

Ensure file path are handled correctly for relative file paths

Lists to Anki issue

Hi, I really appreciate your contribution, i wish you can help me.

• [ Hey there it seems an error has occurred while running the importer:
  the error was model was not found: basic. ] Error report provided by Anki
• [ windows 10] Operatin system
• [html ] Original file type
  -the text has the same format you report in your instruction video.

Improvement: option to disable html formatting

It would be nice to have an option to disable formatting.

For instance, you'd have the following in your org file:

# type=Basic
# formatting=false
* Question
** Answer

Which would result in the following card being generated:

Instead of:

Add images to the Anki app page

Should have a before and after image. Easier for new users

Support for .html not available as answer, rather than only .png

In contrast to .pdf .html could work, tested also this, but also .html as answer did not work here. Could be another useful alternative to .png (or images) in certain situations, meaning where you have already some .html but no .png, so why not just using that .html directly.

Remove usage references to Word in the readme

No long term plan to support word documents as the html they generate is too unpredictable

Correct format of json string that is used to send data to anki connect

single quote should not be used. Double quotes that are within the card should be correctly escaped. Json library should probably be used instead

Support sync of text files to Anki deck - edits / deletions / new cards

Hello,

I am using Ubuntu 18 and .tex files to write my notes. org_to_anki seems a very useful addition to my workflow because it will allow me to write my Latex notes in Vim, which brings all the magic of snippets, keybindings, etc. However, while I can upload brand new cards to Anki, I cannot:

• Remove cards from my deck. If the server has any residual cards that are not present in my my_deck.txt file anymore, org_to_anki won't delete them, so they will hang around forever or until I delete them manually through the Anki graphical interface (this could cause a mismatch with the server, which requires to run Tools > Check database to correct, which is inconvenient).
• Update cards. If I change a card, org_to_anki won't replace the old version by the new version.

Am I doing something wrong? Is there a configuration parameter that I can set, or a command, in order to remove old cards and update modified cards?

Thanks a lot.