noembryo/kohighlights

KOHighlights is a utility for viewing KOReader's highlights and/or export them to simple text, html, csv or markdown files.

kohighlights's Issues

Highlight Sync is not working and unreliable (NEVER SOLVED).

1. Make highlight (and nothing more on device)
2. Connect device to KOHighlights > "Archive" highlights
3. Connect other device with KOHighlights > "Sync with archived"
4. Connect other device with KOHighlights > "Sync with archived"
5. Also, if you can connect all your devices at once, you can use the option "Sync with file", and use your most recent metadata.xxx.lua file to sync the others..

Step 1-4(did not work with different screen size and highlight removal) only worked for same size devices. I also noticed the following,

• All the highlight + notes + bookmarks synced/ported if I manually copy the .sdr folder along with .epub from Android and paste in anywhere on Linux (this is probably KOHighlight "Sync with file" does although I do not need to match hash as I know my most updated file) and vice versa.

I don't know whether it is supposed to be for all .epub or there are exceptions. It seems to me KOReader (.sdr+epub) is portable at least worked on Linux and Android. This is probably also possible to do using Syncthing over USB.

Another problem was KOHighlight browser did not see my mounted phone or windows drive!

Devices used:

1. Laptop with Fedora 36
2. Samsung phone with Android 12

Include Chapter Titles But Only Once Per Chapter (not on every annotation)

When exporting markdown directly from KoReader, it groups the highlights with the titles of the chapters, which I find quite useful.

Example:

The Molecule of More
##### Daniel Z. Lieberman, Michael E. Long
## Chapter 1 LOVE
### Page 35 @ 01 March 2022 05:19 PM
*The scientists who...

Being able to do that with this brilliant plugin would be fantastic.

Does not work with Android devices

The app will not recognize Android devices connected to PC as accessible folders/drives (MTP/PTP protocol) and therefore cannot access files to sync highlights. The only workaround is to copy files from device to PC. Would be great if this could be fixed.

Unreadable book titles on dark theme (KDE/Linux)

On KDE desktop with default dark theme enabled, all text in the app inverts correctly except book titles, which stays black:

Android Device not recognized in "Scan Directory"-dialogue

Hi,

your tool is greatly useful all in all, I have however one little issue that makes exporting notes more cumbersome than it needs to be. I use a Likebook Mars as ereader and this device is unfortunately not recognized when I want to scan its book directory. I attached a screenshot, where you can see this:

My ereader appears as Likebook-T80D in the file explorer, but not in KoHighlights. Is there any way to fix this?

Best regards and thanks in advance.

book paths mangled

I wanted to try KoHighlights, but… things went wrong with the book pathnames.

• koreader on inkpad with homedir set to /mnt/ext1
• files and sidecars on inkpad in /mnt/ext1/news
• homedir exported through sshfs as /mnt/ext1
• sshfs-mounted inkpad:/mnt/ext1 to linux-host:$HOME/nfs/koreader
• KoHighlights-1.7.3.0
• KoHighlights.v1.7.3.0.PySide2
• KoHighlights started by cli
  python $HOME/.local/bin/KoHighlights-1.7.3.0/main.py $HOME/nfs/koreader/news/

The books’ paths are cut to an impossible absolute destination. I would expect the path to be relative to some known directory, e.g. the CWD, or an absolute system path, where it was last found (not ideal)

The paths to the sidecar files are correct – being absolute system paths perhaps also not ideal.

It seems, KoHighlights cuts away the Home folder from the doc_path in the lua file, in my case: /mnt/ext1. How it knows about the homedir of koreader, escapes me. I would expect the basename to be relative to the input path given to KoHighlights.

• When news directory is rsync’ed to local FS, the book paths do not change.
• When the mount is changed to /mnt/ext1/news $HOME/nfs/koreader the paths do not change either. The only thing koreader’s dropbear-ssh-plugin allows to export, is the home-dir of koreader.

KoHighlights as database?

Is there a possibility to re-invent KoHighlights not just as utility to view/save highlights but as utility that can save highlights unto itself. That way every time we open KoHighlights we can browse our highlights without connecting device.

Much love from 📚 friend

fatal error

Hi, thank you very much for your work. It'll be very useful, but I'm having a problem installing it.

I get a "Fatal error detected" - "Failed to execute script main"

Could you please help me?

Thanks

release as a python pip package ? make it easy to install and available for linux users ?

Hi, this looks like a fantastic tool, thank you for your effort.
I'm trying to run this on Linux but failed, because probably lack of python skills.
My interest is in creating a package for ArchLinux.

I have tried installing the needed python modules dependencies, but I'm not sure which ones.I see a lot of python imports for bs4 beautifulsoup, mechanize, pyside ?

1. Could you please detail on the README.md page the requirements and python dependencies needed ? Which ones belong to the standard library ? Which ones are extra foreign ?

2. Would it be possible for you to release this as a python pip package ?
   https://pip.pypa.io/en/stable/
   https://packaging.python.org/

A lot of potential users KoHighlights are for sure Linux users.
For example deb/apt Debian/Ubuntu packages and ArchLinux AUR packages
A clean and easy way to install this would be great, to avoid littering your system with untracked/uneeded/incompatible/coliding.

Suggestion: Merge highlights also if one of the book version has no highlights at all.

I often read a book on one device and take the second with me on my travels. I would like to copy highlights from the first to the second.
If I open book on second device I have to make fake highlight to be able to merge it with highlights on other device.

KoHighlight on Flathub

It would be awesome to have KoHighlights packaged as flatpak on Flathub.org so it is easily installable on every Linux distribution.

Unable to connect to MTP directory

I want to sync the highlights from my Kindle device with my Boox device (Android system). However, I cannot select the folder directory on Boox because it is connected to the computer via MTP.

removing some books info removes whole sdr folder

Hi,
today I discover when I select few lua files belonging to the same book (but not all) and press DEL, KoHighlight will remove whole sdr folder with all lua files. Quite shocking. Please take a look at he screen recording (no audio).

screen-record.mp4

Would it be possible to additionaly provide a command line interface

Example pyglossary, https://github.com/ilius/pyglossary/
Although it is a GUI python application (using another framework, GTK)
the command-line interface, works in all operating systems without any specific requirements, just type:
python3 pyglossary.pyw --help

building for linux and python3

Hi,
I am trying to build it with python3 on linux, but failing. Do you have any suggestions?
Thanks

"failed to execute script main"

I'm tempted to use kohighlights on win11 but it doesn't work. This error always appears

I have a version 1.6 that I managed to make work after a friend edited some things, but I always try to update and the same error appears

No macOS binary in downloads

I noticed there is no macOS binary included on the downloads page, so I created one using Platypus (https://sveinbjorn.org/platypus). Feel free to download and include it. Obviously, you are welcome to test it or have it tested first. The package is supposed to be self-contained with no need for Python or library installations (the whole Python3 venv is included). I have also included the few resource files
that I have used to compile the package for future reference and reproduction purposes.

A screenshot:

file explorer not opening in Linux - Manjaro GNOME

Hi,
It is failing to open folder with lua file on Linux. In Windows it works fine. After selecting menu Open location following is dumped in terminal stderr and no file browser is opened.

xdg-mime: no method available for querying MIME type of '/home/richo/CloudStation/aKnihy/_books/Atomic Habits - James Clear.sdr'
xdg-mime: mimetype argument missing
Try 'xdg-mime --help' for more information.
/usr/bin/xdg-open: line 881: x-www-browser: command not found
/usr/lib/firefox/firefox: /tmp/_MEIlYRQpG/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by /usr/lib/firefox/firefox)
/usr/bin/xdg-open: line 881: iceweasel: command not found
/usr/bin/xdg-open: line 881: seamonkey: command not found
/usr/bin/xdg-open: line 881: mozilla: command not found
/usr/bin/xdg-open: line 881: epiphany: command not found
/usr/bin/xdg-open: line 881: konqueror: command not found
/usr/bin/xdg-open: line 881: chromium: command not found
/usr/bin/xdg-open: line 881: chromium-browser: command not found
/usr/bin/xdg-open: line 881: google-chrome: command not found
/usr/bin/xdg-open: line 881: www-browser: command not found
/usr/bin/xdg-open: line 881: links2: command not found
/usr/bin/xdg-open: line 881: elinks: command not found
/usr/bin/xdg-open: line 881: links: command not found
/usr/bin/xdg-open: line 881: lynx: command not found
/usr/bin/xdg-open: line 881: w3m: command not found
xdg-open: no method available for opening '/home/richo/CloudStation/aKnihy/_books/Atomic Habits - James Clear.sdr'

Looking at your code it should use *xdg-open folder_path" which works from my command line even I made little copy of your code (as follows) and it is working from it.

import os, sys, re
import subprocess

path="//home"

try:
    if sys.platform == "win32":
        os.startfile(path)
    else:
        opener = "open" if sys.platform == "darwin" else "xdg-open"
        subprocess.call([opener, path])
except OSError:
    self.popup(_("Error opening target!"),
               _('"{}" does not exists!').format(path))

It is something which is hidden in the executable build which is ignoring some environment variables available in my normal environment?

Enhancement request - sync jobs and other features

The following is inteface / feature ideas you may consider for future development. The proposed interface is inspired by an old nifty program called "allway sync". The main feature allows creating sync jobs for each book. Each job resides in its own tab where it can be configured and run.

The second file show the sync direction configurator. This is how it is implemented in allway sync. Extremely flexible since one can select the sync direction, participating devices, as well as weather or not to propogate changes, deletions, or both.

The "Analyze" button produces the summary of changes for each device and the "Synchoronize" button - executes the job.

I hope this is in line with the direction you are taking your great little program. Thanks again for your service to the community!

The following shows some of the sync configuration options that one can change upon clicking on the "change" button:

Customize date format

Would it be possible to customize the format of the date ? For example, to something like:

Page 27 [[Jul 19th, 2022]]:
Mackay had a low opinion of all Crusades. The Children’s Crusade struck him as only slightly more sordid than the ten Crusades for 

The double brackets would be to better integrate the highlights with Logseq.
Thank you !

Possibility to transfer highlights to epub file

Hi guys, this is not an issue but an improvement suggestion: would it be possible to transfer all the highlights from the LUA file created from koreader, directly into the epub format?

I'd very much appreciate this feature.. I need to "save" the highlights created from koreader into all the epub books I'm reading, but doing it manually would be very, very very tedious.

How do I set this up?

Hello,

I want to set this up to sync and export my KOReader highlights to my computer, but I don't know how to set this up. I downloaded the KoHighlights-1.3.0.0 zip file with all the Python files. Not sure what I'm supposed to click on or do now.

Saving doesn't store all highlights

This is absolutely great piece of software I am so grateful that you made it. This is primary reason I am using Koreader everywhere.

I notice recently that not all highlights are exported. There are more showed in app on the right side them they ends up in exported text file. please see screenshots showing the right side with list of highlights. I am attaching it as well as the .lua file as well as the exported text file.

If I could code in python I would look why it is, but python is not something I could code.
Thank you in advance for your time.

-r-
All files in zip.zip

Sorting of books titles doesn't ignore A & The

Hello again ;-)
I am using KoHighlights very very often, one ting is that sorting on book title is different then in KOReader and in Calibre too. This makes the certain action a bit difficult.

Would it be possible to make sorting to ignore the A & The part of title? This would make it same in all three programs.

suggestion: compare highlights metadata files for same book's in sdr folder

Hello,
often I end up with two or more metadata lua files in the sdr books file, normally there is one .old and sometime som others. The reason is that I read books on more devices and sometime they do not manage synchronize and there is a conflict in file later.

It would be great if there would be sort of merge tool, which would merge those conflicting file into one nice. Merging only highlights and setting position to latest one.

It could be part of this or it could be some kind of command line tool.
Do you think this could be useful and not too complex to make?

Can't export to csv

Python 3.10 on Linux
When choosing to export to csv i get following error:

Traceback (most recent call last):
  File "/media/hdds/download/KoHighlights.v1.4.4.0.PySide2/secondary.py", line 362, in on_export_btn_clicked
    self.base.on_export()
  File "/media/hdds/download/KoHighlights.v1.4.4.0.PySide2/main.py", line 2178, in on_export
    if self.save_sel_highlights():          # combine to one file
  File "/media/hdds/download/KoHighlights.v1.4.4.0.PySide2/main.py", line 2433, in save_sel_highlights
    text += get_csv_row(data) + "\n"
  File "/media/hdds/download/KoHighlights.v1.4.4.0.PySide2/main.py", line 103, in get_csv_row
    value = data[key].replace('"', '""')
AttributeError: 'int' object has no attribute 'replace'
Traceback (most recent call last):
  File "/media/hdds/download/KoHighlights.v1.4.4.0.PySide2/secondary.py", line 362, in on_export_btn_clicked
    self.base.on_export()
  File "/media/hdds/download/KoHighlights.v1.4.4.0.PySide2/main.py", line 2178, in on_export
    if self.save_sel_highlights():          # combine to one file
  File "/media/hdds/download/KoHighlights.v1.4.4.0.PySide2/main.py", line 2433, in save_sel_highlights
    text += get_csv_row(data) + "\n"
  File "/media/hdds/download/KoHighlights.v1.4.4.0.PySide2/main.py", line 103, in get_csv_row
    value = data[key].replace('"', '""')
AttributeError: 'int' object has no attribute 'replace'

Probably page key is understood as int
so as quick fix i used str(data[key]).replace('"', '""') to fix issue for me

Suggestion for some enhancement

Once I am at it :-) I have some suggestions, If I may:

• One the right side, in book info, can you put there number of comments?
• Would it me possible to sort comments (highlights) based on page number and make it possible to revers so from 1-xxx and than from xxx-1 ?

Thank you for considering.
-r-

Error when exporting individual book files when a filename is too long

Steps:

• Mount Kobo e-reader (that is running koreader)
• Start KoHighlights from command line
• Scan Directory (/media/user/KOBOeReader/)
• Select all files in list with Ctrl-A
• Ctrl-S to Export selected highlights
• Select 'As individual book files'

The interface can still be used, but the three progress dots continue to animate, and at the command line, I see the error:

IOError: [Errno 36] File name too long: "/home/neil/org/book-highlights/p2pmodels.eu - David Bollier_ \xe2\x80\x9cIn digital contexts, the whole social dimensions of the stewardship and curation of a Commons don't really get adequate attention\xe2\x80\x9d - P2P Models.txt"

And not all of the files are exported because it has got stuck on this one.

OS: Linux Mint
KoHighlights: 1.5.0.0

You can help support the export of My Clippings in Kindle format Txt format?

If I use the koreader export format (html or text), I cannot import it into the flomo through the kindle2Flomo
https://tit1e.github.io/kindle2Flomo/

Metadata File Not Parsed

The attached file exists in a folder structure that contains many other folders with similar files, all of which are processed by KOHighlights and rendered in the GUI. This one alone is not parsed at all - it does not appear in the GUI.

(removed as issue resolved)

I've tried removing the multi-line strings on a lark, but that had no effect.

It is not archived.

Exporting to Markdown

First of all, thank you so much for developing this very useful software.

Just for your consideration, I would love to be able to export directly to markdown.

Thanks!

Choose which details (Date, Title..) to export and which not ... possible?

Tried Kohighlight today. Its great!

I just wonder if its possible to select the details one want to export. even possible to export to .md?
Otherwise my exports a cluttered and the essential highlights dont get the attention..

cherio

example Kobonotes with less details:

Not finding *.sdr directories recursively?

Hello,

Thank you for KoHighlights. I have used it for a few years to transfer highlights over. Recently I have noticed it is not finding sdr directories recursively, and to show highlights for a book I have individually go into each books *.sdr folder to add it to KoHighlights. In the past, just opening the Kobo root directory would populate KoReader with all the books with highlights.

Is there something I need to change?

Export/ to individual markdown files

I'm trying to export the highlights using the command "Export/ to individual markdown files", but all the highlights continue to go to a single file, regardless of the option I choose

any ideas?