bandantonio/obsidian-apple-books-highlights-plugin

Import highlights and notes from your Apple Books to Obsidian

Obsidian Apple Books highlights import plugin

Overview

This plugin aims to be a fast, customizable, reliable, and up-to-date solution to import your Apple Books highlights to Obsidian:

• ⚡️ Fast: It takes less than 1 second to import ~1000 highlights.

• 🎨 Customizable: Use Handlebars and Markdown to customize the output of your highlights the way you want. Check the Template variables section in documentation for more information.

• 🦾 Reliable:

  • Import actual highlights with only the metadata you need. No visual noise with the deleted but still exported highlights, or, on the contrary, highlights and notes that make no sense without the context.
  • Back up your highlights before each import to avoid accidental data loss (optional, but recommended).

• 📚 Dataview-ready: The default template creates highlights in a format that is compatible with Dataview, so you can use your highlights in Dataview queries to unleash possible use cases even further (for example, to create a list of books you've read).

• 🔄 Up-to-date: The plugin is updated regularly to support the latest versions of Obsidian, and Apple Books, as well as react to the users' feedback.

What users say

• 💬 Nice work. This is really quick. - cmyplay
• 💬 I just wanted to say that I love your plugin. It’s exactly what I need with no frills. - civilserf
• 💬 I am very grateful for the plugin you developed, which has made organizing my reading notes much more convenient. - JeffreyGH10
• 💬 Love this! great work! - dschense
• 💬 Thanks for sharing this nice plugin. - modatwork
• 💬 Your plugin is super fast and useful! - weslau

Try it out

Documentation

• Get Started
• Settings
• Templates and variables

Contributing

Ready to contribute? Check the Contributing guidelines to get started.

Your feedback and ideas are more than welcome and highly appreciated! Join the discussion in:

• Obsidian Forum
• GitHub Discussions

obsidian-apple-books-highlights-plugin's Issues

apostrophe

A similar affair across fields with '

It's..
You're...

etc

Not that big a deal, but would be good to catch.

Issue with Template Import

Description the bug
Hello bandantonio,
The template you provided in the document cannot be imported. Although it shows "Import Successful," the file does not appear in the folder. Could you please check what the issue might be? Thank you.

The template is as follows:

Title:: 📕 {{{bookTitle}}}
Author:: {{{bookAuthor}}}
Genre:: {{#if bookGenre}}{{{bookGenre}}}{{else}}N/A{{/if}}
Language:: {{#if bookLanguage}}{{bookLanguage}}{{else}}N/A{{/if}}
Last Read:: {{dateFormat bookLastOpenedDate "YYYY-MM-DD hh:mm:ss A Z"}}
Link:: [Apple Books Link](ibooks://assetid/{{bookId}})

{{#if bookCoverUrl}}![Book Cover]({{{bookCoverUrl}}}){{/if}}

## Annotations

Number of annotations:: {{annotations.length}}

{{#each annotations}}
----

- 📖 Chapter:: {{#if chapter}}{{{chapter}}}{{else}}N/A{{/if}}
- 🔖 Context:: {{#if contextualText}}{{{contextualText}}}{{else}}N/A{{/if}}
{{#if (eq highlightStyle "0")}}- 🎯 Highlight:: <u>{{{highlight}}}</u>{{/if}}
{{#if (eq highlightStyle "1")}}- 🎯 Highlight:: <mark style="background:rgb(175,213,151); color:#000; padding:2px;">{{{highlight}}}</mark>{{/if}}
{{#if (eq highlightStyle "2")}}- 🎯 Highlight:: <mark style="background:rgb(181,205,238); color:#000; padding:2px;">{{{highlight}}}</mark>{{/if}}
{{#if (eq highlightStyle "3")}}- 🎯 Highlight:: <mark style="background:rgb(249,213,108); color:#000; padding:2px;">{{{highlight}}}</mark>{{/if}}
{{#if (eq highlightStyle "4")}}- 🎯 Highlight:: <mark style="background:rgb(242,178,188); color:#000; padding:2px;">{{{highlight}}}</mark>{{/if}}
{{#if (eq highlightStyle "5")}}- 🎯 Highlight:: <mark style="background:rgb(214,192,238); color:#000; padding:2px;">{{{highlight}}}</mark>{{/if}}
- 📝 Note:: {{#if note}}{{{note}}}{{else}}N/A{{/if}}
- <small>📅 Highlight taken on:: {{dateFormat highlightCreationDate "YYYY-MM-DD hh:mm:ss A Z"}}</small>
- <small>📅 Highlight modified on:: {{dateFormat highlightModificationDate "YYYY-MM-DD hh:mm:ss A Z"}}</small>

{{/each}}

General information

• Plugin version: [1.2.4]
• Obsidian version: [1.6.5]
• MacOS version: [monterey12.7.5]

Plugin Roadmap

1.3.0

• #12

---

✅ 1.2.0 (Release notes)

• #2

---

✅ 1.1.1 (Release notes)

• Fix escaped HTML entities: #3
• Fix escaped HTML entities: #4

---

✅ 1.1.0 (Release notes)

• Import highlights from a specific book
• Option to import all books at startup of Obsidian

[Enhancement]: When backups are disabled, content overwrite during subsequent imports is not obvious

Bug Report Checklist

• I have searched the existing issues to avoid duplicates
• I have checked the plugin's documentation

Plugin version

1.3.0

Obsidian version

1.6.5

macOS version

macOS Monterey

What happened?

Hello bandantonio,
I encountered an issue while using the plugin. For example, when I choose to import the highlights from book "A," the import is successful, and the import file for "A" appears in the folder. However, when I subsequently choose to import the highlights from book "B," the import is successful, and the import file for "B" appears in the folder, but the import file for "A" disappears. I have repeated the test several times, and the same issue occurs every time. I hope you can check whether this is a bug. Thank you.

The issue occurs when

Importing a single book

How many books do you have in your Apple Books library?

20 or less

Code of Conduct

• I agree to follow this project's Code of Conduct

FR: Naming of Highlight File

I have some whole books that I've converted from ePub to HTML to Markdown in my vault.

---

With my workflow, I have daily notes in which I link the book if I read some part of it that day.
I collect this info in other periodic notes for other purposes.

Long story short, if the highlights coming in have the same title, some links in my daily notes will assume to link to the file with the highlights, rather than the book, so as well as folder, it would be a useful tweak if I could append something to the name of the file created. Eg: " - highlights"

Refactor: make helper functions more universal/modular

This issue is a continuation of the discussion from #28

Problem overview

All the highlights are accompanied by a so-called representative text that acts as a full version of the sentence from which the highlight is taken. Depending on a book structure and highlighted fragments, the representative text may contain excessive symbols, like trailing spaces, newlines, tabs, or a combination of them. This results in newlines within imported highlight blocks, that add confusion as it may seem like two separate highlight blocks. For example:

- 📖 Chapter:: N/A
- 🔖 Context:: əˈtämik
an extremely small amount of a thing; the single irreducible unit of a larger system.

- 🎯 Highlight:: əˈtämik
an extremely small amount of a thing; the single irreducible unit of a larger system.
- 📝 Note:: Link to atomic notes

The initial solution was proposed in #28 and it acts as a good starting point. However, the solution doesn't cover following cases (yet):

• Trailing spaces - along the way.
• Newlines and tabs
  • reboot?\n\n
  • project.\n\n\n
  • simple answers.\n\t\t
  • success.\n\t\t\t
  • instead.\n\t\t\t\t\n\t\t\t\t\n\t\t\t\t\t
  • a book\n\t\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\n\t\t\t\t\t\t\n\t\t\t\t\t\t\t

The proposed solution is similar to the function created for preserving indentation in text blocks: check preserveNewlineIndentation function in the src/methods/aggregateDetails.ts.

Proposal

Below are options under discussion:

1. Make the new helper more universal/modular to cover all the above mentioned cases. Modularity will keep the helper easy maintain and test, but may result in reduced code readability because if a text block requires multiple modifications, there will be a chain of helper functions performing these modifications.
2. Make a god-like helper function that takes all the helper functions and a text block as parameters and performs all the required modifications. As a result, there will be only one helper function to call but it may be harder to maintain and test this function with multiple "side effects".
3. More elegant solution to take the best from both worlds ?

Parse error near line 2: no such table: ZAEANNOTATION

Description the bug
The plugin is not working. not with the commands, not with the icon in sidebar.
getting this error in console output:

Error: Command failed: echo " SELECT ZASSETID, ZTITLE, ZAUTHOR, ZGENRE, ZLANGUAGE, ZLASTOPENDATE, ZCOVERURL FROM ZBKLIBRARYASSET WHERE ZPURCHASEDATE IS NOT NULL" | sqlite3 ~/Downloads/test-BKLibrary.sqlite -json Parse error near line 2: no such table: ZBKLIBRARYASSET

Error: Command failed: echo " SELECT ZANNOTATIONASSETID, ZFUTUREPROOFING5, ZANNOTATIONREPRESENTATIVETEXT, ZANNOTATIONSELECTEDTEXT, ZANNOTATIONNOTE, ZANNOTATIONCREATIONDATE, ZANNOTATIONMODIFICATIONDATE, ZANNOTATIONSTYLE FROM ZAEANNOTATION WHERE ZANNOTATIONSELECTEDTEXT IS NOT NULL AND ZANNOTATIONDELETED IS 0" | sqlite3 ~/Downloads/empty-AEAnnotation_v10312011_1727_local.sqlite -json Parse error near line 2: no such table: ZAEANNOTATION

Also include the following details:

• Error behavior: intermittent/permanent.
• Error occurs when:
  • Importing a single book
  • Importing all books
  • All the above
• Number of books and highlights (approximately)

General information

• Plugin version: 1.2.2
• Obsidian version: 1.5.12
• MacOS version: Sonoma 14.4

To Reproduce
Steps to reproduce the behavior:

1. option+command --> Apple Books import command
2. with icon in sidebar
3. See error

Expected behavior
importing some highlights

Screenshots

Additional context
getting this error in console:

[Bug] Error when callout meets newline

Hi bandantonio, thanks for sharing this nice plugin.

Description the bug

Also include the following details:

• Error behavior: permanent.
• Error occurs when:
  • Importing a single book
  • Importing all books
  • All the above
• 1 book and 1 highlight

General information

• Plugin version: 1.2.3
• Obsidian version: 1.6.5
• MacOS version: 13.5.2

To Reproduce
Steps to reproduce the behavior:

1. Create a highlight for a paragraph which contains newlines
2. Add Obsidian Callout syntax to the plugin template as showed below
3. Import highlights
4. The highlight is not quoted correctly

Expected behavior

All the text from the highlight is quoted.

Screenshots

the current output:

expected output:

Additional context

The template used:

{{#each annotations}}
----

> [!QUOTE]
>  {{{highlight}}}

{{/each}}

[Bug] ERR_CHILD_PROCESS_STDIO_MAXBUFFER error when trying to import highlights

Description the bug
A clear and concise description of what the bug is.

Also include the following details:

• Error behavior: intermittent/permanent.
• Error occurs when:
  • Importing a single book
  • Importing all books
  • All the above
• Number of books and highlights (approximately):

General information

• Plugin version: [e.g. 1.2.0]
• Obsidian version: [1.5.8]
• MacOS version: [Sonoma 14.3.1]

To Reproduce
Steps to reproduce the behavior:

1. Go to '...' Apple Books - Import Highlights -- Single Book
2. A blank popup appears, doesn't show my books
3. See error

Expected behavior
Highlights come through.

Screenshots

Cannot install the plugin on Obsidien v1.6.5

When trying to install the plugin in Obsidien, the following error appears:

Failed to install plugin "Apple Books - Import Highlights"

Using Obsidien (Current version: v1.6.5, Installer version: v1.6.5)

Tested on macOS Sonoma 14.2.1

Console tab

Obsidien (app.js)

It seems there is an error when making the request to fetch the plugin.

I think this is a problem with Obsidien itself and not to do with your plugin as I was able to install the plugin manually.

Update:
So I've managed to install it 'manually' using 'Build a plugin'
https://docs.obsidian.md/Plugins/Getting+started/Build+a+plugin

FR: Ordering Highlights

Had a thought considering you’ve just added the ability to import highlights of a specific book.

It might be of use to have the option (maybe a setting toggle) to be able to choose the order in which the highlights appear.

Some use cases that come to mind:

• by Creation Date
• by Last modified
• sequentially in the book

Pushing this even further, if used regularly, it might be useful to have different templates (layouts) for each use case.

In which case, keeping the one default as is, perhaps each sort option could have a text field assigned to use a specific_template.md from the vault?

FR: Colored annotations and additional fields to pass to the template

Have raised a PR for the same.

Aiming for something like this:

Will share a suggested template for anyone looking to do the same, once we have the fields.

bookTitle formatting

eg: "Edit Better: Hollywood-Tested Strategies for Powerful Video Editing"

This works for inline fields, but not for frontmatter due to formatting.
If the same code you have to create the title of the file could be applied to the field bookTitle this would format correctly.

Else I'm having to manually adjust these:

Highlights not listed in chronological order as expected.

The display of the highlights and notes appears to be shuffled. Not quite random, but not in sequential order; i.e. the order they appear in a book, resulting in confusion.

Error console when importing any

Description the bug
The plugin is not working. not with the commands, not with the icon in sidebar.
getting this error in console output:

Error importing Apple Books highlights: SyntaxError: Unexpected end of JSON input
eval @ plugin:apple-books-import-highlights:6154
Promise.catch (async)
eval @ plugin:apple-books-import-highlights:6152

Also include the following details:

• Error behavior: intermittent/permanent.
• Error occurs when:
  • Importing a single book
  • Importing all books
  • All the above
• Number of books and highlights (approximately)

General information

• Plugin version: 1.2.1
• Obsidian version: 1.5.12
• MacOS version: Sonoma 14.4.1

To Reproduce

• option+command --> Apple Books import command
• with icon in sidebar

Expected behavior
importing some highlights

Screenshots
no screeny - error text

Additional context
getting this error in console:

[Tests: e2e]:Add basic end-to-end tests

Feature Request Checklist

• I have searched the existing issues to avoid duplicates

What feature would you like to request?

At this moment, the plugin has important functionality covered with unit tests, but there are no end-to-end tests yet to provide closer to real-life user experience. There are many repositories on GitHub doing e2e for Obsidian plugins, so I tried to follow this path as well.

Currently, the test/basic-e2e branch contains the initial (super basic) configuration for Electron in WebdriverIO that creates a test vault environment inside Obsidian for testing (check helpers.js)

The problem is that after launching tests, the initialization fails with the following error:

 Error: Can't call click on element with selector "div.modal.mod-trust-folder > div.modal-button-container > button:nth-child(1)" because element wasn't found
at async Object.initializeTestVault (/test/e2e/helpers.js:26:3)
at async Context.<anonymous> (/test/e2e/e2e.spec.js:4:2)

The selector is correct, so it's kinda tricky for me to understand what I'm doing wrong with the webdriver.

The perfect goals at this stage would be:

1. Fix the initialization of the test vault configuration.
2. Go to the plugin's settings and check that the default configuration is set.

Tip

❗️Anyone with QA/e2e experience are welcome to contribute. 🙏🎉

Code of Conduct

• I agree to follow this project's Code of Conduct