Buku

buku is a powerful bookmark management utility written in Python3 and SQLite3. When I started writing it, I couldn't find a flexible cmdline solution with a private, portable, merge-able database along with browser integration. Hence, buku (after my son's nickname).

With tagging and multiple options to search bookmarks, including regex and a deep scan mode (particularly for URLs), finding a bookmark is very easy. Multiple search results can be opened in the browser at once.

Though a terminal utility, it's possible to add bookmarks to buku without touching the terminal! Refer to the section on GUI integration. If you prefer the terminal, thanks to the shell completion scripts, you don't need to memorize any of the options. There's an Easter egg to revisit random forgotten bookmarks too.

Buku is too busy to track you - no history, obsolete records, usage analytics or homing.

Features
Installation
Shell completion
Usage
- Cmdline options
- Operational notes
GUI integration
- Add bookmarks from anywhere
- Import bookmarks to browser
Sync database across systems
As a library
Examples
Contributions
Mentions
Copyright

Features

Add, open, tag, comment on, update, remove, shorten URLs
Fetch page title from web, refresh all titles in a go
Multiple search options, continuous search at prompt
Portable, merge-able database to sync between systems
Import/export in Markdown or HTML (FF, Chrome compatible)
Open (multiple) search results directly in default browser
Manual password protection using AES256 encryption
Completion scripts (Bash, Fish, Zsh), man page with examples
Several options for power users (see help or man page)
Fast and clean interface, distinct symbols for record fields
Minimal dependencies

Installation

Dependencies

buku requires Python 3.3 or later.

To install package dependencies, run:

$ sudo pip3 install urllib3 cryptography beautifulsoup4 requests

or on Ubuntu:

$ sudo apt-get install python3-urllib3 python3-cryptography python3-bs4 python3-requests

Installing from this repository

If you have git installed, run:

$ git clone https://github.com/jarun/Buku/

or download the latest stable release or development version.

Install to default location (/usr/local):

$ sudo make install

To remove, run:

$ sudo make uninstall

PREFIX is supported. You may need to use sudo with PREFIX depending on your permissions on destination directory.

Running as a standalone utility

buku is a standalone utility. From the containing directory, run:

$ chmod +x buku.py
$ ./buku.py

Debian package

If you are on a Debian (including Ubuntu) based system visit the latest stable release and download the .deb package. To install, run:

$ sudo dpkg -i buku-$version-all.deb

Please substitute $version with the appropriate package version.

Installing with a package manager

buku is also available on

AUR for Arch Linux
Ubuntu
Homebrew for OS X
Debian Sid
Ubuntu PPA
Void Linux repos ( $ sudo xbps-install -S buku )

Shell completion

Shell completion scripts for Bash, Fish and Zsh can be found in respective subdirectories of auto-completion/. Please refer to your shell's manual for installation instructions.

buku has a rofi frontend written by Rasmus Steinke.

Usage

Cmdline options

usage: buku [OPTIONS] [KEYWORD [KEYWORD ...]]

Powerful command-line bookmark manager. Your mini web!

general options:
  -a, --add URL [tag, ...]
                       bookmark URL with comma-separated tags
  -u, --update [...]   update fields of bookmark at DB indices
                       accepts indices and ranges
                       refresh all titles, if no arguments
                       refresh titles of bookmarks at indices,
                       if no edit options are specified
  -d, --delete [...]   delete bookmarks. Valid inputs: either
                       a hyphenated single range (100-200),
                       OR space-separated indices (100 15 200)
                       delete results with search options
                       delete all bookmarks, if no arguments
  -h, --help           show this information and exit

edit options:
  --url keyword        specify url, works only with -u option
  --tag [+|-] [...]    set comma-separated tags with -a and -u
                       clear tags, if no arguments
                       append to tags, if preceded by '+'
                       remove from tags, if preceded by '-'
  -t, --title [...]    manually set title, works with -a, -u
                       if no arguments:
                       -a: do not set title, -u: clear title
  -c, --comment [...]  description of the bookmark, works with
                       -a, -u; clears comment, if no arguments
  --immutable N        disable title fetch from web on update
                       works with -a, -u
                       N=0: mutable (default), N=1: immutable

search options:
  -s, --sany keyword [...]
                       find records with ANY search keyword
  -S, --sall keyword [...]
                       find records with ALL search keywords
                       special keywords -
                       "blank": entries with empty title/tag
                       "immutable": entries with locked title
  --deep               match substrings ('pen' matches 'opens')
  --sreg expression    run a regex search
  --stag [...]         search bookmarks by a tag
                       list all tags, if no arguments

encryption options:
  -l, --lock [N]       encrypt DB file with N (> 0, default 8)
                       hash iterations to generate key
  -k, --unlock [N]     decrypt DB file with N (> 0, default 8)
                       hash iterations to generate key

power toys:
  -e, --export file    export bookmarks to Firefox format html
                       use --tag to export only specific tags
  -i, --import file    import bookmarks from html file
                       FF and Google Chrome formats supported
  --markdown           use markdown with -e and -i
                       format: [title](url), 1 per line
  -m, --merge file     merge records from another buku DB file
  -p, --print [...]    show details of bookmark by DB index
                       accepts indices and ranges
                       show all bookmarks, if no arguments
  -f, --format N       limit fields in -p or Json search output
                       1: URL, 2: URL and tag, 3: title
  -r, --replace oldtag [newtag ...]
                       replace oldtag with newtag everywhere
                       delete oldtag, if no newtag
  -j, --json           Json formatted output for -p and search
  --noprompt           do not show the prompt, run and exit
  -o, --open [N]       open bookmark at DB index N in browser
                       open a random index if N is omitted
  --shorten N/URL      fetch shortened url from tny.im service
                       accepts either a DB index or a URL
  --tacit              reduce verbosity
  --upstream           check latest upstream version available
  -z, --debug          show debug information and verbose logs

symbols:
  >                    title
  +                    comment
  #                    tags

Operational notes

The database file is stored in:
- $XDG_DATA_HOME/buku/bookmarks.db, if XDG_DATA_HOME is defined (first preference) or
- $HOME/.local/share/buku/bookmarks.db, if HOME is defined (second preference) or
- the current directory.
If the URL contains characters like ;, & or brackets they may be interpreted specially by the shell. To avoid it, add the URL within single or double quotes ('/").
URLs are unique in DB. The same URL cannot be added twice.
Bookmarks with immutable titles are listed with bold (L) after the URL.
Tags:
- Comma (,) is the tag delimiter in DB. A tag cannot have comma(s) in it. Tags are filtered (for unique tags) and sorted. Tags are stored in lower case and can be replaced, appended or deleted.
Update operation:
- If --title, --tag or --comment is passed without argument, clear the corresponding field from DB.
- If --url is passed (and --title is omitted), update the title from web using the URL.
- If indices are passed without any other options (--url, --title, --tag, --comment and --immutable), read the URLs from DB and update titles from web. Bookmarks marked immutable are skipped.
Delete operation:
- When a record is deleted, the last record is moved to the index.
- Delete doesn't work with range and indices provided together as arguments. It's an intentional decision to avoid extra sorting, in-range checks and to keep the auto-DB compaction functionality intact. On the same lines, indices are deleted in descending order.
- Can delete bookmarks matching a search, when combined with any of the search options.
Search works in mysterious ways:
- Case-insensitive.
- Matches exact words in URL, title and tags.
- --sany : match any of the keywords in URL, title or tags.
- --sall : match all the keywords in URL, title or tags.
- --deep : match substrings (match matches rematched) in URL, title and tags.
- --sreg : match a regular expression (ignores --deep).
- --stag : search bookmarks by a tag, or list all tags alphabetically with usage count (if no arguments).
- Search results are indexed serially. This index is different from actual database index of a bookmark record which is shown in bold within [] after the URL.
Encryption is optional and manual. AES256 algorithm is used. To use encryption, the database file should be unlocked (-k) before using buku and locked (-l) afterwards. Between these 2 operations, the database file lies unencrypted on the disk, and NOT in memory. Also, note that the database file is unencrypted on creation.
Proxy support: environment variable https_proxy, if defined, is used to tunnel data for both http and https connections. The supported format is:
```
  http[s]://[username:password@]proxyhost:proxyport/
```

GUI integration

buku can be integrated in a GUI environment with simple tweaks.

Add bookmarks from anywhere

With support for piped input, it's possible to add bookmarks to buku using keyboard shortcuts on Linux and OS X. CLIPBOARD (plus PRIMARY on Linux) text selections can be added directly this way. The additional utility required is xsel (on Linux) or pbpaste (on OS X).

The following steps explore the procedure on Linux with Ubuntu as the reference platform.

To install xsel on Ubuntu, run:
```
 $ sudo apt install xsel
```
Create a new script bukuadd with the following content:
```
 #!/bin/bash

 xsel | buku -a
```

-a is the option to add a bookmark. 3. Make the script executable:

    $ chmod +x bukuadd

Copy it somewhere in your PATH.
Add a new keyboard shortcut to run the script. I use <Alt-b>.

Test drive

Select a URL anywhere or copy a link and press the keyboard shortcut to add it to the buku database. The addition might take a few seconds to reflect depending on your internet speed and the time buku needs to fetch the title from the URL. To avoid title fetch from the web, add the -t option to the script.

To verify that the bookmark has indeed been added, run:

$ buku -p | tail -3

and check the entry.

Tips

To add the last visited URL in Firefox to buku, use the following script:

  #!/bin/bash

  sqlite3 $HOME/.mozilla/firefox/*.default/places.sqlite "select url from moz_places where last_visit_date=(select max(last_visit_date) from moz_places)" | buku -a

If you want to tag these bookmarks, look them up later using:
```
  $ buku -S blank
```

Use option -u to tag these bookmarks.

Import bookmarks to browser

buku can export (or import) bookmarks in HTML format recognized by Firefox, Google Chrome and Internet Explorer.

To export all bookmarks, run:

$ buku --export path_to_bookmarks.html

To export specific tags, run:

$ buku --export path_to_bookmarks.html --tag tag 1, tag 2

Once exported, import the html file in your browser.

Sync database across systems

buku has the capability to import records from another buku database file. However, users with a cloud service client installed on multiple systems can keep the database synced across these systems automatically. To achieve this store the actual database file in a synced directory and create a symbolic link to it in the location where the database file would exist otherwise. For example, $HOME/.local/share/buku/bookmarks.db can be a symbolic link to ~/synced_dir/bookmarks.db.

As a library

buku can be used as a powerful bookmark management library. All functionality are available through carefully designed APIs. main() is a good usage example. It's also possible to use a custom database file in multi-user scenarios. Check out the documentation for the following APIs which accept an optional argument as database file:

BukuDb.initdb(dbfile=None)
BukuCrypt.encrypt_file(iterations, dbfile=None)
BukuCrypt.decrypt_file(iterations, dbfile=None)

NOTE: This flexibility is not exposed in the program.

Examples

Add a bookmark with tags linux news and open source, comment Informative website on Linux and open source, fetch page title from the web:

 $ buku -a https://tuxdiary.com linux news, open source -c Informative website on Linux and open source
 Title: [TuxDiary – Linux, open source, command-line, leisure.]
 Added at index 336

 336. https://tuxdiary.com
 > TuxDiary – Linux, open source, command-line, leisure.
 + Informative website on Linux and open source
 # linux news,open source

where, >: title, +: comment, #: tags 2. Add a bookmark with tags linux news and open source & immutable custom title Linux magazine:

    $ buku -a http://tuxdiary.com linux news, open source -t 'Linux magazine' --immutable 1
    336. http://tuxdiary.com (L)
    > Linux magazine
    # linux news,open source

Note that URL must precede tags. 3. Add a bookmark without a title (works for update too):

    $ buku -a http://tuxdiary.com linux news, open source -t

Update existing bookmark at index 15012014 with new URL, tags and comments, fetch title from the web:

 $ buku -u 15012014 --url http://tuxdiary.com/ --tag linux news, open source, magazine -c site for Linux utilities

Fetch and update only title for bookmark at 15012014:
```
 $ buku -u 15012014
```

Update only comment for bookmark at 15012014:

 $ buku -u 15012014 -c this is a new comment

Applies to --url, --title and --tag too. 7. Export bookmarks tagged tag 1 or tag 2 to HTML and markdown:

    $ buku -e bookmarks.html --tag tag 1, tag 2
    $ buku -e bookmarks.md --markdown --tag tag 1, tag 2

All bookmarks are exported if --tag is not specified. 8. Import bookmarks from HTML and markdown:

    $ buku -i bookmarks.html
    $ buku -i bookmarks.md --markdown

Delete only comment for bookmark at 15012014:
```
 $ buku -u 15012014 -c
```

Applies to --title and --tag too. URL cannot be deleted without deleting the bookmark. 10. Update or refresh full DB with page titles from the web:

    $ buku -u
    $ buku -u --tacit (show only failures and exceptions)

This operation does not modify the indexes, URLs, tags or comments. Only title is refreshed if fetched title is non-empty. 11. Delete bookmark at index 15012014:

    $ buku -d 15012014
    Index 15012020 moved to 15012014

The last index is moved to the deleted index to keep the DB compact. 12. Delete all bookmarks:

    $ buku -d

Delete a range or list of bookmarks:
```
$ buku -d 100-200
$ buku -d 100 15 200
```
Search bookmarks for ANY of the keywords kernel and debugging in URL, title or tags:
```
$ buku -s kernel debugging
```
Search bookmarks with ALL the keywords kernel and debugging in URL, title or tags:
```
$ buku -S kernel debugging
```
Search bookmarks tagged general kernel concepts:
```
$ buku --stag general kernel concepts
```
List all unique tags alphabetically:
```
$ buku --stag
```
Run a search and delete the results:
```
$ buku -s kernel debugging -d
```
Encrypt or decrypt DB with custom number of iterations (15) to generate key:
```
$ buku -l 15
$ buku -k 15
```

The same number of iterations must be specified for one lock & unlock instance. Default is 8, if omitted. 20. Show details of bookmarks at index 15012014 and ranges 20-30, 40-50:

    $ buku -p 20-30 15012014 40-50

Show all bookmarks with real index from database:
```
$ buku -p
$ buku -p | more
```
Replace tag 'old tag' with 'new tag':
```
$ buku -r 'old tag' new tag
```
Delete tag 'old tag' from DB:
```
$ buku -r 'old tag'
```
Append (or delete) tags 'tag 1', 'tag 2' to (or from) existing tags of bookmark at index 15012014:
```
$ buku -u 15012014 --tag + tag 1, tag 2
$ buku -u 15012014 --tag - tag 1, tag 2
```
Open URL at index 15012014 in browser:
```
$ buku -o 15012014
```
List bookmarks with no title or tags for bookkeeping:
```
$ buku -S blank
```
List bookmarks with immutable title:
```
$ buku -S immutable
```

Shorten URL www.google.com and the URL at index 20:

$ buku --shorten www.google.com
$ buku --shorten 20

the-wayvy / buku Goto Github PK

buku's Introduction

Buku

Table of Contents