michaelcurrin/url-manager

Centralize, view, edit, label and organize collections of your favorite URLs 🔗 📙

URL Manager 🔗 📙

Centralize, view, edit, label and organize collections of your favorite URLs

A meta bookmark manager to rule them all.

About

The aim is to help you create and find your URL more efficiently and within a personalised structure. The result is a tool for you to import browser bookmark-related URL data from various sources and manage them in a single structured, easy-to-search data source.

Import data from the following data sources:

• Bookmark files
  • Firefox
  • Chrome
• History file
• OneTab data
• A CSV created by hand in the required format

You can also add and manage records with a command-line tool.

Documentation

• Read this project's docs directory.
• plyvel Python library to access LevelDB.
  • This is used for getting OneTab data from Chrome's LevelDB storage.
  • This have proven to cause some issues and the location changed I think, so scraping the frontend (for Firefox and Chrome) or using the text export might be easier than using LevelDB.
  • See also this LevelDB Wiki page and article.

License

Released under MIT.

url-manager's Issues

Split browser tools into new repo

Create a new Python repo. Handle just OneTrust and Bookmark parsing for browsers.

Use Github action to publish package?

Then install from URL manager. Use Github URL? PyPI?

Name? just-browsing. high-brow

Add unit tests for parsing content.
Add integration tests to run on each system to check file path is still valid and content can be parsed in some minimal way. Like identify format or structure or count rows or checking keys.

Write OneTab data to configed directory.

See browser_one_tab_extraction.md, Instead of writing to stdout and a full path in the below sections, write to a file in the project using detailed parameters or a filename.

Add args to transformer.py

Use args to limit to printing preview or counting items without writing.
And to restrict to read only certain file types or file name instead of everything available.
It will called on everything in some cases but one individual files in others such as for development or when new data is needed to be used which wasn't around on initial imports.

Add config

Such as to set the profile for Firefox

abcd1234.default

If Firefox OneTab integration part of the script still works.

Split transformer.py

Split transformer into lib files which can be run standalone like a util. Or create user-facing parts of utils and transformer can use those or be a util itself.

Update transformer paths

transformer.py

Add logic to ensure folder's data is written to a unique key. As duplicate folder names are lost here at

folders[subfolder_name] = child_data

For identify_chrome_profiles.sh get the OS type from bash lib

Consider different DB and ORM

Lookup MongoDB example which uses links.

SQLObject gives flexibility of SQLite / Postgres for easy deployment pipeline.

Add Firefox instructions to browser_bookmark_extraction.md