plandes/zotsite

Export Zotero to a stand-alone web site

Zotsite: A Zotero Export Utility

This project exports your local Zotero library to a usable HTML website. This generated website has the following features:

• Easily access your papers, site snapshots, notes from a navigation tree.
• Provides metadata from collections and attachments (i.e. referenes etc).
• Display PDF papers and website snapshot (the latter as framed).
• Search function dynamically narrows down the papers you're looking for.
• Embed links to a specific collection, article, item, note etc.
• Export only a portion of your collection with regular expressions using the collection name.
• BetterBibtex integration.
• Snazzy look and feel from the latest Bootstrap CSS/Javascript library.

Documentation

See the full documentation.

Obtaining

The easist way to install the command line program is via the pip installer:

pip3 install zensols.zotsite

Binaries are also available on pypi.

Process

The tool does the following:

1. Exports the meta data (directory structure, references, notes, etc) from your Zotero library. On MacOS, this is done by querying the file system SQLite DB files.
2. Copies a static site that enables traversal of the exported data.
3. Copies your Zotero stored papers, snapshot (sites) etc.
4. Generates a navigation tree to easily find your papers/content.

Sample Site Demonstration

See the live demo, which provides a variety of resources found in my own library. Note: To my knowledge, all of these resources are free to distribute and violate no laws. If I've missed one, please create an issue.

Requirements

BetterBibtex plugin for Zotero.

Usage

The command line program has two modes: show configuration (a good first step) and to create the web site. You can see what the program is parsing from your Zotero library:

zotsite print

To create the stand-alone site, run the program (without the angle brackets):

zotsite export

If your library is not in the default $HOME/zotero directory you will need to change that path by making a zotsite.conf config file:

zotsite export --collection zotsite.conf

This will create the html files in the directory ./zotsite

See usage for more information.

Configuration File

Either an environment variable ZOTSITERC must be set or a -c configuration option must be given and point to a file to customize how the program works. See the test configuration file for an example and inline comments for more detail on how and what can be configured.

Ubuntu and Linux Systems with Python 3.5 or Previous Version

Please read this issue if you are installing a Ubuntu or any Linux system with Python 3.5 or previous version.

Command Line Help

Command line usage as provided with the --help option:

Usage: zotsite [list|export|print] [options]:

This project exports your local Zotero library to a usable HTML website.

Options:
  -h, --help [actions]                         show this help message and exit
  --version                                    show the program version and exit
  --level X                                    the application logger level, X is one of: debug, err, info, warn
  -c, --config FILE                            the configuration file

Actions:
list                                           list all actions and help
  --lstfmt <json|name|text>              text  the output format for the action listing

export (default)                               generate and export the zotero website
  --collection REGEX                           a regular expression used to filter "collection" nodes
  -o, --outputdir DIR                          the directory to dump the site; default to configuration file
  -s, --show                                   whether to browse to the created site (needs "pip install zensols.showfile")

print                                          print (sub)collections and papers in those collections as a tree
  --collection REGEX                           a regular expression used to filter "collection" nodes

Attribution

This software uses:

• Python 3
• jQuery version 3
• DataTables version 1.12
• Bootstrap version 4
• Tree View for Bootstrap
• Popper for tooltips
• Copy to Clipboard function

Screenshot

Also see the live demo.

Todo

• Make the site portion a proper Javascript site. Right now, all the mins are added in the distribution to same directory as the main navigation/content file.
• Add functionality to the disabled View button that drills down in a paper and finds a PDF or site to view withouth the user having to do this.
• Use something like zotxt to make this work with a plugin rather than directly against the SQLite DB.

Zotero Plugin Listing

This is listed as a plugin on the Zotero site.

Changelog

An extensive changelog is available here.

Community

Please star this repository and let me know how and where you use this API. Contributions as pull requests, feedback and any input is welcome.

License

MIT License

Copyright (c) 2019 - 2023 Paul Landes

zotsite's Issues

Download bibtex-entry from webpage

New feature:
It would be very useful for people browsing the publications if they could download the bibtex-entry for a item from the website, and/or the entire bibtex-file for all items listed on the site.

I could take a stab at a PR if I get some feasibility estimation from the author (and thanks for a super useful package)!

Integrating zensols.zotsite into Ubuntu 16.04 LTS

I have spent some time integrating zensols.zotsite into my Ubuntu 16.04 environment and I encountered a few issues. I have managed to solve them so that I could better understand the inner workings of zensols.zotsite. But I add these notes for other users of Ubuntu 16.04.

My starting point was with Python 2.7 and Python 3.5 installed. I made the mistake of installing zensols.zotsite using pip and then pip3. But invalid syntax messages showed in running the command zotsite print.

zensols.zotsite depends on Python 3.6 not Python 3.5. I installed Python 3.6 via a PPA installation but could not get pip3 to place the package with Python 3.6 site-packages (as in 3.5). Also another side issue popped up. The default Ubuntu 16.04 gnome-terminal (shell) depends on Python 3.5 and terminal does not launch in Python 3.6. This bug was read here ..

https://stackoverflow.com/questions/49794432/how-to-setup-a-pipenv-python-3-6-project-if-os-python-version-is-3-5

I realised that I should be using a virtual environment so that different Python versions can be used and I referred to a few more documents.

https://help.dreamhost.com/hc/en-us/articles/115000695551-Installing-and-using-virtualenv-with-Python-3

After going through the above steps I created a virtual environment - virtualenv-zotsite - and within that virtual environment I ran pip3 to install

pip3 install -U zensols.zotsite

The next hurdle was that zensols.zotsite was looking for a non existent config file ~/.zoterorc and I created this with an entry pointing to the target Zotero data directory.

Finally tested again and encountered an error in opening sqlite3 file and found that Zotero has to be shutdown before using the zotsite commands. This lock of file is explained in zotsite README.

Conclusion. The advice to Ubuntu 16.04 users is to install and run zensols.zotsite in a Python 3.6 virtualenv.

The zotsite commands now work as documented.
My intention is to now export output into an Electron app.

TypeError: __init__() got an unexpected keyword argument 'default_expect'

Hi, thanks for this helpful tool. When I was doing some test with it, errors was encountered. Here is the situation.

Reproduce

I want to try it with a fresh environment with docker.

1. docker pull python:3
2. docker run -it --rm -v /home/work/Zotero:/root/Zotero python:3 bash
3. pip3 install zensols.zotsite
4. zotsite print

The following error message was thrown.

Traceback (most recent call last):
  File "/usr/local/bin/zotsite", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/site-packages/zensols/zotsite/cli.py", line 53, in main
    cl.invoke()
  File "/usr/local/lib/python3.9/site-packages/zensols/cli/simple.py", line 245, in invoke
    meth, exec_obj = self.create_executor(args)
  File "/usr/local/lib/python3.9/site-packages/zensols/cli/simple.py", line 233, in create_executor
    exec_obj = self.executors[exec_name](params)
  File "/usr/local/lib/python3.9/site-packages/zensols/zotsite/cli.py", line 33, in <lambda>
    'executor': lambda params: SiteCli(**params),
  File "/usr/local/lib/python3.9/site-packages/zensols/zotsite/cli.py", line 10, in __init__
    config = AppConfig.from_args(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/zensols/zotsite/config.py", line 28, in from_args
    self = cls()
  File "/usr/local/lib/python3.9/site-packages/zensols/zotsite/config.py", line 12, in __init__
    super(AppConfig, self).__init__(*args, default_expect=True, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'default_expect'

It seems that zotsite is out of sync with https://github.com/plandes/util . Would you please fix it?

Choosing user profile

Greetings, I have multible Zotero Libraries. I switch between them with the -P flag. See this in the documentation: https://www.zotero.org/support/kb/multiple_profiles How do I make Zotsite connect to the correct library. Note: these are not collections. Zotsite does find one library but I want it to find a different one.

Include groups

It would be great if group libraries could be exported as well.

bug

$zotsite -c /Users/xxx/Documents/zotero/zotero.sqlite print

Traceback (most recent call last):
File "/usr/local/bin/zotsite", line 10, in
sys.exit(main())
File "/usr/local/lib/python3.7/site-packages/zensols/zotsite/cli.py", line 53, in main
cl.invoke()
File "/usr/local/lib/python3.7/site-packages/zensols/cli/simple.py", line 245, in invoke
meth, exec_obj = self.create_executor(args)
File "/usr/local/lib/python3.7/site-packages/zensols/cli/simple.py", line 217, in create_executor
config = self.get_config(params)
File "/usr/local/lib/python3.7/site-packages/zensols/cli/peraction.py", line 255, in get_config
conf = self._create_config(conf_file)
File "/usr/local/lib/python3.7/site-packages/zensols/cli/peraction.py", line 308, in _create_config
isinstance(conf.parser._interpolation, ExtendedInterpolation):
File "/usr/local/lib/python3.7/site-packages/zensols/config/iniconfig.py", line 94, in parser
self._conf = self._create_and_load_parser()
File "/usr/local/lib/python3.7/site-packages/zensols/config/iniconfig.py", line 80, in _create_and_load_parser
self._read_config_content(cpath, writer)
File "/usr/local/lib/python3.7/site-packages/zensols/config/iniconfig.py", line 62, in _read_config_content
writer.write(f.read())
File "/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa0 in position 27: invalid start byte

bug

zotsite print
reading SQLite file: /Users/time/Zotero/zotero.sqlite
lib (l1) (Library)
数学 (c79,i3158) (Collection)
微积分.pdf [document]None (i3158) (Item)
Traceback (most recent call last):
File "/usr/local/bin/zotsite", line 10, in
sys.exit(main())
File "/usr/local/lib/python3.7/site-packages/zensols/zotsite/cli.py", line 53, in main
cl.invoke()
File "/usr/local/lib/python3.7/site-packages/zensols/cli/simple.py", line 249, in invoke
getattr(exec_obj, meth)()
File "/usr/local/lib/python3.7/site-packages/zensols/zotsite/cli.py", line 14, in print_structure
self.site.print_structure()
File "/usr/local/lib/python3.7/site-packages/zensols/zotsite/site.py", line 93, in print_structure
self.walker.walk(self.library, PrintVisitor(writer))
File "/usr/local/lib/python3.7/site-packages/zensols/zotsite/domain.py", line 279, in walk
self.walk(c, visitor)
File "/usr/local/lib/python3.7/site-packages/zensols/zotsite/domain.py", line 279, in walk
self.walk(c, visitor)
File "/usr/local/lib/python3.7/site-packages/zensols/zotsite/domain.py", line 279, in walk
self.walk(c, visitor)
File "/usr/local/lib/python3.7/site-packages/zensols/zotsite/domain.py", line 276, in walk
visitor.enter_parent(parent)
File "/usr/local/lib/python3.7/site-packages/zensols/zotsite/domain.py", line 256, in enter_parent
self.writer.write(f"{' ' * (self.depth * 4)}{str(parent)} " +
File "/usr/local/lib/python3.7/site-packages/zensols/zotsite/domain.py", line 31, in str
return '{} ({})'.format(self.format_zobj(), self.id)
File "/usr/local/lib/python3.7/site-packages/zensols/zotsite/domain.py", line 150, in format_zobj
abs_path = self.path
File "/usr/local/lib/python3.7/site-packages/zensols/persist/annotation.py", line 379, in wrapped
return pwork(*argv, **kwargs)
File "/usr/local/lib/python3.7/site-packages/zensols/persist/annotation.py", line 219, in call
obj = self._do_work(*argv, **kwargs)
File "/usr/local/lib/python3.7/site-packages/zensols/persist/annotation.py", line 134, in _do_work
obj = self.do_work(*argv, **kwargs)
File "/usr/local/lib/python3.7/site-packages/zensols/persist/annotation.py", line 228, in do_work
return self.worker(*argv, **kwargs)
File "/usr/local/lib/python3.7/site-packages/zensols/zotsite/domain.py", line 141, in path
raise ValueError(f'unknown storage and not a file: {path}')
ValueError: unknown storage and not a file: attachments:微积分.pdf

Keyerror: 'nodes'

Hi,
after figuring out, that --datadir is not parsed (its not in the Helpfile so why should it) and I got everything to rund with the Config file, I am now stopped at the above error, which I cannot cure.

I updated the Zenols.actioncli an zenols.zotsite to current versions allready

`
c:\Program Files (x86)\Microsoft Visual Studio\Shared\Python39_64\Scripts>zotsite export -o c:/LaufwerkU/Sites/Zotero -c c:/laufwerkU/scripte/zotsite.cnf

copying static data -> c:\LaufwerkU\Sites\Zotero
reading SQLite file: c:\LaufwerkU\Zotero\zotero.sqlite
creating js nav tree: c:\LaufwerkU\Sites\Zotero\js\zotero-tree-data.js

Traceback (most recent call last):

File "c:\program files (x86)\microsoft visual studio\shared\python39_64\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "c:\program files (x86)\microsoft visual studio\shared\python39_64\lib\runpy.py", line 87, in run_code
exec(code, run_globals)
File "c:\Program Files (x86)\Microsoft Visual Studio\Shared\Python39_64\Scripts\zotsite.exe_main.py", line 7, in
File "c:\program files (x86)\microsoft visual studio\shared\python39_64\lib\site-packages\zensols\zotsite\cli.py", line 53, in main
cl.invoke()
File "c:\program files (x86)\microsoft visual studio\shared\python39_64\lib\site-packages\zensols\cli\simple.py", line 251, in invoke
getattr(exec_obj, meth)()
File "c:\program files (x86)\microsoft visual studio\shared\python39_64\lib\site-packages\zensols\zotsite\cli.py", line 17, in export
self.site.export()
File "c:\program files (x86)\microsoft visual studio\shared\python39_64\lib\site-packages\zensols\zotsite\site.py", line 171, in export
self._create_tree_data()
File "c:\program files (x86)\microsoft visual studio\shared\python39_64\lib\site-packages\zensols\zotsite\site.py", line 123, in _create_tree_data
f.write(json.dumps(visitor.primary_roots, indent=2))
File "c:\program files (x86)\microsoft visual studio\shared\python39_64\lib\site-packages\zensols\zotsite\navvisitor.py", line 52, in primary_roots
return self.root['nodes'][0]['nodes']
KeyError: 'nodes'
`
Any Idea what this is about?

Michael

Environment for Windows

Hi!
I am trying to use zotsite (version 0.2.3) for Windows 10 as OS. Unfortunaly an error occurs calling zotsite from the command line. The error indicates that the environment 'Home' defined in cli.py is inkown. According to this link 'Home' is unkown in Windows OS. Therefor I changed the cli.py from
default_config_file = '%s/.zotsiterc' % os.environ['HOME']
to

            if platform.system()=='Linux':
                default_config_file = '%s/.zotsiterc' % os.environ['HOME']
            if platform.system()=='Windows':
                default_config_file = '%s/.zotsiterc' % os.path.expanduser('~')

and added 'import platform'.

But unfortunaly a new error occurs:

Traceback (most recent call last):
File "c:\users\XXXX\appdata\local\programs\python\python37-32\lib\runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "c:\users\XXXX\appdata\local\programs\python\python37-32\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\Users\XXXX\AppData\Local\Programs\Python\Python37-32\Scripts\zotsite.exe_main.py", line 9, in
File "c:\users\friebe\appdata\local\programs\python\python37-32\lib\site-packages\zensols\zotsite\cli.py", line 62, in main
cl.invoke()
File "c:\users\XXXX\appdata\local\programs\python\python37-32\lib\site-packages\zensols\actioncli\simple.py", line 198, in invoke
meth, exec_obj = self.create_executor(args)
File "c:\users\XXXX\appdata\local\programs\python\python37-32\lib\site-packages\zensols\actioncli\simple.py", line 176, in create_executor
config = self.get_config(params)
File "c:\users\XXXX\appdata\local\programs\python\python37-32\lib\site-packages\zensols\actioncli\peraction.py", line 236, in get_config
logger.debug('created config: %s' % conf)
File "c:\users\XXXX\appdata\local\programs\python\python37-32\lib\site-packages\zensols\actioncli\config.py", line 327, in str
format(self.config_file, self.sections))
File "c:\users\XXXX\appdata\local\programs\python\python37-32\lib\site-packages\zensols\actioncli\config.py", line 287, in sections
secs = self.parser.sections()
File "c:\users\XXXX\appdata\local\programs\python\python37-32\lib\site-packages\zensols\actioncli\config.py", line 179, in parser
raise IOError('no such file: {cfile}')
OSError: no such file: {cfile}

BTW after installing zotsite, no '/.zotsiterc' was created in the Home-Environment.

I will appreciate any help.

Best regards!

Please add >=python3.9 to requirements section

Thank you for this wonderful tool! I would like to suggest that you add >=python3.9 as a requirement on the README.md page. I am still running Ubuntu 20.04 with Python3.8 as default and I kept getting tracebacks until I switched to a venv with python3.9. I think specifying the Python version would be helpful for people like me that are stuck in dependency Hell and are afraid of breaking their systems by installing a version of Python outside of the repos. 😅

What is <sitedir>

Is that my domain?

Zotfile incompatability again

I'm having exactly the same issue as what was reported here:
#8

But I'm using zotsite version 0.9.1, with zotfile installed this year (2024) and Zotero version 6.0.30

When I export a collection that has .pdfs that have not been moved by zotfile it works fine.

However any items that do have attached .pdfs moved with zotfile fail with a FileNotFoundErrror, such as:
FileNotFoundError: [Errno 2] No such file or directory: '/home/m/Zotero/storage/73T7AU4N/Conference Paper/.pdf'
Showing that it seems to be mixing the original zotero storage with the zotfile path (as I have it put it into a directory by type, so Journals are in a folder, Thesis are in a folder etc).

Example site down

Hello,

the live demo points to a 404 page.

bug2

$zotsite -c /Users/xxx/Documents/zotero/zotero.sqlite print

Traceback (most recent call last):
File "/usr/local/bin/zotsite", line 10, in
sys.exit(main())
File "/usr/local/lib/python3.7/site-packages/zensols/zotsite/cli.py", line 53, in main
cl.invoke()
File "/usr/local/lib/python3.7/site-packages/zensols/cli/simple.py", line 245, in invoke
meth, exec_obj = self.create_executor(args)
File "/usr/local/lib/python3.7/site-packages/zensols/cli/simple.py", line 217, in create_executor
config = self.get_config(params)
File "/usr/local/lib/python3.7/site-packages/zensols/cli/peraction.py", line 255, in get_config
conf = self._create_config(conf_file)
File "/usr/local/lib/python3.7/site-packages/zensols/cli/peraction.py", line 308, in _create_config
isinstance(conf.parser._interpolation, ExtendedInterpolation):
File "/usr/local/lib/python3.7/site-packages/zensols/config/iniconfig.py", line 94, in parser
self._conf = self._create_and_load_parser()
File "/usr/local/lib/python3.7/site-packages/zensols/config/iniconfig.py", line 80, in _create_and_load_parser
self._read_config_content(cpath, writer)
File "/usr/local/lib/python3.7/site-packages/zensols/config/iniconfig.py", line 62, in _read_config_content
writer.write(f.read())
File "/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa0 in position 27: invalid start byte

zotsite -c /Users/time/Documents/zotero/zotero.sqlite export ./zot
Traceback (most recent call last):
File "/usr/local/bin/zotsite", line 10, in
sys.exit(main())
File "/usr/local/lib/python3.7/site-packages/zensols/zotsite/cli.py", line 53, in main
cl.invoke()
File "/usr/local/lib/python3.7/site-packages/zensols/cli/simple.py", line 245, in invoke
meth, exec_obj = self.create_executor(args)
File "/usr/local/lib/python3.7/site-packages/zensols/cli/simple.py", line 217, in create_executor
config = self.get_config(params)
File "/usr/local/lib/python3.7/site-packages/zensols/cli/peraction.py", line 255, in get_config
conf = self._create_config(conf_file)
File "/usr/local/lib/python3.7/site-packages/zensols/cli/peraction.py", line 308, in _create_config
isinstance(conf.parser._interpolation, ExtendedInterpolation):
File "/usr/local/lib/python3.7/site-packages/zensols/config/iniconfig.py", line 94, in parser
self._conf = self._create_and_load_parser()
File "/usr/local/lib/python3.7/site-packages/zensols/config/iniconfig.py", line 80, in _create_and_load_parser
self._read_config_content(cpath, writer)
File "/usr/local/lib/python3.7/site-packages/zensols/config/iniconfig.py", line 62, in _read_config_content
writer.write(f.read())
File "/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa0 in position 27: invalid start byte

Zotfile incompatibility

I use Zotfile, which renames and moves my PDF to a different folder.

When trying to do a zotsite export I get this kind of error:

FileNotFoundError: [Errno 2] No such file or directory: '/Users/odelalleau/Zotero/storage/BCH5KBQL/Volumes/GoogleDrive/Mon disque/bib/Cobbe+al-2019_Leveraging-procedural-generation-to-benchmark-reinforcement-learning.pdf'

Here, the folder /Users/odelalleau/Zotero/storage/BCH5KBQL exists but is empty (or at least almost empty, it contains some full-text index information), and the PDF is actually inside /Volumes/GoogleDrive/Mon disque/bib/ (Zotfile's custom file location)

Live demo not working

Hi,

Seems a great plugin but cann't see it in action. Live demo site gives a 404 error

Zotsite cannot run on Windows

When I run zotsite on Windows 10, I get the error:

Traceback (most recent call last): 
File "c:\anaconda\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
File "c:\anaconda\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
File "C:\Anaconda\Scripts\zotsite.exe\__main__.py", line 9, in <module>
File "c:\anaconda\lib\site-packages\zensols\zotsite\cli.py", line 56, in main
    cl = ConfAppCommandLine()
File "c:\anaconda\lib\site-packages\zensols\zotsite\cli.py", line 14, in __init__
    default_config_file = '%s/.zotsiterc' % os.environ['HOME']
File "c:\anaconda\lib\os.py", line 669, in __getitem__
    raise KeyError(key) from None
KeyError: 'HOME'

Zotsite cannot run on Windows because there is no environment variable HOME in Windows. I think maybe %USERPROFILE% is a good alternate.

.OperationalError: database is locked

File "c:\users\mydjt\appdata\local\programs\python\python39\lib\site-packages\zensols\zotsite\db.py", line 174, in _select_collections
for row in conn.execute(self._collection_sql(wparams)):
sqlite3.OperationalError: database is locked

Manuscript items cause zotsite to fail

I've replicated this issue on two different machines. With a small set of test entries in a library, zotsite works fine.

(base) user@Machine ~ $ zotsite export -o www/html/zotero
reading SQLite file: /home/user/Zotero/zotero.sqlite
copy: resources/site -> www/html/zotero
copy: resources/site/fonts -> www/html/zotero/fonts
copy: resources/site/img -> www/html/zotero/img
copy: resources/site/css -> www/html/zotero/css
copy: resources/site/js -> www/html/zotero/js
creating js nav tree: www/html/zotero/js/zotero-tree-data.js
copying storage /home/user/Zotero/storage -> www/html/zotero/storage
storage directory already exists--skipping: www/html/zotero/storage

As soon as I change the type of an entry from journal article to manuscript, zotsite fails with the following error:

(base) user@Machine ~ $ zotsite export -o www/html/zotero
reading SQLite file: /home/user/Zotero/zotero.sqlite
copy: resources/site -> www/html/zotero
copy: resources/site/fonts -> www/html/zotero/fonts
copy: resources/site/img -> www/html/zotero/img
copy: resources/site/css -> www/html/zotero/css
copy: resources/site/js -> www/html/zotero/js
creating js nav tree: www/html/zotero/js/zotero-tree-data.js
Traceback (most recent call last):
  File "/home/user/anaconda3/bin/zotsite", line 11, in <module>
    sys.exit(main())
  File "/home/user/anaconda3/lib/python3.6/site-packages/zensols/zotsite/cli.py", line 47, in main
    cl.invoke()
  File "/home/user/anaconda3/lib/python3.6/site-packages/zensols/actioncli/simple.py", line 156, in invoke
    getattr(exec_obj, meth)()
  File "/home/user/anaconda3/lib/python3.6/site-packages/zensols/zotsite/site.py", line 70, in export
    self._create_tree_data()
  File "/home/user/anaconda3/lib/python3.6/site-packages/zensols/zotsite/site.py", line 33, in _create_tree_data
    ZoteroObject.walk(lib, walker)
  File "/home/user/anaconda3/lib/python3.6/site-packages/zensols/zotsite/domain.py", line 38, in walk
    ZoteroObject.walk(c, walker)
  File "/home/user/anaconda3/lib/python3.6/site-packages/zensols/zotsite/domain.py", line 38, in walk
    ZoteroObject.walk(c, walker)
  File "/home/user/anaconda3/lib/python3.6/site-packages/zensols/zotsite/domain.py", line 35, in walk
    walker.enter_parent(parent)
  File "/home/user/anaconda3/lib/python3.6/site-packages/zensols/zotsite/nav.py", line 49, in enter_parent
    new_par = self.create_node(parent)
  File "/home/user/anaconda3/lib/python3.6/site-packages/zensols/zotsite/nav.py", line 30, in create_node
    icon = self.icon_name(item)
  File "/home/user/anaconda3/lib/python3.6/site-packages/zensols/zotsite/nav.py", line 22, in icon_name
    'webpage': 'bookmark'}[node.type]
KeyError: 'manuscript'

Deletion of the entry alone does not resolve the issue; the trash must be emptied as well. Once the trash has been emptied, zotsite works fine again.

*.md files cause issues

Thanks for the extension. I can't see any warning about my issue. I added solution here for those who cannot fix it. FYI.

I made it work after discovered the problem in debug message;
ValueError: unknown storage and not a file: /Users/alptugan/Documents/Obsidian/Extracted/simondonModeExistenceTechnical2011.md

Some of my Zotero items have *.md files because of another extension. After I've deleted all *.md files in Zotero app, I could run the 'zotsite export' command successfully.

KeyError: 'nodes'

Hi,

I'm super new to using git but I love you plugin so I wanted to give it a try. I installed zotsite in mac tru terminal using pip3 and tried running the following command: zotsite export -o https://antiracismhmsgenetics.github.io/DiversityJC

I'm getting the following error:
copying static data -> https:/antiracismhmsgenetics.github.io/DiversityJC
reading SQLite file: /Users/yasmin/Zotero/zotero.sqlite
creating js nav tree: https:/antiracismhmsgenetics.github.io/DiversityJC/js/zotero-tree-data.js
Traceback (most recent call last):
File "/opt/anaconda3/bin/zotsite", line 8, in
sys.exit(main())
File "/opt/anaconda3/lib/python3.7/site-packages/zensols/zotsite/cli.py", line 53, in main
cl.invoke()
File "/opt/anaconda3/lib/python3.7/site-packages/zensols/cli/simple.py", line 202, in invoke
getattr(exec_obj, meth)()
File "/opt/anaconda3/lib/python3.7/site-packages/zensols/zotsite/cli.py", line 17, in export
self.site.export()
File "/opt/anaconda3/lib/python3.7/site-packages/zensols/zotsite/site.py", line 162, in export
self._create_tree_data()
File "/opt/anaconda3/lib/python3.7/site-packages/zensols/zotsite/site.py", line 114, in _create_tree_data
f.write(json.dumps(visitor.primary_roots, indent=2))
File "/opt/anaconda3/lib/python3.7/site-packages/zensols/zotsite/navvisitor.py", line 52, in primary_roots
return self.root['nodes'][0]['nodes']
KeyError: 'nodes'

Should I have installed it a different way? Thanks in advance.

Support Zotero tags

New feature: import Zotero tags from zotero.sqlite and allow filtering items by tag on the web page.
A list of Tags is stored in the zotero.sqlite table tags and the itemTags table is the junction table linking items to tags.

I can try to implement this feature, but it surely requires huge adaptations to the structure of zotero-tree-data.js, to treeview and other key parts of zotsite

error pointing towards inexisting folder when exporting collection with REGEX-option

Situation: I have a collection (which in itself is a 'sub-collection') called 'CC Impacts' with several sub-folders inside:

When I run the command

zotsite export --collection ".*CC\s*Impacts.*" 

(1) the website is created in the root folder (which by the way looks and works great!), which includes the folder 'CC Impacts', all the sub-folders and all references.

but there seems to be an issue when the script looks for attachments as (2) following error appears (full output of command line):

zotsite: exporting site: None
zotsite: copying static data -> zotero-site
zotsite: creating js nav tree: zotero-site\js\zotero-tree-data.js
zotsite: reading SQLite file: C:\Users\USER\Zotero\zotero.sqlite
zotsite: no such icon found for dataset
zotsite: no such icon found for document
zotsite: no such icon found for document
zotsite: no such icon found for document
zotsite: no such icon found for document
zotsite: no such icon found for document
zotsite: no such icon found for document
zotsite: no such icon found for document
zotsite: no such icon found for document
zotsite: no such icon found for document
zotsite: no such icon found for document
zotsite: no such icon found for document
zotsite: no such icon found for document
zotsite: creating js metadata: zotero-site\js\zotero-meta.js
zotsite: copying storage to zotero-site
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\USER\AppData\Local\anaconda3\envs\zotsite\Scripts\zotsite.exe\__main__.py", line 7, in <module>
  File "C:\Users\USER\AppData\Local\anaconda3\envs\zotsite\Lib\site-packages\zensols\zotsite\cli.py", line 20, in main
    harness.invoke(args, **kwargs)
  File "C:\Users\USER\AppData\Local\anaconda3\envs\zotsite\Lib\site-packages\zensols\cli\harness.py", line 347, in invoke
    return cli.invoke(args[1:])
           ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\USER\AppData\Local\anaconda3\envs\zotsite\Lib\site-packages\zensols\cli\app.py", line 758, in invoke
    return self._handle_error(e)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\USER\AppData\Local\anaconda3\envs\zotsite\Lib\site-packages\zensols\cli\app.py", line 735, in _handle_error
    raise ex
  File "C:\Users\USER\AppData\Local\anaconda3\envs\zotsite\Lib\site-packages\zensols\cli\app.py", line 754, in invoke
    app_res: ApplicationResult = app.invoke()
                                 ^^^^^^^^^^^^
  File "C:\Users\USER\AppData\Local\anaconda3\envs\zotsite\Lib\site-packages\zensols\cli\app.py", line 409, in invoke
    res: Any = invokable()
               ^^^^^^^^^^^
  File "C:\Users\USER\AppData\Local\anaconda3\envs\zotsite\Lib\site-packages\zensols\cli\app.py", line 230, in __call__
    return self.method(*self.args, **self.kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\USER\AppData\Local\anaconda3\envs\zotsite\Lib\site-packages\zensols\zotsite\app.py", line 57, in export
    self.site_creator.export()
  File "C:\Users\USER\AppData\Local\anaconda3\envs\zotsite\Lib\site-packages\zensols\zotsite\screate.py", line 192, in export
    self._copy_storage()
  File "C:\Users\USER\AppData\Local\anaconda3\envs\zotsite\Lib\site-packages\zensols\zotsite\screate.py", line 153, in _copy_storage
    self.walker.walk(self.library, fsvisitor)
  File "C:\Users\USER\AppData\Local\anaconda3\envs\zotsite\Lib\site-packages\zensols\zotsite\domain.py", line 306, in walk
    self.walk(c, visitor)
  File "C:\Users\USER\AppData\Local\anaconda3\envs\zotsite\Lib\site-packages\zensols\zotsite\domain.py", line 306, in walk
    self.walk(c, visitor)
  File "C:\Users\USER\AppData\Local\anaconda3\envs\zotsite\Lib\site-packages\zensols\zotsite\domain.py", line 306, in walk
    self.walk(c, visitor)
  File "C:\Users\USER\AppData\Local\anaconda3\envs\zotsite\Lib\site-packages\zensols\zotsite\domain.py", line 305, in walk
    visitor.visit_child(c)
  File "C:\Users\USER\AppData\Local\anaconda3\envs\zotsite\Lib\site-packages\zensols\zotsite\fscpvisitor.py", line 53, in visit_child
    shutil.copyfile(src, dst)
  File "C:\Users\USER\AppData\Local\anaconda3\envs\zotsite\Lib\shutil.py", line 256, in copyfile
    with open(src, 'rb') as fsrc:
         ^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\USER\\Zotero\\storage\\9QFSS3TC\\false'

The folder C:\Users\dxf\Zotero\storage\9QFSS3TC\ does not exist (but the storage folder is the correct destination for Zotero-items).

When I run the debug mode (zotsite export --collection ".*CC\s*Impacts.*" --level debug), the full output is too large for my powershell to contain, but in the end it looks like the script is looking for an item named false in this inexisting folder (although I would expect a file-name as in the preceding example) :

zotsite: child: C:\Users\USER\Zotero\storage\9AYV8DDI\2017 - Iyer et al - Measuring progress from nationally determined contributions to mid-century.pdf
zotsite: copy: C:\Users\USER\Zotero\storage\9AYV8DDI\2017 - Iyer et al - Measuring progress from nationally determined contributions to mid-century.pdf -> zotero-site\storage\i25315.pdf
zotsite: child: None
zotsite: child: C:\Users\USER\Zotero\storage\9QFSS3TC\false
zotsite: copy: C:\Users\USER\Zotero\storage\9QFSS3TC\false -> zotero-site\storage\i26558
zotsite: shcopy: C:\Users\USER\Zotero\storage\9QFSS3TC\false -> zotero-site\storage\i26558
Traceback (most recent call last):
# same output as output without --level debug

To find this (seemingly inexisting?) reference in the SQlite-database, I used DB Browser for SQLite.

I am not familiar with the Zotero SQLite database-structure, but from the item-table it appears that the key 9QFSS3TC corresponds to itemID 26558, proving its existence:

In another table itemAttachments the itemID 26558 indicates that the storage location of this item is defined as storage:false.

This seems to be the culprit. I don't know whether this is native Zotero-behaviour or because of an add-on, but multiple items do have the value storage:false under path here.

In the table itemCreators it appears there is no author associated with the item.

In the table itemData there are three valueIDs associated with the item

In the table itemDataValues those three valueIDs point towards:

valueID	value
79617	2018 - papalexiou et al - global, regional, and megacity trends in the highest temperature of the year.pdf
79618	http://onlinelibrary.wiley.com/doi/10.1002/2017EF000709/pdf
79619	2018-01-24 14:33:53

The item itself look like any other item in Zotero, but the file is just only stored online (and not synced locally). This probably explains the value storage:false ?

Or maybe the problem is that Zotero did not yet create a folder associated with this item (because it's only online), and normally it should have created one?

Now I'm wondering: does this error cause the chain of parsing attachments to break, or did the code skip just only this erroneous file and proceed to the other files?

Export by Collection includes entire storage folder

When exporting by collection, I think it would make sense for the script to check against the actual files being referenced in that collection instead of copying the entire storage folder. I have well over 1GB in my library, but want to create a website for a small subset.

Sort by year

It would be a nice feature if the entries could be sorted by year

Zotsite doesn't find better-bibtex.sqlite

Since the access to better-bibtex.sqlite from outside Zotero has changed, zotfile doesn't work anymore in my Windows 11 installation of Zotero 6.0.27:
"Error: malformed database schema (autoexport_setting_insert) - trigger autoexport_setting_insert cannot reference objects in database betterbibtex"
According to
retorquere/zotero-better-bibtex#2684
there has to be issued the following code:
ATTACH DATABASE '' AS betterbibtex
As far as I understand @retorquere, this has to be done somewhere inside the zotsite code.
Anything I can do to fix it myself?

ImportError: No module named zensols.zotsite.cli

Hello all,
I installed zotsite via pip install zensols.zotsite in debian 9. No error.

When I try to give any command, I have the following output.
Am I missing something silly?

Thanks!

:~/Zotero$ zotsite print
Traceback (most recent call last):
  File "/usr/local/bin/zotsite", line 7, in <module>
    from zensols.zotsite.cli import main
ImportError: No module named zensols.zotsite.cli

Include custom CSS file in site generation

New feature:
Would be useful to be able to include a custom css file with the static site generation for styling according to e.g. institute standards or dark-mode.