Giter Site home page Giter Site logo

bookbook's Introduction

Bookbook converts a set of notebooks in a directory to HTML or PDF, preserving cross references within and between notebooks.

This code is in early development, so use it at your own risk.

Installation

Bookbook requires Python 3.5.

pip install bookbook

To install locally as an editable install, run:

pip install flit
git clone https://github.com/takluyver/bookbook.git
cd bookbook
flit install --symlink

Running bookbook

bookbook expects a directory of notebooks whose names indicate their order. Specifically, the file names must have the form ``x-y.ipynb``, where typically x is a number indicating the order and y is a chapter title; e.g.: 01-introduction.ipynb.

To run bookbook:

python3 -m bookbook.html           # HTML output under html/
python3 -m bookbook.latex [--pdf]  # Latex/PDF output as combined.(tex|pdf)

Add --help to either command for more options.

Chapters and sections

Each top-level header (# xyz in markdown) will be converted to a top-level latex section (a chapter if using the book or report document class). Lower-level headers (##, ###, etc.) are converted to subsections, subsubsections, etc. A latex label will also be inserted for each. The first cell of each notebook must start with a top-level header.

Cross-references

Markdown references will be converted automatically to latex references. For instance, if the markdown contains the hyperlink [02-foo](02-foo.ipynb) and 02-foo.ipynb is a notebook in the same directory, the link will appear as Chapter \ref{sec:02-foo}. The label \label{sec:02-foo} will be inserted at the start of that notebook, so when the latex is compiled to PDF it will appear as Chapter 2.

References to sections within a notebook work similarly. If a notebook contains (in markdown) the section heading ## bar within the notebook starting with top-level header # foo, then the markdown hyperlink [foo](foo.ipynb#bar) will be converted to the latex reference Section \ref{bar} and when compiled to PDF it will be rendered as something like Section 2.1.

Latex formatting

bookbook uses nbconvert under the hood. Custom formatting of latex output can be accomplished by using a template, in the same way as would be done using nbconvert by itself. See the nbconvert documentation for more details.

Examples of projects using bookbook

  • Book on Riemann solvers (in development) by David Ketcheson, Mauricio del Razo, and Randall LeVeque. This example uses a custom nbconvert template and shows how to store your notebooks with no output (for version control) while automatically executing them before running bookbook, so that PDF and HTML versions include the output.

Related tools

If you are writing a book in Jupyter notebooks, you may also find these to be useful:

  • nbopen: open notebooks from the command line without launching a new notebook server. We find it useful to launch a single server in your home directory; then nbopen will use that to open each notebook.
  • nbdime: diff/merge for notebooks; includes terminal or graphical output.
  • nbstripout: remove output from notebooks before committing them.

bookbook's People

Contributors

ketch avatar mpacer avatar takluyver avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

bookbook's Issues

Configuring Latex cross-link text

As of PR #7, the labels on Latex cross links look like: Chapter 2 or Section 2.1.

How do we let people configure these? Command line options? Notebook metadata? Nbconvert config? A separate bookbook config file? Custom templates?

AssertionError

Hi Thomas,

I try to run boobook with some notebooks, but I got an assertion error, maybe it is caused by my source_dir, any wrong with it?:
Thanks in advance, JL

C:\github\bookbook>python -m bookbook.latex --pdf --output-file test3 **C:/github/bookbook/Capitulo/**
Traceback (most recent call last):
  File "C:\Anaconda3\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Anaconda3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\github\bookbook\bookbook\latex.py", line 163, in <module>
    main()
  File "C:\github\bookbook\bookbook\latex.py", line 160, in main
    combine_and_convert(args.source_dir, args.output_file, args.pdf, args.template)
  File "C:\github\bookbook\bookbook\latex.py", line 144, in combine_and_convert
    combined_nb = combine_notebooks(notebook_files)
  File "C:\github\bookbook\bookbook\latex.py", line 77, in combine_notebooks
    combined_nb.cells.extend(add_sec_label(nb.cells[0], nbname))
  File "C:\github\bookbook\bookbook\latex.py", line 46, in add_sec_label
    assert cell.cell_type == 'markdown', cell.cell_type
AssertionError: code

Notebook links not converted correctly

I'm having trouble with the treatment of links to notebooks in latex output. No matter what I do, running bookbook just converts them to hyperlinks (\href).

If I make a test markdown file with a link like

[link](01-notebook.ipynb)

and run

pandoc --filter ./filter_links.py -f markdown -t latex test.md

it works fine; I get Section \ref{sec:01-notebook} as expected. I've also tried patching nbconvert by pasting bookbook's filter_links.py code into nbconvert's version, and then running nbconvert on a notebook with links; that works correctly.

It seems to me like the convert_link code in bookbook simply never gets called. Do you have a working example so that I could verify whether the problem is with my setup?

Rules for notebook filenames?

Currently, I think notebooks have to match the regex (\d+)-(.+)\.ipynb (e.g. 01-introduction.ipynb) for everything to work correctly. They are sorted lexicographically, and then the latex export uses its own numbering (so 10, 15, 20 would become 1, 2, 3), while HTML uses the numbers from the filenames.

  • Do we want to make the sorting smarter, so you can use 1-introduction instead of 01-introduction?
  • Do we want to make HTML renumber the sequence like Latex does?

docs?

This looks amazing. Could you maybe write some docs? Maybe I overlooked them. What's the syntax for specifying links? Currently nbconvert doesn't even have links inside a single notebook, right?

pathlib support bug - TypeError: can only concatenate str (not "WindowsPath") to str

raise NoHeader("Failed to find header in " + filename)

Full Traceback:

Traceback (most recent call last):
  File "C:\Python37\lib\site-packages\bookbook\latex.py", line 77, in combine_notebooks
    combined_nb.cells.extend(add_sec_label(nb.cells[0], nbname))
  File "C:\Python37\lib\site-packages\bookbook\latex.py", line 53, in add_sec_label
    raise NoHeader
__main__.NoHeader

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Python37\lib\site-packages\bookbook\latex.py", line 163, in <module>
    main()
  File "C:\Python37\lib\site-packages\bookbook\latex.py", line 160, in main
    combine_and_convert(args.source_dir, args.output_file, args.pdf, args.template)
  File "C:\Python37\lib\site-packages\bookbook\latex.py", line 144, in combine_and_convert
    combined_nb = combine_notebooks(notebook_files)
  File "C:\Python37\lib\site-packages\bookbook\latex.py", line 79, in combine_notebooks
    raise NoHeader("Failed to find header in " + filename)
TypeError: can only concatenate str (not "WindowsPath") to str

Looks like the line linked above just needs str(filename) so that the NoHeader error gets raised correctly. This won't solve my real problem of something to do with a header, but it's a bug nonetheless.

support for --TagRemovePreprocessor...

I have to say first that bookbook is great!

but not without some problems... If I try to use bookbook on notebooks where I have used the relatively new cell tags mechanism, as in

python3 -m bookbook.latex --TagRemovePreprocessor.remove_input_tags='{"hide-code"}' --template ./art.tplx --TagRemovePreprocessor.remove_cell_tags='{"hide-cell"}'

everything breaks and I'm told

usage: latex.py [-h] [--output-file OUTPUT_FILE] [--pdf] [--template TEMPLATE]
                [source_dir]
latex.py: error: unrecognized arguments: --TagRemovePreprocessor.remove_input_tags={"hide-code"} --TagRemovePreprocessor.remove_cell_tags={"hide-cell"}

Why don't you simply pass to nbconvert the options not understood by bookbook? I have made the example of the tags preprocessor, but there are many other aspects of nbconvert that are configurable via command line options.

Thank you ፨ gb

flit

Hi,

Previous install commands run well, but when I try to run the sentence:
flit install --symlink

it throws this error:

Symlinking .\bookbook -> c:\anaconda3\Lib\site-packages\bookbook I-flit.install
Traceback (most recent call last):
File "c:\anaconda3\lib\runpy.py", line 193, in run_module_as_main
"main", mod_spec)
File "c:\anaconda3\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\Anaconda3\Scripts\flit.exe_main
.py", line 7, in
File "c:\anaconda3\lib\site-packages\flit_init
.py", line 188, in main
pth=args.pth_file).install()
File "c:\anaconda3\lib\site-packages\flit\install.py", line 407, in install
self.install_directly()
File "c:\anaconda3\lib\site-packages\flit\install.py", line 305, in install_directly
os.symlink(osp.abspath(self.module.path), dst)
OSError: symbolic link privilege not held

Any advice? is it windows10 related? anaconda3-related? (flit version 2.3.0)
Thanks in advance, JL

References not converted to latex when template extends report.tplx

If one uses bookbook.latex with a template that starts with

((*- extends 'report.tplx' -*))

then cross references are not properly converted to latex; they are simply formatted as hyperlinks with \href{}. This seems to be caused by the following section of nbconvert/templates/latex/report.tplx:

((* block markdowncell scoped *))
((( cell.source | citation2latex | strip_files_prefix | convert_pandoc('markdown', 'latex', extra_args=["--chapters"]) )))
((* endblock markdowncell *))

--template is broken since nbconvert 6.0

Starting from nbconvert 6.0, trying to use a template.tex file such as:

((* extends 'style_python.tex.j2' *))

((* block docclass *))
\documentclass[11pt]{book}
((* endblock docclass *))

by invoking the --template template.tex option gives:

python -c "import nbconvert;print(nbconvert.__version__)"
6.0.7
python -m bookbook.latex --pdf --template template.tex 
INFO:__main__:Combined 2 files
INFO:__main__:Converting to pdf
Traceback (most recent call last):
  File "/Users/boileau/opt/anaconda3/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Users/boileau/opt/anaconda3/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/boileau/Documents/Git/Python/bookbook/.venv/lib/python3.8/site-packages/bookbook/latex.py", line 163, in <module>
    main()
  File "/Users/boileau/Documents/Git/Python/bookbook/.venv/lib/python3.8/site-packages/bookbook/latex.py", line 160, in main
    combine_and_convert(args.source_dir, args.output_file, args.pdf, args.template)
  File "/Users/boileau/Documents/Git/Python/bookbook/.venv/lib/python3.8/site-packages/bookbook/latex.py", line 145, in combine_and_convert
    export(combined_nb, output_file, pdf=pdf, template_file=template_file)
  File "/Users/boileau/Documents/Git/Python/bookbook/.venv/lib/python3.8/site-packages/bookbook/latex.py", line 139, in export
    output, resources = exporter.from_notebook_node(combined_nb, resources)
  File "/Users/boileau/Documents/Git/Python/bookbook/.venv/lib/python3.8/site-packages/nbconvert/exporters/pdf.py", line 168, in from_notebook_node
    latex, resources = super().from_notebook_node(
  File "/Users/boileau/Documents/Git/Python/bookbook/.venv/lib/python3.8/site-packages/nbconvert/exporters/latex.py", line 77, in from_notebook_node
    return super().from_notebook_node(nb, resources, **kw)
  File "/Users/boileau/Documents/Git/Python/bookbook/.venv/lib/python3.8/site-packages/nbconvert/exporters/templateexporter.py", line 384, in from_notebook_node
    output = self.template.render(nb=nb_copy, resources=resources)
  File "/Users/boileau/Documents/Git/Python/bookbook/.venv/lib/python3.8/site-packages/nbconvert/exporters/templateexporter.py", line 148, in template
    self._template_cached = self._load_template()
  File "/Users/boileau/Documents/Git/Python/bookbook/.venv/lib/python3.8/site-packages/nbconvert/exporters/templateexporter.py", line 355, in _load_template
    return self.environment.get_template(template_file)
  File "/Users/boileau/Documents/Git/Python/bookbook/.venv/lib/python3.8/site-packages/jinja2/environment.py", line 883, in get_template
    return self._load_template(name, self.make_globals(globals))
  File "/Users/boileau/Documents/Git/Python/bookbook/.venv/lib/python3.8/site-packages/jinja2/environment.py", line 857, in _load_template
    template = self.loader.load(self, name, globals)
  File "/Users/boileau/Documents/Git/Python/bookbook/.venv/lib/python3.8/site-packages/jinja2/loaders.py", line 429, in load
    raise TemplateNotFound(name)
jinja2.exceptions.TemplateNotFound: template.tex

While, with the equivalent template file for nbconvert < 6.0

((* extends 'style_python.tplx' *))

((* block docclass *))
\documentclass[11pt]{book}
((* endblock docclass *))

it is ok with nbconvert < 6.0 :

python -c "import nbconvert;print(nbconvert.__version__)"
5.6.1
python -m bookbook.latex --pdf --template template.tex 
INFO:__main__:Combined 2 files
INFO:__main__:Converting to pdf
/Users/boileau/Documents/Git/Python/bookbook/.venv/lib/python3.8/site-packages/nbconvert/filters/highlight.py:138: UserWarning: IPython3 lexer unavailable, falling back on Python 3
  warn("IPython3 lexer unavailable, falling back on Python 3")
INFO:traitlets:Writing 16279 bytes to ./notebook.tex
INFO:traitlets:Building PDF
INFO:traitlets:Running xelatex 3 times: ['xelatex', './notebook.tex', '-quiet']
INFO:traitlets:Running bibtex 1 time: ['bibtex', './notebook']
WARNING:traitlets:bibtex had problems, most likely because there were no citations
INFO:traitlets:PDF successfully created
INFO:traitlets:Writing 20361 bytes to ./combined.pdf

I will propose a PR soon!

Build an index

Thanks for this nice tool!
Any idea how we could build a word index the same way makeidx does?

Throwing error

Hi

I tried using your bookbook, but I am getting error as below. could not understand what is going wrong. I had simplified my file names as required.

image

Any idea why this appear? Below is my current directory from where I run the command.
image

Handling of Latex macros

One can use \newcommand in a Jupyter notebook to define a macro, by putting it inside a math environment; e.g.

$$\newcommand{\bigO}{{\mathcal O}}$$

However, the conversion to Latex naturally then puts the \newcommand inside a math environment, which makes it available only within that set of math delimiters.

Furthermore, in notebooks one must define a given macro again in each notebook that uses it. But when these notebooks are combined into a book, the resulting .tex file should only define each macro once.

It would be great if bookbook collected all the macros from the set of notebooks, and placed them together in the preamble of the .tex file (without math delimiters around them, of course).

Possible syntax for alternate label specification from multimarkdown

Alternatively, you can include an optional label of your choosing to help disambiguate cases where multiple headers have the same title:

Overview [MultiMarkdownOverview]

from Multimarkdown syntax guide

I'm pretty sure the trailing hashes are optional. it's just after any heading, you could place an alternate label in the brackets.

This will probably be a better solution for long file section headers with spaces than having to put their entirety down and converting to -.

Not sure how that should work for the top level reference and file names though…I'm thinking if we can figure out a nice way to do even have alternate labels for those it would be useful in order to avoid needing to have someone put a complete path when they're in different directories.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.