mpastell/pweave

Pweave is a scientific report generator and a literate programming tool for Python. It can capture the results and plots from data analysis and works well with numpy, scipy and matplotlib.

About Pweave

Pweave is a scientific report generator and a literate programming tool for Python. Pweave can capture the results and plots from data analysis and works well with NumPy, SciPy and matplotlib. It is able to run python code from source document and include the results and capture matplotlib plots in the output.

Pweave is good for creating reports, tutorials, presentations etc. with embedded python code It can also be used to make websites together with e.g. Sphinx or rest2web.

Features:

• Python 3.4, 3.5 and 3.6 compatibility
• Code is run using jupyter_client giving the possibility to run code using any installed kernel (including python2) via --kernel argument. Some chunk options only work for Python.
• Support for IPython magics and rich output.

• Produce reST, Sphinx, Latex, HTML and markdown (pandoc and leanpub) output from several input formats.

  • Noweb uses noweb syntax for separating code from documentation.
  • Markdown. Run code from markdown code blocks.
  • Script. Python script with special markup in comments.

• Execute python code in the chunks and capture input and output to a report.
• Use hidden code chunks, i.e. code is executed, but not printed in the output file.
• Capture matplotlib graphics.
• Evaluate inline code in documentation chunks marked using <% %> and <%= %>.
• Cache all code and results from previous runs for fast report generation when you are only working with documentation. Inline code will be hidden in documentation mode.
• Supports reST, LaTeX, HTML or markdown for document chunks
• Publish reports from Python scripts. Similar to R markdown.
• Run from command line or interpreter.

Install

From PyPi:

pip install --upgrade Pweave

with conda:

conda install -c conda-forge pweave

or download the source and run:

python setup.py install

Pweave documentation can be found from the website http://mpastell.com/pweave

Release Notes

See CHANGELOG.txt for changes in each release.

License information

See the file "LICENSE" for information on the history of this software, terms & conditions for usage, and a DISCLAIMER OF ALL WARRANTIES.

pweave's Issues

Bug: setting `-g` or `--figure-format` causes Pweave to fail

The title says it. I try to run

Pweave -f pandoc -g pdf filename.Pnw

...because I want Pandoc to convert the document to .tex later on.
It fails with the error message:

Traceback (most recent call last):
  File "/home/trive/anaconda/bin/Pweave", line 9, in <module>
    load_entry_point('Pweave==0.21.2', 'console_scripts', 'Pweave')()
  File "/home/trive/anaconda/lib/python2.7/site-packages/pweave/scripts.py", line 52, in weave
    cachedir = options.cachedir, figformat = figfmt, listformats = options.listformats)
  File "/home/trive/anaconda/lib/python2.7/site-packages/pweave/pweb.py", line 55, in pweave
    doc.formatdict['figfmt'] = figformat
AttributeError: 'Pweb' object has no attribute 'formatdict'

Bokeh, Altair plotting

Hi there,

Only discovered this project today. I was wondering, whether reports can be generated with Bokeh/altair doing the plotting as opposed to Matplotlib?

Both Altair and Bokeh render web-based graphics.

RuntimeError: Kernel died before replying to kernel_info

While trying to find minimal failing example for #60 I discovered another issue (2f888e8 Python 3.5)

START: run 660/1000
E
======================================================================
ERROR: testTerminalEmulation (__main__.ReproducibleTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/test_nondeterministicFail.py", line 29, in testMethod
    pweave.weave(infile, doctype=doctype, **kwargs)
  File "/home/jkowalski/Pweave/pweave/__init__.py", line 60, in weave
    doc.weave()
  File "/home/jkowalski/Pweave/pweave/pweb.py", line 181, in weave
    self.run()
  File "/home/jkowalski/Pweave/pweave/pweb.py", line 118, in run
    self.wd
  File "/home/jkowalski/Pweave/pweave/processors/jupyter.py", line 151, in __init__
    super(IPythonProcessor, self).__init__(*args)
  File "/home/jkowalski/Pweave/pweave/processors/jupyter.py", line 28, in __init__
    cwd = path)
  File "/home/jkowalski/anaconda/envs/Pweave35/lib/python3.5/site-packages/jupyter_client/manager.py", line 452, in start_new_kernel
    kc.wait_for_ready(timeout=startup_timeout)
  File "/home/jkowalski/anaconda/envs/Pweave35/lib/python3.5/site-packages/jupyter_client/blocking/client.py", line 115, in wait_for_ready
    raise RuntimeError('Kernel died before replying to kernel_info')
RuntimeError: Kernel died before replying to kernel_info

Inline strings from python code

Any way for me to create a code chunk that will evaluate into Tex/Md? I'd love to write some inline statistics and interpretations.

Thanks!

Python 2.* Compatibility

Looks like some of the current syntax requires Python 3, i.e.

(pweave-env)$ python setup.py develop
Traceback (most recent call last):
  File "setup.py", line 4, in <module>
    import pweave
  File "/home/bwillar0/projects/python/Pweave/pweave/__init__.py", line 7, in <module>
    from .pweb import *
  File "/home/bwillar0/projects/python/Pweave/pweave/pweb.py", line 26
    def __init__(self, source, doctype = None, *, informat = None, kernel = "python3",
                                                ^
SyntaxError: invalid syntax

Is there no longer Python 2.* compatibility?

Pweave-convert brakes f_size option

Change convert code to use raw string instead of converting back from dictionary

Add proper handling of tikz/pgf files

As of version 1.2, matplotlib has the ability to write pgf files for rendering in LaTeX documents (see http://matplotlib.org/users/whats_new.html#pgf-tikz-backend). This format is useful for scientific documents where the text and measurement used in the plot need to match the text and measurements in the LaTeX document. However, they cannot be included using the standard LaTeX \includegraphics command, and require the \input command instead.

I propose adding a simple check in PwebTexFormatter that checks if figfmt is "pgf", then inserts the appropriate \input statement rather than \includegraphics.

Last line of code block not displayed when using texminted and wrap=False

An example:

\documentclass{article}
\usepackage{palatino}
\usepackage{minted}


\begin{document}

\section{wrap=True block}

<<wrap=True>>=
first_line = 1
second_line = 2
last_line = 'last'
@

\section{wrap=False block}

<<wrap=False>>=
first_line = 1
second_line = 2
last_line = 'last'
@

\end{document}

Compiling with -f texminted, the result is:

\documentclass{article}
\usepackage{palatino}
\usepackage{minted}


\begin{document}

\section{wrap=True block}


\begin{minted}[mathescape, fontsize=\small, xleftmargin=0.5em]{python}
first_line = 1
second_line = 2
last_line = 'last'
\end{minted}


\section{wrap=False block}


\begin{minted}[mathescape, fontsize=\small, xleftmargin=0.5em]{python}
first_line = 1
second_line = 2
last_line = 'last'\end{minted}


\end{document}

which produces the following output:

The last \end{minted} just needs bumped to the next line.

I'm using Pweave 0.23.

Pweave 0.3 works in Python 3.6.1!

Probably should be added to the release notes. I just installed pweave from source in a new conda environment with python 3.6.1 and can confirm it works.

The current version in conda, v0.25 will raise UnsatisfiableError in python 3.6*

UnsatisfiableError: The following specifications were found to be in conflict:
  - pweave -> python 3.3* -> openssl 1.0.1*
  - pweave -> python 3.3* -> xz 5.0.5
  - python 3.6*

Fix title block in pandoc2latex formatter

The formatter crashes if there is no titleblock.

Better syntax highlighting for shell chunks

Pweave can now also execute shell commands from code chunks using engine="shell" option, but chunks still use Python syntax highlighting.

fignos compatibiliy

I have started to work with pweave and found it great. I am know trying to use fignos within my mdw file. It works fine except one aspect. fignos uses the following convention fig:id to reference a figure. When I use such reference as name/label in my chunk the figure file is not produced due to the colon in the file name, e.g.:

Exemple (simplified!):

<< name="fig:id", echo=False, caption="caption" >>=
make_figure()
@

produce:

![caption](C:\...\figures/rapport_fig:id_1.png){#fig:id}

Could it possible to treat the file name and the reference differently/separatly. I see two options:

1. Clean the string name from special character such as colon
2. Make the reference customizable, by default it use the name/label attribute, or it can be user defined for such a special case (my favorite)

or maybe I've missed something!

Thanks.

bug: pypublish incorrectly handles non ascii characters

If you use any non-standard ascii character in the text, they are translated with the wrong set of characters. This is more likely an encoding/decoding error.

try any accented letter, like òàùèì, the results on my computer are reliably incorrect (using python3.4, Pweave 0.23)

Add ability to run files from urls

Term mode doesn't generate a block when there's not output

# Markdown test
 
<<term=True>>=
some_variable = 1
@

After executing pweave test.md

# Markdown test
 

Expected:

# Markdown test
 
'''python
some_variable = 1
'''

Figure rendering bug with sphinx output to html

I am using matplotlib code in my .Pnw script to create PDF files, which are also automatically being saved as PNG in the figures/ folder of my project (so both formats are created there). I can get PNG figures to be loaded into my html doc from my .Pnw script if I use the '-f rst' weave option, but then I can't use python syntax highlighting. So, I've been outputting using the '-f sphinx' option and, using a stylesheet from pygments, using rst2html. This gets the syntax highlighting but the figures are missing.

Despite a comment in your code saying that Sphinx automatically works out figure file extensions (where you leave figfmt="." in class PwebSphinxFormatter), the final rST doc still has the . extension on the filenames. I see that class states that acceptable saved formats are PNG and PDF, but I force figmt to PDF it still doesn't embed the figure. However, forcing to PNG works. I'm using docutils 0.11.

New maintainer

@abukaj You can add yourself as a maintainer to README now if you wish. You should be able to push directly to mpastell/Pweave now.

EDIT: I mean as a second maintainer. I'm not abandoning the project.

Add IPython example to docs

Pweave 0.22 supports IPython shell including magics, the feature needs a proper example.

Pypublish embed plots in html

See: https://groups.google.com/forum/#!topic/pweave/lIPtywbY4t4

Add chunk "main" option to code chunk for tangling

As requested in: https://groups.google.com/forum/#!topic/pweave/8GLRbgHFJMY

Feature request: interpret ansi output correctly.

At the moment, converting markdown -> markdown, tracebacks etc. cause many uninterpreted unicode characters to appear in the output.

pygments.sty

Not sure if this is an issue, but when I install Pweave using pip there is no "pygments.sty" file. So when I attempt to run pdflatex after generating the .tex file I get the following error:

! LaTeX Error: File `pygments.sty' not found.

I solved this by cloning the repo from github and pointing it to the "pygments.sty" file in the examples directory. I am running RHEL6 x86 and I have texlive installed. Do other distributions come with the "pygements.sty" file?

Add manpages for the console scripts

It is a recommended practice in the Linux community to provide corresponding manpages to publicly installed command-line scripts. Pweave installs a few console scripts, which lacks a manpage currently.

One solution could be as simple as providing a corresponding $SCRIPT.1.md or $SCRIPT.1.rst with a plain text formatted manpage, and let Linux distributions run pandoc to convert it to a manpage and install it wherever appropriate.

Otherwise, assuming you have already got some Sphinx documentation written (#53), is to tweak Sphinx to generate these manpages for you.

effect of `shell` argument in Pweb()

using v0.25

I haven't seen documentation for this, but it seems like the python shell remains alive between calls to Pweb().weave. At least, I can define a variable in one document's code chunk and use it in a second when creating multiple Pweb documents. This seems to break when with ipython as the shell. Can I get the same behavior with ipython?

I've noticed that if I do doc.weave(shell='ipython'), then ipython does get invoked but variables do not persist. If I do doc = Pweb(shell='ipython') followed by doc.weave(), then I get python. I'm hoping this is related?

Here is the code that allows multiple markdown files in docs/_slides_pmd/ to reference the same variable using the default python shell. Changing the last line to doc.weave(shell='ipython') gives me ipython, but gives undefined variables.

from pweave import Pweb, PwebPandocFormatter, rcParams
import yaml

rcParams['chunk']['defaultoptions'].update({
    'term': True,
    })

class Formatter(PwebPandocFormatter):

    def __init__(self, *args, **kwargs):
        super(Formatter, self).__init__(*args, **kwargs)
        self.formatdict.update({
            'termstart': '~~~%s',
            'termend': '~~~\n{:.term}\n\n',
            })
        return
    
    def make_figure_string(self, *args, **kwargs):
        f_str = super(Formatter, self).make_figure_string(*args, **kwargs)
        return f_str.replace('..', '{{ site.baseurl }}')
        

with open('docs/_config.yml') as f:
    config = yaml.load(f)
    
for fname in config['slide_sorter']:
    doc = Pweb(
        file='docs/_slides_pmd/{}.pmd'.format(fname),
        output='docs/_slides/{}.md'.format(fname),
        figdir='../images',
        )
    doc.setreader('markdown')
    doc.setformat(Formatter=Formatter)
    doc.weave()

Support for executing scripts using Spyder cells

See: https://pythonhosted.org/spyder/editor.html#how-to-define-a-code-cell

This would only require small changes to current script reader.

Related to:
spyder-ide/spyder#2548

Absolute paths with figures cause error

Reproduce (where example.mdw has a code block that would cause a matplotlib call):

# ok
Pweave example.mdw
# not ok
Pweave /path/to/example.mdw

IOError: [Errno 2] No such file or directory: 'figures//tmp/simple_figure2_1.png'

Missing macro on LaTeX

Just installed it using pip3 install pweave. I downloaded FIR_design_verb.texw and ran:

pweave FIR_design_verb.texw
ptangle FIR_design_verb.texw

No errors are output, but the result .tex file is missing a macro definition: \PY. Actually, many macros starting with \PY.

Evaluate and Cache new Code Chunks in Documentation Mode

If I add a new chunk after the previous chunks are cached, I get the following exception:

Pweave -f texminted -c -d missing_chunk_test.texw
Traceback (most recent call last):
  File "/usr/local/bin/Pweave", line 9, in <module>
    load_entry_point('Pweave==0.23', 'console_scripts', 'Pweave')()
  File "/usr/local/lib/python2.7/dist-packages/Pweave-0.23-py2.7.egg/pweave/scripts.py", line 53, in weave
    pweave.weave(infile, **opts_dict)
  File "/usr/local/lib/python2.7/dist-packages/Pweave-0.23-py2.7.egg/pweave/__init__.py", line 69, in weave
    doc.weave(shell)
  File "/usr/local/lib/python2.7/dist-packages/Pweave-0.23-py2.7.egg/pweave/pweb.py", line 141, in weave
    self.run(shell)
  File "/usr/local/lib/python2.7/dist-packages/Pweave-0.23-py2.7.egg/pweave/pweb.py", line 109, in run
    runner.run()
  File "/usr/local/lib/python2.7/dist-packages/Pweave-0.23-py2.7.egg/pweave/processors.py", line 53, in run
    success = self._getoldresults()
  File "/usr/local/lib/python2.7/dist-packages/Pweave-0.23-py2.7.egg/pweave/processors.py", line 260, in _getoldresults
    executed.append(self._oldresults[i].copy())
IndexError: list index out of range
Makefile:14: recipe for target 'missing_chunk_test.tex' failed
make: *** [missing_chunk_test.tex] Error 1

I was assuming that the caching mechanism would notice the missing chunk, evaluate and cache it, then proceed. Is that the intended functionality?

Return full stack trace if code chunk raises exception

We're using pweave as part of our Python data processing pipeline, and it's great!

Early data work and charting can be fiddly, though, and it would be helpful if we could see the full stack trace if a code chunk raises an exception. Is this currently possible through a chunk option or flag (we're using the CLI to weave an .mdw file to .tex)? If not, you would consider implementing? (Happy to discuss and help if I can.)

Thank you!

Non-evaluated Plot Chunks Issues

I'm using the code from ee902f3, compiling/weaving with Pweave -f texminted -c -d ....
If I have a chunk with settings like

<< not_evaled_plot, fig=True, caption='A plot caption', f_pos='h!', evaluate=False >>=
...

then the output tex will look like

\begin{minted}[mathescape, fontsize=\small, xleftmargin=0.5em]{%s}
...

which implies that a value isn't being substituted ("python" perhaps?) and the TeX won't compile.

sqlite3.OperationalError: database is locked

While testing for #61 I encountered a following crash:

Traceback (most recent call last):
  File "/home/jkowalski/anaconda3/envs/Pweave35/lib/python3.5/site-packages/IPython/core/history.py", line 87, in catch_corrupt_db
    return f(self, *a, **kw)
  File "/home/jkowalski/anaconda3/envs/Pweave35/lib/python3.5/site-packages/IPython/core/history.py", line 257, in init_db
    end timestamp, num_cmds integer, remark text)""")
sqlite3.OperationalError: database is locked

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jkowalski/anaconda3/envs/Pweave35/lib/python3.5/site-packages/IPython/core/history.py", line 87, in catch_corrupt_db
    return f(self, *a, **kw)
  File "/home/jkowalski/anaconda3/envs/Pweave35/lib/python3.5/site-packages/IPython/core/history.py", line 257, in init_db
    end timestamp, num_cmds integer, remark text)""")
sqlite3.OperationalError: disk I/O error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jkowalski/anaconda3/envs/Pweave35/bin/pweave", line 11, in <module>
    load_entry_point('Pweave==0.30a1', 'console_scripts', 'pweave')()
  File "/home/jkowalski/Pweave/pweave/scripts.py", line 53, in weave
    pweave.weave(infile, **opts_dict)
  File "/home/jkowalski/Pweave/pweave/__init__.py", line 60, in weave
    doc.weave()
  File "/home/jkowalski/Pweave/pweave/pweb.py", line 192, in weave
    self.run()
  File "/home/jkowalski/Pweave/pweave/pweb.py", line 127, in run
    self.wd
  File "/home/jkowalski/Pweave/pweave/processors/jupyter.py", line 168, in __init__
    super(IPythonProcessor, self).__init__(*args, embed_kernel=embed)
  File "/home/jkowalski/Pweave/pweave/processors/jupyter.py", line 34, in __init__
    km.start_kernel(cwd=path, stderr=open(os.devnull, 'w'))
  File "/home/jkowalski/anaconda3/envs/Pweave35/lib/python3.5/site-packages/ipykernel/inprocess/manager.py", line 46, in start_kernel
    self.kernel = InProcessKernel(parent=self, session=self.session)
  File "/home/jkowalski/anaconda3/envs/Pweave35/lib/python3.5/site-packages/ipykernel/inprocess/ipkernel.py", line 72, in __init__
    super(InProcessKernel, self).__init__(**traits)
  File "/home/jkowalski/anaconda3/envs/Pweave35/lib/python3.5/site-packages/ipykernel/ipkernel.py", line 46, in __init__
    kernel      = self,
  File "/home/jkowalski/anaconda3/envs/Pweave35/lib/python3.5/site-packages/traitlets/config/configurable.py", line 412, in instance
    inst = cls(*args, **kwargs)
  File "/home/jkowalski/anaconda3/envs/Pweave35/lib/python3.5/site-packages/IPython/core/interactiveshell.py", line 460, in __init__
    self.init_history()
  File "/home/jkowalski/anaconda3/envs/Pweave35/lib/python3.5/site-packages/IPython/core/interactiveshell.py", line 1564, in init_history
    self.history_manager = HistoryManager(shell=self, parent=self)
  File "/home/jkowalski/anaconda3/envs/Pweave35/lib/python3.5/site-packages/IPython/core/history.py", line 537, in __init__
    **traits)
  File "/home/jkowalski/anaconda3/envs/Pweave35/lib/python3.5/site-packages/IPython/core/history.py", line 229, in __init__
    self.init_db()
  File "<decorator-gen-16>", line 2, in init_db
  File "/home/jkowalski/anaconda3/envs/Pweave35/lib/python3.5/site-packages/IPython/core/history.py", line 115, in catch_corrupt_db
    self.init_db()
  File "<decorator-gen-16>", line 2, in init_db
  File "/home/jkowalski/anaconda3/envs/Pweave35/lib/python3.5/site-packages/IPython/core/history.py", line 113, in catch_corrupt_db
    os.rename(self.hist_file, newpath)
FileNotFoundError: [Errno 2] No such file or directory: '/home/jkowalski/.ipython/profile_default/history.sqlite' -> '/home/jkowalski/.ipython/profile_default/history-corrupt.sqlite'

I guess it was due to some kind of race condition since I have been testing #61 paralelly in several shells.

I am opening the issue to preserve information about the problem in case it hit again.

Testing fails when building the module for Debian Stretch

I am using the latest stable version on PyPI (version 0.25) with the packaged dependencies in Debian Stretch. The package fails to run the tests with the following output:

cd /<<PKGBUILDDIR>>/.pybuild/pythonX.Y_3.5/build; python3.5 -m nose tests
.............Warning, only the first figure gets a caption
Warning, only the first figure gets a caption
Warning, only the first figure gets a caption
Warning, only the first figure gets a caption
..S............./usr/lib/python3/dist-packages/IPython/nbformat.py:13: ShimWarning: The `IPython.nbformat` package has been deprecated. You should import from nbformat instead.
  "You should import from nbformat instead.", ShimWarning)
..DOCUMENTATION MODE ERROR:
Can't find stored results, running the code and caching results for the next documentation mode run
..FFFF.
======================================================================
FAIL: Test external python shell
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/<<PKGBUILDDIR>>/.pybuild/pythonX.Y_3.5/build/tests/test_shell.py", line 19, in test_external_shell
    assertSameContent("tests/processors/processor_test.txt", "tests/processors/processor_external_ref.md")
  File "/<<PKGBUILDDIR>>/.pybuild/pythonX.Y_3.5/build/tests/test_shell.py", line 30, in assertSameContent
    assert (out.read() == ref.read())
AssertionError: 
-------------------- >> begin captured stdout << ---------------------
Processing chunk 1 named None from line 5
Processing chunk 2 named None from line 12
Processing chunk 3 named None from line 16
Processing chunk 4 named None from line 20
Processing chunk 5 named None from line 24
Processing chunk 6 named None from line 29
Processing chunk 7 named None from line 33
Processing chunk 8 named None from line 37
Processing chunk 9 named None from line 41
Processing chunk 10 named None from line 46
Processing chunk 11 named None from line 50
Pweaved tests/processors/processor_test.mdw to tests/processors/processor_test.txt

--------------------- >> end captured stdout << ----------------------

======================================================================
FAIL: Test ipython python shell
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/<<PKGBUILDDIR>>/.pybuild/pythonX.Y_3.5/build/tests/test_shell.py", line 24, in test_ipython_shell
    assertSameContent("tests/processors/ipy_processor_test.txt", "tests/processors/ipy_processor_ref.md")
  File "/<<PKGBUILDDIR>>/.pybuild/pythonX.Y_3.5/build/tests/test_shell.py", line 30, in assertSameContent
    assert (out.read() == ref.read())
AssertionError: 
-------------------- >> begin captured stdout << ---------------------
Processing chunk 1 named None from line 5
Processing chunk 2 named None from line 9
Processing chunk 3 named None from line 13
Processing chunk 4 named None from line 17
Processing chunk 5 named None from line 21
Processing chunk 6 named None from line 26
Processing chunk 7 named None from line 30
Processing chunk 8 named None from line 34
Processing chunk 9 named None from line 38
Processing chunk 10 named None from line 43
Processing chunk 11 named None from line 47
Pweaved tests/processors/ipy_processor_test.mdw to tests/processors/ipy_processor_test.txt

--------------------- >> end captured stdout << ----------------------

======================================================================
FAIL: Test pypublish
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/<<PKGBUILDDIR>>/.pybuild/pythonX.Y_3.5/build/tests/test_shell.py", line 38, in test_publish
    assert (test[:len(test) - 400] == ref[:len(test) - 400])
AssertionError: 
-------------------- >> begin captured stdout << ---------------------
Processing chunk 1 named None from line 7
Processing chunk 2 named None from line 59
Processing chunk 3 named None from line 66
Published tests/publish/publish_test.txt to tests/publish/publish_test.html

--------------------- >> end captured stdout << ----------------------
-------------------- >> begin captured logging << --------------------
MARKDOWN: DEBUG: Successfully loaded extension "pweave.markdownmath.MathExtension".
MARKDOWN: DEBUG: Successfully loaded extension "pweave.markdownmath.MathExtension".
MARKDOWN: DEBUG: Successfully loaded extension "pweave.markdownmath.MathExtension".
MARKDOWN: DEBUG: Successfully loaded extension "pweave.markdownmath.MathExtension".
--------------------- >> end captured logging << ---------------------

======================================================================
FAIL: Test pypublish cell format
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/<<PKGBUILDDIR>>/.pybuild/pythonX.Y_3.5/build/tests/test_shell.py", line 46, in test_publish_cell
    assert (test[:len(test) - 400] == ref[:len(test) - 400])
AssertionError: 
-------------------- >> begin captured stdout << ---------------------
Processing chunk 1 named None from line 7
Processing chunk 2 named None from line 59
Processing chunk 3 named None from line 66
Published tests/publish/publish_test_cell.txt to tests/publish/publish_test_cell.html

--------------------- >> end captured stdout << ----------------------
-------------------- >> begin captured logging << --------------------
MARKDOWN: DEBUG: Successfully loaded extension "pweave.markdownmath.MathExtension".
MARKDOWN: DEBUG: Successfully loaded extension "pweave.markdownmath.MathExtension".
MARKDOWN: DEBUG: Successfully loaded extension "pweave.markdownmath.MathExtension".
MARKDOWN: DEBUG: Successfully loaded extension "pweave.markdownmath.MathExtension".
--------------------- >> end captured logging << ---------------------

----------------------------------------------------------------------
Ran 38 tests in 23.741s

FAILED (SKIP=1, failures=4)

Any idea what is happening here?

Add source of documentation

Please consider adding the source of the documentation (generated with Sphinx + rtd-theme I suppose) to the project's tree and shipping them in future PyPI release tarballs. Linux distributions primarily source themselves from PyPI and it would be desirable to enable builds of binary packages for both the Python modules / scripts and corresponding documentation.

Thanks,
Ghis

Add support for new pandoc link attributes

See http://pandoc.org/README.html#images

Nondeterministic failure of `test_pweave.py`

Regression test of weaving tests/weave/tex/term_test.texw sometimes fails and sometimes passes.

Examplary output:

\begin{verbatim}
for i in range(3):
    print(i)
\end{verbatim}
\begin{verbatim}
0
\end{verbatim}
\begin{verbatim}
1
2
\end{verbatim}

instead of simpler

\begin{verbatim}
for i in range(3):
    print(i)
\end{verbatim}
\begin{verbatim}
0
1
2
\end{verbatim}

Feature request: label prompts

Currently, there is no way to generate IPython-like prompts, even when IPython is used as the interpreter.

Unable to hide the code

I am using pypublish -f pdf FIR_design.py to make PDF. I would like to hide the code and I need only Document.

I tried the following, neither did work

```{python, echo=False}
n = 1001
a = signal.firwin(n, cutoff = [0.2, 0.5], window = 'blackmanharris', pass_zero = False)
mfreqz(a)
show()

and

<<echo=False>>=
n = 1001
a = signal.firwin(n, cutoff = [0.2, 0.5], window = 'blackmanharris', pass_zero = False)
mfreqz(a)
show()
@

Tangle does not work for markdown fenced code blocks

Fenced code blocks in markdown work with pweave, but ptangle does not recognize them as code blocks. For example, the following input produces an empty file:

```{python}
print('Hello')
```

Better line wrapping

Make line wrapping use textwrap module and break on whitespace.

• Consider different wrapping for source and console.
• Option to set line width?

Add tests to release tarball

Same background as #53. Tests are desirable for QA / CI purposes on the packaging side.

BUG: os.chdir() inside a snippet affects Pweave

Hi there!

If a code-snippet in a Pweave document contains a os.chdir() line, then this will affect Pweave as well. Meaning all files (output file + figures) will also land in the new directory. This is probably not intended.

Cheers

Thomas

FIR_design.py example bug: undefined control sequence \textquotesingle

This was sent as an email but no response. Maybe this is a better place to put this.

I just did a fresh install of pweave on a new win 7 machine. I'm getting an error in handling single quotes using your example. If I remove line 10 and 21 from the FIR_design.py file the error does not occur. It seems you may not be including the textcomp package. Can you confirm this bug and fix it?

Details below.

Possible fix?:
http://tex.stackexchange.com/questions/202166/undefined-control-sequence-textquotesingle

I'm running your example file with "pypublish -f pdf FIR_design.py"

I get the following error in the log file:

! Undefined control sequence.
#\textquotesingle
{}
l.123 ...ting with \texttt{\#\textquotesingle{}} '' and ? ! Emergency stop. <argument> \#\textquotesingle {} l.123 ...ting with\texttt{#\textquotesingle{}}
'' and
End of file on the terminal!

Here is how much of TeX's memory you used:
6554 strings out of 493921
95649 string characters out of 3144873
183050 words of memory out of 3000000
9798 multiletter control sequences out of 15000+200000
9654 words of font info for 33 fonts, out of 3000000 for 9000
841 hyphenation exceptions out of 8191
29i,6n,31p,245b,281s stack positions out of 5000i,500n,10000p,200000b,50000s
! ==> Fatal error occurred, no output PDF file produced!

Attached is a list of the versions of software I have installed.

TortoiseSVN
1.8.11 64bit
Chrome
Cygwin
Python
2.7.10 and add to path
pip install matplotlib 1.4.3
pip install pweave
Pip install scipy 0.15.1
From http://www.lfd.uci.edu/~gohlke/pythonlibs/#scipy
Scipy-0.15.1-cp27-none-win32.whl
pip install numpy 1.9.2 from:
numpy‑1.9.2+mkl‑cp27‑none‑win32.whl
Pip install pygments for pweave
Pip install markdown for pweave
Pysvn
Visual C for Python:
http://aka.ms/vcpython27
http://www.microsoft.com/en-us/download/confirmation.aspx?id=44266

MiKTeX
2.9.5105 Win32 (for pweave) from miktex.org/download
Set to Letter size default

Pandoc
To support pweave
1.15.0.6
https://github.com/jgm/pandoc/releases/tag/1.15.0.6
Pandoc.org

Move parsing of chunk options to processors

Chunk options are now parsed with readers, moving parsing to processors allows to use code in the document to manipulate options.

using multiple input files

Hi Matti

thanks for Pweave.

Is there an equivalent of \SweaveInput{} for LaTeX input (eg \PweaveInput{}), in order to manage multiple input file?

thanks,
Luca

BUG: newline between subsequent figures in Latex

Hi there!

If a single chunk in a Pweave document produces multiple figures while producing Latex, this will result in an output like:

\includegraphics[width= \textwidth]{figures/....}
\includegraphics[width= \textwidth]{figures/....}

The problem here is that since there are no newlines between the \include's, Latex (at least: pdflatex) will interpret this as "put them all on the same line". Which in turn results in all of the images landing on the same page. If the images are larger than the page, the later images will be cropped / not displayed at all.

Having an empty between the \includegraphics instructions solves this problem.

pip install Pweave hits a configparser.DuplicateOptionError

I'm not sure what has changed, but pip is now failing to install the Pweave scripts. I've included the traceback below, but apparently configparser lower-cases keys (I found the error mentioned at http://www.gilesorr.com/blog/python-3-configparser.html), and so Ptangle and ptangle look the same to it and they collide. Running "python setup.py install" works just fine, so presumably the bug is in pip, not in Pweave. That said, pip is failing to correctly install Pweave....

$ pip install Pweave
Collecting Pweave
Installing collected packages: Pweave
Exception:
Traceback (most recent call last):
  File "/home/grant/Envs/py34test/lib/python3.4/site-packages/pip/basecommand.py", line 211, in main
    status = self.run(options, args)
  File "/home/grant/Envs/py34test/lib/python3.4/site-packages/pip/commands/install.py", line 311, in run
    root=options.root_path,
  File "/home/grant/Envs/py34test/lib/python3.4/site-packages/pip/req/req_set.py", line 646, in install
    **kwargs
  File "/home/grant/Envs/py34test/lib/python3.4/site-packages/pip/req/req_install.py", line 803, in install
    self.move_wheel_files(self.source_dir, root=root)
  File "/home/grant/Envs/py34test/lib/python3.4/site-packages/pip/req/req_install.py", line 998, in move_wheel_files
    isolated=self.isolated,
  File "/home/grant/Envs/py34test/lib/python3.4/site-packages/pip/wheel.py", line 345, in move_wheel_files
    console, gui = get_entrypoints(ep_file)
  File "/home/grant/Envs/py34test/lib/python3.4/site-packages/pip/wheel.py", line 225, in get_entrypoints
    cp.readfp(data)
  File "/usr/lib64/python3.4/configparser.py", line 736, in readfp
    self.read_file(fp, source=filename)
  File "/usr/lib64/python3.4/configparser.py", line 691, in read_file
    self._read(f, source)
  File "/usr/lib64/python3.4/configparser.py", line 1070, in _read
    fpname, lineno)
configparser.DuplicateOptionError: While reading from '<???>' [line  4]: option 'ptangle' in section 'console_scripts' already exists

I ran this using a fresh python 3.4 virtualenv:

$ pip list
pip (7.1.2)
Pweave (0.23)
setuptools (18.2)
wheel (0.24.0)

Pweave does apparently get installed, albeit without the scripts, despite the pip failure.

Evaluate inline code in cached documentation mode

Is there any way to include inline code when using the caching feature?

Release 0.30?

@rlaverde @abukaj @ccordoba12

I received a request to release for 0.3. As far as I know (with not very active use lately) there are no blocking bugs or incomplete features in the current master.

The blocking issue right now is that the documentation is not up to date. Would you like to see a release anyway with partially outdated documentation? Or would anyone have time see whats is broken in the docs at the moment?

The docs are in https://github.com/mpastell/pweave-docs, and should also be moved under this repo.

IPython notebook formatter

There are currently these output formats:

formats = {'tex' : {'class' : PwebTexFormatter, 'description' : 'Latex with verbatim for code and results'},
            'texminted' : {'class' : PwebMintedFormatter, 'description' : 'Latex with predefined minted environment for codeblocks'},
            'texpweave' : {'class' : PwebTexPweaveFormatter, 'description' : 'Latex output with user defined formatting using named environments (in latex header)'},
            'texpygments' : {'class' : PwebTexPygmentsFormatter, 'description' : 'Latex output with pygments highlighted output'},
            'rst' : {'class' : PwebRstFormatter, 'description' : 'reStructuredText'},
            'pandoc' : {'class' : PwebPandocFormatter, 'description' : 'Pandoc markdown'},
            'sphinx' : {'class' : PwebSphinxFormatter, 'description' : 'reStructuredText for Sphinx'},
            'html' : {'class' : PwebHTMLFormatter, 'description' : 'HTML with pygments highlighting'},
            'md2html' : {'class' : PwebMDtoHTMLFormatter, 'description' : 'Markdown to HTML using Python-Markdown'},
            'pandoc2latex' : {'class' : PwebPandoctoTexFormatter, 'description' : 'Markdown to Latex using Pandoc, requires Pandoc in path'},
            'pandoc2html' : {'class' : PwebPandocMDtoHTMLFormatter, 'description' : 'Markdown to HTML using Pandoc, requires Pandoc in path'}
            }

A really powerful addition would be an IPython notebook formatter. Then docs
written in markdown could be converted to notebooks.

I don't think it would be too hard, as the IPython notebook is just a JSON.

Feature request: allow error ignore

Currently, some errors cause execution to pause. Allow ignoring all errors.