Comments (13)
I have the same problem, please fix this. The way it works now is that i have to cache all chunks again with Pweave -f texminted -c %.texw when adding a new chunk
from pweave.
I don't have time to work on this at the moment. I agree that the implementation is not ideal, you're welcome to submit a pull request if you have a suggestion on how to fix it.
Note that Pweave only caches input and output text and not Python objects, so if new chunks need the data from old ones there is no easy fix to this problem.
from pweave.
Gotcha. I’ve been making some small changes toward those ends,
so—hopefully—I’ll have a pull request for you.
On Tue, Mar 31, 2015 at 3:34 PM, Matti Pastell [email protected]
wrote:
I don't have time to work on this at the moment. I agree that the
implementation is not ideal, you're welcome to submit a pull request if you
have a suggestion on how to fix it.Note that Pweave only caches input and output text and not Python objects,
so if new chunks need the data from old ones there is no easy fix to this
problem.—
Reply to this email directly or view it on GitHub
#19 (comment).
from pweave.
Seems like one could simply bypass caching in documentation mode and use the caching magic in an IPython processor. A subclass of PwebIPythonProcessor
that loads the extension and adds the magic before the self.IPy.run_*
statements might do the trick.
from pweave.
Has there been any activity on this?
I'd really appreciate chunk-level caching functionality, which seems like it would be closely related.
My use case: I have an increasingly long document with more and more pweave-generated figures, where I'd like to only have to recompile the one I'm currently working on.
Thanks for creating pweave! It's encouraged me to plot more graphs, which is always good :-)
from pweave.
I've been slowly taking a shot at improved caching (see here), but progress has been slow due to multiple competing interests. Namely, a desire to
- fold inline chunks into the general chunk framework,
- provide multi-line chunk options,
- provide generalized caching
- e.g. naive output-only caching that considers changes in buffer content/source and chunk settings,
- make everything work almost entirely within the Jupyter ecosystem
- every chunk evaluation engine is necessarily a Jupyter kernel
- use of nbformat as the underlying parsed document format,
- and provide precision Python-only caching
- bytecode-aware caching, via the mechanics behind the
with
hack given here.
- bytecode-aware caching, via the mechanics behind the
from pweave.
@brandonwillard Those are multiple big changes that you are talking about. Please don't submit them as one pull request, but split it into separate ones.
Note:
- Every chunk evaluation engine is already a Jupyter kernel
- I don't see the benefit of using nbformat as the parsed document format, you can already use it for output.
I suggest you first do:
- fold inline chunks into the general chunk framework
- provide generalized caching e.g. naive output-only caching that considers changes in buffer content/source and chunk settings,
I have decided not to allow multi-line chunk options as it breaks editor support and I haven't seen a compelling need for it. If you can up with a proper implementation with tests I can accept it, but put it as separate pull request.
from pweave.
Oh, sorry, I hadn't done that work with a PR in mind; it was just a test branch that started with caching and turned into all sorts of stuff. If there's an interest in those latter two goals, I can separate them and make PRs. As for the nbformat idea, I can start an issue discussing my reasons.
from pweave.
@brandonwillard how were you thinking of implementing save_chunk_state
master...brandonwillard:caching-changes#diff-2747ccbd23b5ea3c1c42eb01071e5a6eR166
from pweave.
Ah, yeah, I left off with the idea of incrementally pickling the session in _[save|load]_chunk_state
. This idea isn't all that efficient/feasible without, perhaps, an incremental approach.
At around the same time, I was experimenting with a more granular, variable-level caching that uses code/ASTs extracted from with
bodies and had intended to port this idea instead of using (incremental) session caching.
Regardless, I've gone full org-mode nowadays, so I don't know when I'll get time to jump back into this!
from pweave.
Thanks @brandonwillard.
from pweave.
@brandonwillard, both of the approaches you considered seem particular to python. Currently, it looks like Pweave is trying to not be tied to Python by using Jupyter to allow different kernels. Do you know if Jupyter kernel managers have a language-independent means to serialize the state of a kernel?
Stack Overflow seems to suggest no
from pweave.
Yeah, I think that any non-naive caching (e.g. more than just caching output and validating against source text differences) is necessarily language-specific.
However, it seems like more than a few popular languages have straight-forward runtime bytecode tools, AST generation and — at the very least — introspection capabilities. As with Python, it's possible to implement a less naive caching with those.
Regarding Jupyter, it would be fantastic to see an abstraction of bytecode and/or AST objects exposed by the client protocol. The project has a somewhat related idea in its instrospection messages. Otherwise, one can always implement smart caching at the kernel level and use custom messages.
from pweave.
Related Issues (20)
- Remove `Published using Pweave` Notice HOT 1
- File Failed to Load MathJax HOT 1
- Broken since python3.8 upgrade HOT 3
- pweave using `results='tex'` in chunk headers creates empty lines in the tex file
- Feature request: include code from chunks with 'source' option in ptangle output
- chunk option `f_spines` documented but does not seem to be implemented
- Is there anyway to change the table format as in fig width
- Not compatible with `jupyter_client>=6` HOT 9
- `<%= %>` and `<% %>` have the same functionality
- figformat removes in >0.3?
- md to pdf using pweave HOT 1
- Any changes to handling of tikz diagrams?
- figure numbers in pandoc
- Is the project still active? HOT 6
- publish can easily support markdown format
- Can chunk option values be expressions?
- customizing fonts, code font size/syntax coloring, document color, etc.
- Not compatible with jupyter-client >= 7.0.0 HOT 6
- Syntax warning from __init__.py: assert file != "" is not None
- WeaveTest.testFIR_FilterExampleTex failing
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pweave.