IPython magic extension

[jiffyclub]

IPython magic extension

[embray]

I wonder how well it would work to launch a web server from within IPython. I mean it's not that big a deal, but we might have to include some magic to kill the server too.

[embray]

Or, for that matter, the magic should also be able to reuse an existing wsv server instance to open different profile outputs--shouldn't be a big deal. The server itself shouldn't need much/any modification for that.

[embray]

Would be awesome to integrate this as an IPython Notebook widget, if such a thing even makes sense. I hope we can get back to working on this sometime.

[jiffyclub]

Should add a pointer to #20 here, where I did add a (lame) snakeviz magic.

[jiffyclub]

Yeah it would be awesome to make an IPython widget out of this. I think you can even make full-screen widgets. Things I've learned can happen, though not totally how to do:

• you can dynamically add routes to the running Tornado process
• you can install static assets to nbextensions in .ipython and load them from there

[jiffyclub]

Working on snakeviz has been hovering near the top of my work queue for a while, but a) it's a big job because it basically needs a re-write, and b) my work queue moves very slowly. 🐌

[embray]

Ditto to that. I need to check in with the IPython folks on where things are with nbextension installation too. There was talk at SciPy about making them easier to install and that Min has thoughts on that. But I dunno if there's been any action on it yet. Seems like something I might be able to help with somewhere...

[stefanseefeld]

Forgive my ignorance, but what issue do you expect to prevent the snakeviz output to become an inlined notebook widget ?
I'm mainly asking because I have attempted just that, but ran into javascript issues that I don't understand. See jupyter/notebook#749 for a bug report, and https://github.com/stefanseefeld/notebook/tree/bug/testcase for my naive attempt to do this.

I would love to get this working, as it makes the iterative profiling / optimization of small chunks of python code so much more convenient.

[embray]

I don't know exactly--last I worked on this loading extra javascript in IPython extensions designed to work with the notebook was a new frontier and required significant hacks. Things have changed a lot since this issue was first opened, and I for one haven't kept up with the details. It should be significantly more doable now as a Jupyter Notebook extension.

[stefanseefeld]

OK, so given that jupyter/notebook is using requirejs to load its javascript modules, it seems one way to make snakeviz compatible with it would be to also use requirejs in snakeviz (which implies AMDifying all the used javascript modules).

[jiffyclub]

SnakeViz could definitely benefit from some modernization on the client side, but I'm unfamiliar enough with all the different JS best practices to know what's best and what challenges there will be. (I've used browserify, but not RequireJS.)

One thing that's a bit odd with SnakeViz is that it uses a web worker to generate the visualization data for D3. There's a (fairly small) limit to how much data can be passed to/from a web worker and that precludes passing the Python profile data into the worker. I define the web worker in <script> tags in the HTML template so that the profile data can be embedded directly in the worker code by the server. That may not be a problem for RequireJS, but I have no idea.

[stefanseefeld]

I made some progress on the "IPython widget". The first insight was that I could work around the javascript incompatibility simply by embedding the snakeviz html via iframe into a notebook. I then had to modify the template to actually embed all the scripts, as the notebook server wouldn't be able to find the additional javascript an CSS unless specially configured for that purpose.
This may not be the most elegant solution, but it works, and thus gives me a nice way to seamlessly integrate profile visualization into notebooks.

[jiffyclub]

Would it help at all if the SnakeViz JS and CSS were available from a CDN?

[stefanseefeld]

I'd still want a fallback solution for offline use, so in that sense this wouldn't make the code cleaner.

[tcwalther]

@stefanseefeld @jiffyclub I realise this is quite an old thread, but I wondered if you had any plans to pick this up again?

This feature would solve another use case of mine, namely running %snakeviz on a remote Jupyter server. Right now, when running the %snakeviz magic, the browser open (of course) doesn't work. I have to copy the path of the temp file that is printed to the notebook output cell, in a separate terminal run snakeviz -H 0.0.0.0 /path/to/tmp/file, then manually copy the http:0.0.0.0:8080/snakeviz/url URL to the browser and substitute 0.0.0.0 with the real IP of the remote machine.

The same problem also applies when running %snakeviz inside a local Docker container. Instead of making a feature request to support remote instances via %snakeviz, a Jupyter widget would solve this problem altogether.

[jiffyclub]

As of #121 the %snakeviz magic embeds within the notebook, hope that helps everyone who brought this up!