jiffyclub / snakeviz Goto Github PK
View Code? Open in Web Editor NEWAn in-browser Python profile viewer
Home Page: https://jiffyclub.github.io/snakeviz/
License: Other
An in-browser Python profile viewer
Home Page: https://jiffyclub.github.io/snakeviz/
License: Other
Cool idea guys.
I can click on rings of the graph and the graph updates. However, hovering does not display any information about what function is associated with the ring. Clicking on rows in the table appears to not update anything either.
I've generated a simple profile with this script:
import time
def other_function():
time.sleep(0.01)
def function():
time.sleep(0.1)
for i in range(10):
other_function()
def main():
for i in range(15):
function()
if __name__ == '__main__':
main()
and then generate profile data with
python -m cProfile -o output.prf profile.py
and then start snakeviz with
snakeviz output.prf
Could you add a super-simple example to the docs that first-time snakeviz
(or maybe even Python profiler) users can run and then play with the GUI?
I first tried this which timed out when trying to load the profile into snakeviz
and second this which does load, but doesn't contain any useful profile info (clicking on JSON gives this).
Maybe there could even be an examples
folder with 3 to 5 interesting profiling examples of increasing complexity that the user can run to learn how it works?
I'm a relative noobie to python so I'm missing something simple because I can't seem to run SnakeViz on my Mac (10.8.5, using fink python).
The website says to simply type snakeviz name_of_profile on the command line, but that doesn't seem to work, with the error command not found. I must be misunderstanding something basic because usually I run python scripts with something like python snakeviz.py name_of_profile, but there doesn't seem to be a snakeviz.py or anything similar that I would be able to call.
I'm pretty sure it's installed correctly because when I enter the python interpreter and type import snakeviz it appears to be successful, I just don't know how to actually invoke the program.
Can you help?
Tom
File "snakeviz/stats.py", line 51
return {keyfmt(*k): v for k, v in d.items()}
^
SyntaxError: invalid syntax
Dict comprehensions are new in 2.7.
This issue is a replacement for #2 and eventually even this issue may need to be broken out into some more specific issues.
At the moment "large" profiles cause a number of problems for websnakeviz:
stats_to_tree_dict
function by creating so many dictionaries that memory usage becomes unsustainable.stats_to_tree_dict
does return the dictionary the tree may be so large that it swamps the json.dumps
function.This issue should be a good place to discuss how to tackle the various aspects of large profile handling and any specific plans can spawn new issues.
I had never seen the Sunburst
visualisation before, and while it is very pretty, I don't find it very intuitive to browse. How much work would it be to add e.g. a square map visualisation?
I recently upgraded my computer to OS X 10.10 Yosemite and re-installed Macports and snakeviz
.
Now snakeviz
works on the command line, but %snakeviz
in IPython doesn't work:
ERROR: Line magic function `%snakeviz` not found.
The snakeviz install instructions don't mention that one has to register %snakeviz
with IPython somehow and I think last time this just worked out of the box for me !?
Any idea what is wrong with my setup or what I have to do?
First off, I wanted to say thanks. I just started using this today, and the visualization is fantastic.
It would be nice to have an option to set the root directory for snakeviz, so the url doesn't need to include the absolute path. For example, if I generate a lot of profiles into a single directory, this would make it much faster to manually change the url to view them
The file browser you have discussed in some other tickets would be a nice feature also.
I installed snakeviz, but cant get it to work with my ipython notebook. I running the follwoing code:
%snakeviz
ms = [5,10,15,20]
ns = range(5,10,1)
thd = 2
for m in ms:
mlabel = "losing prob: m=%s" % m
prob = []
for n in ns:
prob.append(estimate_losing_prob(m,n,thd))
and I'm getting:
*** Profile stats marshalled to file u'c:\users\idoda\appdata\local\temp\tmp03iiiv'.
What am I doing wrong?
Selecting a row in table should highlight that function in the visualization, possibly in a different color than the mouse-over highlighting. What should be done when the user mouses over a node highlighted by a table selection? The Google table implementation currently in use allows for setting up selection event handlers. The table allows for multiple rows to be selected.
Some documentation on the table spec is here: https://developers.google.com/chart/interactive/docs/gallery/table
Currently the stats table data is generated by capturing the output of pstats.print_stats(), which limits the numerical data. Any time smaller than 0.001 is shown as 0. This should be improved, especially so that the table sorting can use the full precision of the data.
Table cells can be be specified with both a numeric value and a string representation. For display purposes it would probably be best to calculate an explicit string representation of numbers (e.g. '10.4g', which automatically switches to exponential notation for large/small numbers) instead of having the table display all of the decimal points.
There are still some occasional errors calculating the correct sizes of nodes, see for example the norm_fit.prof example profile. Some nodes end up with a much larger angular extent than their parents.
Probably related to this there are cases when the summed sizes of a node's children is larger than the node's size. The code currently glosses over this but it's actually an indication that something might come out incorrectly in the display.
This all seems connected to the naive handling of calculated sizes when there are recursive calls, in that case the sum of calling parent.child_cumulative_time
on all of a node's children will result in a value greater than 1. There should be some normalization or other appropriate handling to account for that.
If you start SnakeViz and it fails to open a browser the user has no way to know what URL to use to manually point a browser at the server. SnakeViz should print a complete URL to the terminal that a user can copy and paste into a browser to see the application. (Spun off from #46.)
The current highlighting darkens nodes, which is not always effective. A quick short term solution is to use a distinct, bright color for highlighting. Eventually it would be cool to have animated, pulsed node highlighting to really draw the eye.
Hi! Thanks for this wonderful tool! I'm using it a lot and many times I don't have direct access to where my files and soft are. So I use port forwarding a lot. However, this doesn't work with snakeviz.
Could it be due to absolute vs relative paths in the web application?
Thanks in advance to check this out and let me know if I can help
I would like to include the central snakeviz diagram in a blogpost. Is there a convenient way to make this possible?
Hi,
Just a quick question because I'm really interested by snakeviz. Is it possible to profile memory usage with snakeviz ? Runsnakerun could do it with meliae.
Thanks !
As described at https://github.com/jiffyclub/snakeviz/blob/master/snakeviz/cli.py#L20 the automatic port selection is currently not a thing.
I've had a go using runprofileserver: https://github.com/django-extensions/django-extensions/blob/master/django_extensions/management/commands/runprofileserver.py
but I get:
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pstats.py", line 110, in load_stats
self.stats = marshal.load(f)
ValueError: bad marshal data (unknown type code)
both with and without the --kcachegrind switch. I'm guessing that the output isn't in the right format but I don't know much about this topic so I'm hoping you can shed some light.
Would it be possible to turn filename:lineno(function)
like glob.py:20(iglob)
into links that open the file at that line when clicked?
This would allow quickly browsing and reading the pieces of code that appear of interest due to their profile characteristics.
I don't know if the profiler stores the full file path, but it seems this should be possible for the common case where someone just created a profile and then runs snakeviz on it with the same Python interpreter on the same machine with the same site-packages
?
Hi!
I'm not sure that this is the best place for it. I wanted to make an improvement proposal, it would be really great to be able to view several reports at a time, maybe with some search functionality across the reports. I've bumped into the problem when I'm working with 66 files of profiling data, but maybe I miss some obvious points?
I added a single_user_mode in 91413da that's specifically for a single user running the webserver on their local desktop, and it has very different security assumptions, for example allowing arbitrary files on the filesystem to be opened. This being the case, we can also build in a file browser to make it easy to switch between profile files in a directory.
Obviously this is not high priority, but it could be nice.
If there is an error in the JSON handler the user is presented with a big empty SVG and the stats table. There should be an error message to indicate something has gone wrong.
When JSONHandler
fails to return the d3.json
callback function is called with null
so that function should be expanded to handle the error case.
The tooltip is currently always placed above and to the right of the mouse. It would be nice if it was smart and could change positions based on its position in the page so that, for example, it didn't run into the right hand edge of the browser window.
The sunburst visualization shows bizarre, incorrect behavior when zooming to the time.sleep
leaf nodes in the sleep_recurse2.prof
sample profile. This may or may not be related to issue #8.
Users could potentially load profiles that take a couple seconds to load or completely overwhelm the profile handling. We should add a loading graphic that displays while the profile is being processed, and the processing should timeout after ~10 seconds and the web page should display an error and suggestion like:
"There was an error processing this profile. You could try profiling a smaller piece of your program and displaying that. You can use the table below to inspect which functions are taking the most time."
I try setting the port for snakeviz and it always runs on 8080.
Make it possible to collaps the div containing the figure to make it easier to work with the stats table, which is pretty useful in and of itself.
Environment: Macbook Air 2012, OS X 10.8.2, Safari 8.0.3., bash-3.2.57, python 2.7.5 (Mac default), snakeviz-0.2.1
Repro
Problem:
See this message hanging forever: "snakeviz web server started on 127.0.0.1:8080; enter Ctrl-C to exit", and nothing happens beyond this.
Expected: Browser (Safari by default) should open and show some eye candies.
I pip installed snakeviz and the resulting install didn't have bootstrap's img directory.
Make an IPython magic extension that launches WebSnakeViz. Could work much like the built-in %prun (%%prun) magic (and be built on top of that):
%load_ext wsv
%wsv myfunc(args)
%%wsv func1(arg1)
func2(arg2)
Maybe a missing JS issue? Nothing happens when I click on different parts of the graph. Except the Reset button become enabled. Ubuntu 14.04, Chrome 36.0.1985.125.
Firstly, SnakeViz is awesome. Thanks for putting it together.
We have machines deployed on AWS Ubuntu machines, and use IPython Notebooks on those. When I run SnakeViz in those, I get a response *** Profile stats marshalled to file u'/tmp/tmp8aZzx1'.
, but no new window / tab opens up.
When I try the exact same code on my local MacOS X machine (the glob
example), a new tab pops up with every run, just at it should.
Is this a problem inherent in the notebook? Or is this something that could be solved within SnakeViz?
Thanks,
Max
Give give users the ability to zoom the sunburst by clicking on a node. This will make the node's parent the new root node. After zooming users will be able to zoom out a level by clicking on the sunburst center. A "Home" or "Reset" button should return to the most zoomed out (default) view.
An example: http://www.jasondavies.com/coffee-wheel/
The figure size is currently hard-coded in wheel.js
. We should probably make it dependent on the browser's window size.
Using snakeviz 0.4:
Traceback (most recent call last):
File ".../site-packages/tornado/web.py", line 1413, in _execute
result = method(*self.path_args, **self.path_kwargs)
File ".../site-packages/snakeviz/main.py", line 30, in get
table_rows=table_rows(s), callees=json_stats(s))
File ".../site-packages/snakeviz/stats.py", line 65, in json_stats
(keyfmt(*ck), list(cv)) for ck, cv in stats.stats[k][-1].items())
File ".../site-packages/snakeviz/stats.py", line 65, in <genexpr>
(keyfmt(*ck), list(cv)) for ck, cv in stats.stats[k][-1].items())
TypeError: 'int' object is not iterable
I recently updated snakeviz from 0.1.4 to latest version using pip upgrade. The reason of upgrading it is that the chart was not getting displayed. Although the table with calls and other data was there. After reading this article (http://penandpants.com/2014/12/01/snakeviz-0-2/) , I thought it will work after upgrade. But after upgrade it even failed to show the table and gave following error:
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\tornado\web.py", line 1332, in _execute
result = method(*self.path_args, **self.path_kwargs)
File "C:\Python27\lib\site-packages\snakeviz\main.py", line 30, in get
table_rows=table_rows(s), callees=json_stats(s))
File "C:\Python27\lib\site-packages\snakeviz\stats.py", line 33, in table_rows
tot_time_per = fmt(v[2] / v[0])
ZeroDivisionError: float division by zero
From the error, it looks that error handling for 'Division by zero' is missing.
My Env:
Windows 7
Python2.7.8
I also tried running it on Python 3.4, but it also failed with following error:
Traceback (most recent call last):
File "C:\Python34\lib\site-packages\tornado\web.py", line 1332, in _execute
result = method(*self.path_args, **self.path_kwargs)
File "C:\Python34\lib\site-packages\snakeviz\main.py", line 27, in get
s = Stats(profile_name)
File "C:\Python34\lib\pstats.py", line 69, in __init__
self.init(arg)
File "C:\Python34\lib\pstats.py", line 83, in init
self.load_stats(arg)
File "C:\Python34\lib\pstats.py", line 97, in load_stats
self.stats = marshal.load(f)
ValueError: bad marshal data (unknown type code)
Note: I can't share cprofile output as it has customer data.
Thanks,
Naveen
One useful feature of runsnakerun that might be a good addition to snakeviz would be an option for switching the displayed timings from the current absolute scale (seconds) to percentages of the total runtime.
This might also provide a good discriminator for dropping branches of the call tree with a small cumulative runtime (i.e. by default drop branches with a cumulative time of < 0.1% of the total time, for instance).
One of the bottlenecks described in #12 is that it's not sensible to construct a JSON string of the entire call tree for a lot of profiles. It just takes too long and takes up too much space. We can get around this by only JSON-ifying the pstats.Stats.stats
dictionary and constructing the call tree as a JavaScript structure in much the same way that the runsnake.pstatsloader
module does in Python.
I recently installed Snakeviz for the first time using pip install snakeviz
on my work laptop (thanks for making it pip-installable) which runs 64-bit Windows 7. After the speedy install, I was ready to test it on a profile file I had laying around. I fired up the server with snakeviz test.prof
and was promptly greeted by this in Chrome:
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\tornado\web.py", line 1037, in _execute
self.prepare()
File "C:\Python27\lib\site-packages\tornado\web.py", line 1487, in prepare
raise HTTPError(self._status_code)
HTTPError: HTTP 404: Not Found
I noticed the URL looked funny:
http://127.0.0.1:8080/viz/filec:/Users/mscruggs/Desktop/test.prof
I was able to get the page to load and use the tool by removing the drive portion of the URL (I removed "c:"):
http://127.0.0.1:8080/viz/file/Users/mscruggs/Desktop/test.prof
I placed print self.request
on a new line in tornado/web.py just prior to self.prepare()
on line 1037:
The output:
HTTPRequest(protocol='http', host='127.0.0.1:8080', method='GET', uri='/viz/filec:/Users/mscruggs/Desktop/test.prof', version='HTTP/1.1', remote_ip='127.0.0.1', body='', headers={'Connection': 'keep-alive', 'Accept-Language': 'en-US,en;q=0.8', 'Accept-Encoding': 'gzip,deflate,sdch', 'Host': '127.0.0.1:8080', 'Accept':
'text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8', 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.66 Safari/537.36'})
I'm running Tornado 2.4 and Jinja2 2.7.1 on Python 2.7.5.
Please let me know if you have any questions or requests for additional debugging output/info.
In some situations where a browser is not available it may still be useful to start the SnakeViz server and connect to it by other means. SnakeViz should have an option that causes the CLI to start the server, but not try to open it in a browser. (Spun off from #30.)
I'd find the data in the table much easier to read if it would be in a monospaced font and slightly more dense.
The function stats_to_tree_dict
in upload.py
currently fails for some profiles for unknown reasons. The current known profile which causes it to fail is large, deep, and contains complex recursion.
>>> os.path.abspath('/c:/myfile')
'C:\\c:\\myfile'
This bug in class VizHandler(handler.Handler)
causes snakeviz to just not work on windows. Also, it seems the tab in the browser opened is http://127.0.0.1:8080/viz/fileU:/test.pyprofile
i.e. notice there's no slash after the file
but I haven't debugged that.
As noted in #46 (comment), the help text of the -b
option should note that browsers can be specified by the names described on this list: https://docs.python.org/3.4/library/webbrowser.html.
Using version 0.1.1 on Mac OS X Mavericks, I am noticing some issues with the tool tips when on the right-side of the circle. In particular, in Safari the tool tip flashes then vanishes. In Chrome, it stays visible, but flashes repeatedly whenever the mouse is moved (occasionally it vanishes). Firefox has a behaviour that averages the two - less flicker than Chrome, but it vanishes more frequently.
Tool-tips on the left half of the graph are fine in all three browsers. This is most-likely related to issue #4.
Firefox: 24.0
Safari: 7.0
Chrome: 31.0.1650.57
Michael
P.S. It might be useful to include a --version
option to make it easier for people to report the snakeviz version for bug reports.
P.P.S. (did I mention how fantastic snakeviz is?!?! especially given how difficult it is to get RunSnakeRun to work on a 64bit Mac! thanks!)
I'm on a super-slow internet connection and get this error:
Error Loading Profile
Sorry, we were not able to load this profile! You can try profiling a smaller portion of your code. The statistics table below might help you narrow down which portion of your code to focus on.
I think the problem is that loading jquery.js
and d3.js
times out, but the error message doesn't say so.
The error message is incorrect in that there is not statistics table displayed below.
How about bundling the JS libraries?
(No-one really cares about an extra 1 MB these days, but slow internet connection or offline use happen.)
I'm getting this generic error. Is my script just too big?
Error Loading Profile
Sorry, we were not able to load this profile! You can try profiling a smaller portion of your code. The statistics table below might help you narrow down which portion of your code to focus on.
Looks like there's some syntax issues with trying to run this in python3. I.e.:
$ snakeviz ./Framework/Settings/Classes/tests/String.profileTraceback (most recent call last):
File "/usr/local/bin/snakeviz", line 9, in <module>
load_entry_point('snakeviz==0.1.1', 'console_scripts', 'snakeviz')()
File "/usr/lib/python3/dist-packages/pkg_resources.py", line 351, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/usr/lib/python3/dist-packages/pkg_resources.py", line 2363, in load_entry_point
return ep.load()
File "/usr/lib/python3/dist-packages/pkg_resources.py", line 2088, in load
entry = __import__(self.module_name, globals(),globals(), ['__name__'])
File "/usr/local/lib/python3.4/dist-packages/snakeviz-0.1.1-py3.4.egg/snakeviz/cli.py", line 44
except IOError, e:
^
SyntaxError: invalid syntax
Curious if the scope of python2.x type stuff is know, and if there's some thoughts of making it python3 runable.
One can install this to the python2.x environment, but doesn't help you when trying to read python3 profiles:
File "/usr/lib/python2.7/pstats.py", line 110, in load_stats
self.stats = marshal.load(f)
ValueError: bad marshal data (unknown type code)
I'm trying to run this on my VM so I can look at the results from the host machine which doesn't have Python. However I can't find a way around this error:
snakeviz: error: no web browser found: could not locate runnable browser
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.