vladimir-vg/batiscaph

Currently inactive. Erlang trace visualizer.

batiscaph's Issues

Broken tests

Code that currently reside in test/ directory was written having batiscaph working as a service with two apps in mind. After rejecting the idea of service it was refactored, and as a result current tests are broken.

I used docker to split different parts of the system into containers. Need to test different environments for probe (Erlang, Elixir, Cowboy, etc.).

Current setup is not very clean: I start containers with host network. I want to be able to connect to client erlang node, be able to make rpc and inspect some details -- much easier to test. Haven't figured out how to properly setup docker network to make it possible.

Probably I shouldn't put server code into container, it might be okay to work on host machine. Just for sake of simplicity.

To do:

1. Make testing code more simple and robust, refactor it. Make sure tests pass.
2. Keep docker containers for client nodes, need it to test different environments.
3. Try to make use of client nodes in testing code less verbose.

Feature usage summary

It would be great to mark certain functions as features and collect summary how often they are used. This might be convenient for understanding what features might be removed, what features are more critical.

Ideas:

• I also like the idea of marking certain ports, if they connect two parts of the system.
• It might be possible to use it in future for generating design schema, or attaching to already drawn schema, having it connected to recorded events.

Overall memory distribution

Erlang provides a lot of info about memory usage.

instrument module was rewritten in upcoming 21 release, and now allows to gather information without any special cmd flags: http://blog.erlang.org/Memory-instrumentation-in-OTP-21/

Would be great to have a page where we could see how memory is alocated, which parts take the most amount of memory.

Ideas:

1. Necessary to put on the same scale amount of physical RAM in OS, amount taken by Erlang processes, amount taken by other OS processes. Only inside chunk that taken by erlang processes we draw details about other memory consumptions. This will help to recognize leaks that happened somewhere on VM side, e.g. in NIF.
2. This information should be stored in such a way, that in future it would be possible to implement scrolling back in time, and see how things changed.
3. Use same idea of subscription as with process_info: when you on the memory page, changes are recorded. Unsubsribe on page exit.
4. Somewhat similar to this: https://bl.ocks.org/mbostock/4063582
5. At first I would use sidebar for tree display. Later it might move to main area.

Overall schema of system and use cases

I was thinking about how to make it easier to understand code.

I realized that having documentation with many schemas of different detailzation on different levels and different scenarios would simplify things a lot.

It would also be great if recorded events could be put on these diagrams, pointing out in which part of the system it did happen, what scenario was executed. Display these diagrams along with timeline, showing state at current timestamp.

The problems are:

1. Diagrams are hard to draw, programmers are not good at it
2. When code changes, diagrams often becomes obsolete. Takes too much work to keep them in sync.
3. Automatically generated schemas often too complicated, to verbose. Need human-guided generalization, splitting to layers.

It would be great if we could name parts of code (I called them features in #11), and somehow connect them to parts of the diagram. Also would be great to include these diagrams in documentation, allow to navigate different details of the system simply by clicking on parts of schema.

Maybe just draw SVGs, name some elements using specific ids, then import SVG, recognize ids and link them to different parts of documentation. Use same ids to mark parts of the code, as a result, allow to put some events on schema.

Shell commands history

Would be great to be able to reused previously typed commands.

Quite easy to implement. Just have a list of previously submitted commands and and index (current position in history).

1. If current input is empty, then
2. If key UP is pressed, then decrement index, take text from history by index, set to textarea.
3. If key DOWN is pressed, then inrement index, then do the same.

frontend/pages/ShellPage.jsx

Spawned processes marked as traced, even when they aren't

That's happening because it's implied that if process has spawn event it's already traced (like if all processes have set_on_spawn option enabled).

To fix that we need to catch both spawn and spawned trace events in batiscaph_probe_feature_procs.erl, and issue additional p2 erlang:process trace start on the latter. Also change code in frontend and delta, to take into account only trace start/stop events.

Process shell variable info

It's possible to examine every variable introduced in shell, find all pids, and highlight them on the map.

It's not that hard to implement, just trace shell:server_loop/7 calls. It was done before and might be found in old_code/old_vision_probe_collector.erl.

The hardest thing is to make convenient UI.

1. Same process might be mention in many different variables and sometimes deep inside term.
2. Same variable name might contain references to different processes. That's true because you can forget variables in shell using f(Var).

In past iteration it was displayed just as line from shell process to process mentioned in variable:

Which is not that convenient. For now I have following ideas:

1. If pids in variables are not traced, then place them on map as untraced, display as mention circle.
2. Lines from shell to process are too noisy. Enough to mark them somehow that they were referenced.
3. Might be good idea just to stack process expressions on the top of the shell panel, make them fixed (not scrollable). Process expression is an expression that extracts pid from variable. Most of the time it's just a variable, but sometimes not. For example: Sub#subscriber.consumer_pid, maps:get(pid, Var).

Recognizing a record in a variable is tricky. Actually it requires to store record_info in compile time, which requires parse_transfrom, which I would like to avoid.

Lager does store info about records in order to make it's lager:pr work. Might be good to reuse it for now.

Display current state of supervision tree

Would be great to find a way to collect information about changes in supervision tree without tracing every process. As far as I understand SASL would help us with that. When which process was started, restarted, terminatec.

Having that information collected we need to display it. I think that's better to display a static graph that changes as scrolling of timeline goes. Easier to understand.

Probably we should put on graph all processes visible on timelime plus any other parts of tree that user would decide to expand.

Still not yet clear how UI should look like. Need to think about it.

process_info events are still sent, even after deselection of the process

Just to remind myself to fix it.

Tree diff calculation

Originally events from probe should turn into visual elements following this scenarios:

1. Events sent from probe and stored into database
2. User open webUI, subscribes to specific instance delta.
3. Backend forms a tree from a chunk of events, sends it.
4. When new events arrived, diff-tree is produced and sent to web client.
5. If user scrolled back or forth in time, diff tree is produced from events of relevant events-chunk and sent to client.

The idea was build such a tree, that can be produced from different chunks of events, in ascending and descending order in time, and still get same result.

Currently 4. and 5. do not work as described. Instead of forming diff-tree, whole tree is calculated from scratch in batiscaph_delta_producer:send_recent_delta_for_subscribers/1.

Looks like this kind of tree delta will require to hold some intermediate state, and merge it.

Need to specifically test that exactly same tree is produced, when necessary events are located in different chunks.

Say we have events

timestamp type
1 http-request-start
2 event1
3 event3
4 http-request-stop
5 event4

Need to ensure that exactly same tree is produced when chunks 1-3, 4-5 are consumed (descending order) as well as 4-5, 1-3 (ascending order).

It will require to make such a test suite, that will take every delta producer feature, list of events, and try to produce trees with different splits.

Collecting lager log events

This would allow which process reported what.

Implement log collection only for lager for now. Other providers might be added later.

Even though log events would be displayed on process map, still better to have traditional text viewer somewhere. Probably just add additional Logs page to the sidebar.

I think it's okay to automatically start basic tracing on process that logged a message.

Batiscaf + Jupyter

Just an idea, what if integrate Batiscaf into Jupyter notebook using projects like https://github.com/filmor/ierl or https://github.com/robbielynch/ierlang. You automatically get the basic shell, history, ability to share notebooks, etc.

Pretty cool project!

Limit amount of automatically traced processes

Currently following events enable basic tracing on process:

• spawn from shell
• handling Cowboy request

In future logging would also turn basic tracing, described in #8.

It might be dangerous to allow to trace too many processes. The idea is to decide is to limit automatically enabled tracing for processes. Decide in inside MatchSpec in erlang:trace_pattern/3 using {get_tcw}, {set_tcw, Value}.

Provide user options to enable/disable automatic tracing on requests, logged processes.

Visual display of refcounted binaries for process

erlang:process_info(Pid, binary) provides list of tuples with information about binaries referenced by process.

Would be great to visualize it into set of rectangles, according to their size. Would help to quickly notice big binaries. Somewhat similar to this: https://bl.ocks.org/mbostock/4063582

Thoughts:

1. Linear scale from bytesize to pixels might be not a good idea. Binary size might be couple of hundred bytes, as well as hundreds of megabytes. Need some kind of progressive scale.
2. Just draw divs with fixed width/height, and let css flex do the layouting. Might be not the prettiest way, but certainly the easiest way.

Common test events capture

It's possible to start batiscaph_probe from the Common Test hook and collect info about running tests.

Callbacks like init_per_suite, end_per_testcase might be displayed same way as cowboy requests -- just rect on top of the process.

Apart from displaying tests on map, need to display concise summary on Common Test tab. Some kind of tree of cells representing tests?

Compact processes layout

Currently each process has it's own column to display on. It was very easy to implement, but not very compact.

The idea is to reuse column, if process on it already died.

produceResolveFunc

Currently all spawns are drawn strictly from left to right. This is done in ensureSavedInSpawnTree and enumerateSpawnTree functions in layout.js.

Haven't figured out what's the best way to do it yet.