vrgrouprwth/insite

Insite - A Pipeline Enabling In-Transit Access To Neural Simulation

Documentation

You can find the information provided in this readme and the complete documentation here: https://vrgrouprwth.github.io/insite/

Insite - A Pipeline Enabling In-Transit Access for NEST, TVB and Arbor.

This research was supported by the EBRAINS research infrastructure, funded from the European Union’s Horizon 2020 Framework Programme for Research and Innovation under the Specific Grant Agreement No. 945539 (Human Brain Project SGA3).
This project/research has received funding from the European Union’s Horizon 2020 Framework Programme for Research and Innovation under the Specific Grant Agreement No. 945539 (Human Brain Project SGA3) and Specific Grant Agreement No. 785907 (Human Brain Project SGA2).

Goal

Insite provides a middleware that enables users to acquire data from neural simulators via the in-transit paradigm. In-transit approaches allow users to access data from a running simulation while the simulation is still going on. In the traditional approach data from simulations is written to disk first, and can only be accessed after the simulation has finished. However, this has two main constraints:

1. Data can only be further processed after the whole simulation has finished.
2. Disk speed can be a bottleneck when simulating, as data has to be written out.
3. Data must be completely stored on the machine, leading to large files.

Using Insite allows users to develop data consumer, such as visualizations and analysis tools that allow early insight into the data without storing data with virtually zero dependencies.

Insite was specifically designed to be ease-to-integrate and easy-to-use to allow a wide range of users to take advantage of in-transit approaches in the context of brain simulatiion. Insite uses off-the-shelf dataformats and protocols to make integration as easy as possible. Data can be queried via an HTTP REST API from Insite's access node, which represents a single point of contact for the user.

insite's Issues

From and To filter for spikes

The time based filtering for spikes does not work. Even if to and from parameter are provided the returned spikes are always all the spikes that are in the ring buffer.

Reset stored data between simulation runs

Currently the data in the ring buffer isn't reset between new simulations. Thus, data from older runs are still available in the new run.

[DESIGN] NEST Monitoring Interface Functionality

List of NEST & Insite properties/metrics that would be useful to query via Insite:

• Ringbuffer of population rates
• memory usage
• CPU usage
• power usage
• spikes per process (current load of communication structure)
• pyNest timers (duration last communication step, ...)

First review, first issues.

Hello all,

I am using insite for my project to show a small demo of plotting of spike events.
For all of these issues, I used the develop branch.

docker-compose run
There, I tried setup these containers according to the instruction (https://github.com/VRGroupRWTH/insite/tree/develop).
First I executed this command: docker-compose up --build then I entered the command: docker-compose run insite-nest-module /example/brunel_simulation.py.
It works well but it produces this 500 Internal Server Error, when I call a request of localhost:8080/nest/spikes.
So, I keep starting insite with docker-compose up --build. It works well.

spikes of populations
Next issue, I am able to gather spikes from localhost:8080/nest/spikes. But for different dot colors of spikes in populations, I call ids of spikes localhost:8080/nest/populations, then I want to request spikes for different populations, it returns all spikes instead of the corresponding populations. As example, I called localhost:8080/nest/population/$5104784/spikes.

spikes from specific time
Anyway, I collected all spike but I would like to get spikes of a time window. You implemented from and to arguments.
I assume these arguments set the time window of getting spikes. When I call localhost:8080/nest/spikes?from=50&to=90. It return spikes from 0 to 90. I think to argument works but from argument doesn't.

I summarized these issues:

• docker-compose run
• spikes of populations
• spikes from specific time.

Configuration file for insite

What has to be configurable:

So far:

• Access Node Port
• Ring Buffer Size

[bug] lastFrame of activity request

When the simulation is finished and the frontend requests the last frame from access node.
The lastFrame indicates that there is no further data after this request.
But the lastFrame seems not working correctly. It only shows false.

Expose NEST server

Expose NEST server in the docker build such that both the Insite access node and NEST server are reachable from outside