p2psp/simulator

A complete stand-alone simulator for the P2PSP protocol

Welcome to the Peer-to-Peer Straightforward Protocol (P2PSP) Project!

P2PSP is a protocol for the streaming of real-time video/audio sequences on Peer-to-Peer (P2P) networks. In this repository you can find an implementation.

This is a description of the files/directories:

• doc: information about compiling, using, etc. P2PSP.

• README.md: this file.

• src: the implementation.

• tools: some useful scripts.

More information about the protocol at:

• Visit P2PSP Webpage

Thanks for downloading the P2PSP and happy broadcasting!

The P2PSP team.

simulator's Issues

Implementation of FCS (Free-riding Control Set)

Peers should provide better QoS to those neighbors that provide QoS to it. More information at FCS.

missing .gitignore

.gitignore file is missing so, there is always a possibility of adding unwanted files (like .pyc or venv) to the git file tree.

Playback control for simulation visualisation

It would be great to add a playback control using a slider (timeline in rounds) and buttons (play/pause).

Implementation of a Vorbis/Theora transcoder for Scalable Coding

Similarly to #51, a Scalable Video Coding (SVC) transcoder could improve the performance in variable-bandwidth scenarios. In this case, the forward transcoder would input an OGG stream and output a number of layers (OGG streams), each one presenting a different level of a Laplacian Pyramid.

The inverse transcoder would reconstruct the original OGG sequence (or an approximation of it) using all (or a smaller number of) layers.

Select colormap

Allow to select between different colormaps could be really nice.

https://matplotlib.org/examples/color/colormaps_reference.html

Feeding the simulator with a real source

At this moment, we simulate that the chunks of stream are received by the splitter, at a chunk-rate controlled by a sleep(). In the current implementation, the chunks are "empty" (only a single character is transmitted). This is quite simple, but the simulation times are not accurate.

The splitter, as in a real situation, should be fed by an Icecast or VLC instances. The chunks should be transmitted.

We can control the number of transmitted chunks per second by changing the chunk size and the stream bit-rate (using, for example, stream with different bit-rates).

Implementation of a Vorbis/Theora transcoder for Multiple Description Coding

Multiple Description Coding (MDC) is a coding technique that can improve real-time communications in variable-bandwidth transmission contexts. On the other hand, the Multi-Channel Set (MCS) can be used to receive more than one stream simultaneously.

In this scenario, an forward MDC transcoder would input an OGG stream and output so many OGG streams as descriptions. A good starting point can subsample the images using a simple pattern:

+---+---+
| a | b |
+---+---+
| c | d |
+---+---+

if a, b, c and d are neighbor pixels of an image, description (images) A would be formed only by the pixels a, description B only by b pixels and so on.

The inverse MDC transcoder would compose the original resolution of the video using one or more descriptions (missing descriptions should be interpolated).

Implementation of a streams merger

A streams merger is an HTTP client that inputs two (or more) streams from two (or more) peers that belong to two (or more) teams that are broadcasting the same content, and outputs a stream that is the union of the inputs. A stream merger can be useful in those cases where the loss of chunks in the teams is high.

Some useful stuff can be found here: https://github.com/P2PSP/console/blob/master/src/synchronizer.cc

Avoding burst mode in peers

Using a list of destinations (peers) for every received chunk and indexed by the origin peer of such chunk, it can be possible to send more than one chunk for every received chunk in those situations where peers, by a requirement of the topology, must send more than they receive.

This issue should be implemented in branch "avoding_burst_mode".

Lost data recovery system using error correction codes

The communication channels that we use is not a reliable source of transmission. It is common that the data received after the transmission is changed. This is mainly due to the introduction of noise during the transmission of the message.

This problem can be solved by using a technique called Forward Error Correction. Here, the message is encoded in a redundant way using an error correcting code. The redundancy enables the receiver to detect some of the errors that may occur anywhere in the message, and often to correct these errors without retransmission of data from the source.

Reed-Solomon is one of the oldest algorithms used for this purpose. This process encodes a message of length k into a codeword of length N (>k). Any combinations of k words received can be used to recover the complete data. This process to transmit data with increased reliability over the transmission channel.

More info can be found out on Wikiversity

A peer should be able to send the chunks to more than one player

In the current version of the protocol, one peer only can serve to one player (notice the data flow is controlled by the source, not by the peers or the players). It would be interesting that a peer can serve to more than one player (again, the data flow should be only controlled by the source).

Add examples to README.md

IMHO it would be better if the documentation had some sample examples for using the simulator, so that new-comers can get a better understanding of how it works and also the source code.

Implementation of a splitters synchronizer

Issue #48 introduces a new entity which is able to synchronize two or more streams.

This issue proposes an alternative (and probably much more efficient) way of obtaining the same result. In this case (as it is explained in the MCS layer) if the splitters conform the chunks exactly with the same content, two or more peers (or a peer with the ability of retrieve chunks from more than a team). For achieving this, the load-balancer should provide instructions to the splitters to be synchronized.

Some useful code can be found here: https://github.com/P2PSP/console/blob/master/src/synchronizer.cc

The process to send pending chunks before leaving the team freezes the peer

I commented two lines in peer_dbs (see https://github.com/P2PSP/simulator/blob/master/src/core/peer_dbs.py#L610)

The reason of those lines is to send pending chunks before leaving the team in a polite way. However, it freezes the peer in a receive call.

Representation of a half-duplex or a full-duplex link.

It would be interesting for the network overlay graph.

IndexError when it draws in real time

A IndexError occurs randomly in the draw function when it is drawing in real time (while it is simulating).
It could be because it reads a new line when the line is not fully written, so it splits something in a wrong way, causing that m doesn't have a correct format (there are missing elements).

Now it is catched, but with no action. Maybe we could repeat the reading of the same line in this case.

Defining default parameters for the simulator

What if we use a poisson distribution for peer arrivals instead of a multinomial?

Please, share your thought about it.

Peer lists sorted by RTT

Now, lists of peers are sorted by the order the incoming peers join the team. This implies that the first peers to join will be the first peers to be served by the rest of peers when a chunk is received from the splitter, and the last peers to arrive, the last peers to be served and the peers that will be served in the burst mode with the highest probability (something that can produce network congestion in the downloading link of these peers).

To solve this, peers could sort their list of peers using, for example, the RTT to each of the peers of their lists. This RTT can be computed using a heartbeat mechanism.

The branch associated with this issue is "peerlist_sorted_by_RTT".

Console/graphical isolation

Currently, simulator.py imports mathplotlib even if the graphical output has not been activated. It would be useful if two different scripts were available (simulate.py and play.py, for example) in order to avoid using it in those systems where matplotlib has not been sucessfully ported.

Problem with Matplotlib version 2.1.0

When version 2.1.0 of Matplotlib is installed, there is a problem with the graphics generator.
The chart generated at the end of the simulation is shown empty.
In version 2.0.0 there is no problem.
This is the detailed bug:

2017-12-31 13:50:37,730 - __main__ - CRITICAL - Critical messages enabled.
2017-12-31 13:50:37,730 - __main__ - ERROR - Error messages enabled.
Traceback (most recent call last):
  File "simulator.py", line 482, in <module>
    fire.Fire(Simulator)
  File "/home/billy/.local/lib/python3.5/site-packages/fire/core.py", line 127, in Fire
    component_trace = _Fire(component, args, context, name)
  File "/home/billy/.local/lib/python3.5/site-packages/fire/core.py", line 366, in _Fire
    component, remaining_args)
  File "/home/billy/.local/lib/python3.5/site-packages/fire/core.py", line 542, in _CallCallable
    result = fn(*varargs, **kwargs)
  File "simulator.py", line 349, in draw
    self.update_team(m[1], m[2], m[3])
  File "simulator.py", line 202, in update_team
    self.team_ax.draw_artist(self.lineMonitors)
  File "/usr/local/lib/python3.5/dist-packages/matplotlib/axes/_base.py", line 2453, in draw_artist
    a.draw(self._cachedRenderer)
  File "/usr/local/lib/python3.5/dist-packages/matplotlib/artist.py", line 55, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/matplotlib/lines.py", line 744, in draw
    self.recache()
  File "/usr/local/lib/python3.5/dist-packages/matplotlib/lines.py", line 657, in recache
    xconv = self.convert_xunits(self._xorig)
  File "/usr/local/lib/python3.5/dist-packages/matplotlib/artist.py", line 191, in convert_xunits
    return ax.xaxis.convert_units(x)
  File "/usr/local/lib/python3.5/dist-packages/matplotlib/axis.py", line 1491, in convert_units
    ret = self.converter.convert(x, self.units, self)
  File "/usr/local/lib/python3.5/dist-packages/matplotlib/category.py", line 49, in convert
    vmap = dict(zip(axis.unit_data.seq, axis.unit_data.locs))
AttributeError: 'NoneType' object has no attribute seq

Peer added to the graph again even after removing from the team

Even after removing from the team, some peers are adding again in the graph while plotting. Console shows that peer have been removed from the graph but it doesn't.

DBS2

DBS2 is a simple extension of DBS where peers can create alternative paths for those chunks that do not arrive from a known origin peer.

When a peer detects a lost chunk, it requests it to one of the peers that is in its list of peers. The selected peer should be chosen at random among those peers that have a chunk debt smaller than a given threshold.

The branch associated with this issue is "dbs2".

Concise representation of a peer end-point for better visualisation.

Currently a peer is represented by it's (ip address, port), but it is too much to show on console. So we need a better way to represent an endpoint for better visualisation.

Run-length encoding for buffer in drawing log file.

Using simple Run-length encoding we could greatly reduce the size of the drawing log file.

Example:
Now: B;P1;CCCCCCCCLLLL
After: B;P1;8C4L

Missing Code in peer_sts.py file

In peer_sts.py file line 19 there is an incomplete for loop.

And on line 23 the definition of function salute_the_load_balancer is missing a :

missing requirements.txt

Using Internet Sockets instead of Unix sockets

Turn AF_UNIX socket into AF_INET socket.

To Move the work of painting graphs to different processes.

Painting the network overlay graphs is heavy. Could be a good idea to move the work of painting each graph to one differnte process? Would be a better idea to replace matplotlib with another way for painting graphs in real time (something faster)?

Peers must transmit the stream even if the player is not connected

At this moment, when a peer is run, it waits for the player before starting the transmission of chunks. It is preferable that peers start receiving and transmitting chunks as soon as they are instantiated.

Provide a argsparser to the simulator

The current -h option for the simulator (https://github.com/P2PSP/simulator/blob/master/src/simulator.py) is not informative. It's better to use argparse. Example: https://github.com/P2PSP/simulator/blob/master/src/peer.py

PyQT/play.py KeyError when update_buffer was called

I got the following KeyError:

File "PyQt/play.py", line 274, in draw
    self.update_buffer(m[1], m[2])
  File "PyQt/play.py", line 175, in update_buffer
    ix = self.buffer_order[sender]
KeyError: '127.0.1.1,47525'
Violación de segmento