roc-streaming / roc-toolkit Goto Github PK
View Code? Open in Web Editor NEWReal-time audio streaming over the network.
Home Page: https://roc-streaming.org
License: Mozilla Public License 2.0
Real-time audio streaming over the network.
Home Page: https://roc-streaming.org
License: Mozilla Public License 2.0
It seems that in current implementation there is a case when we're unnecessary dropping received packet(s).
Assume sender generates three packets:
+----+----+----+
| P1 | P2 | P3 |
+----+----+----+
Then they're reordered by interleaver or network:
+----+----+----+
| P3 | P1 | P2 |
+----+----+----+
and P1
and P2
are delayed a bit:
+----+-------+----+----+
| P3 | delay | P1 | P2 |
+----+-------+----+----+
When Depacketizer
wants to read P1
from fec::Reader
, only P3
is already received:
+----+-------+
| P3 | delay |
+----+-------+
Thus, fec::Reader
considers that P1
and P2
are lost and returns P3
. Depacketizer
notes that P1
and P2
are lost. Depacketizer
renders zeros instead of P1
.
Then, P1
and P2
are received:
+----+----+
| P1 | P2 |
+----+----+
After this, Depacketizer
is going to render samples for P2
, previously marked as lost.
Here is the problem:
P2
is received and it's time to extract samples from P2
Depacketizer
renders zeros instead of P2
, because it already got P3
from fec::Reader
and thinks that P2
is lostfec::Reader
drops P2
because it never returns late packetsHowever, technically nothing prevents us from rendering P2
in this situation.
Note that with fec::Reader
, the situation when queue is empty or almost empty is not rare in practice, because network latency jitter value may be close to aim_queue_size - fec_block_size
(7 packets currently).
Is this a problem?
Currently, latency can be configured at runtime for Delayer
, but not for FreqEstimator
. Thus, using non-default latency will break it.
RFC 3550 requires conforming implementations to provide RTCP support. In fact, roc
can function without it, though RTCP provides some benefits:
RTCP consists of the following message types:
SR
(sender report): statistics sent from sender to receiver;RR
(receiver report): statistics and feedback send from receiver to sender;SDES
(source description items): sender identification sent from sender to receiver;BYE
: session termination, initiated by sender or receiver;APP
: arbitrary application-specific data.See alsa_play_tuned.cpp
for full list of ALSA parameters that may affect latency.
See also #44 and #46.
-m32
compiler/linker option and pass it to 3rd-parties.ubuntu:nodep
image).We could use help2man
to generate man pages from --help
.
Implement tests currently marked as IGNORE_TEST
and TODO
.
Opus is a lossy streaming audio codec.
Specs:
Adding Opus support introduces a few challenges:
Our fec::Decoder
is packet-format-independent and works before decoding audio packet payload. Thus, it theoretically may be used in conjunction with Opus decoder, if it makes sense.
--with-3rdparty
)--with-targets
)Current implementation unconditionally uses IPv4:
We can add support for IPv6, activated if it's available at build time and if user provided an IPv6 address.
Currently, output latency is added to session latency. But is this really required?
Check build on various distros, including Debian and Ubuntu. See also #30.
roc-streaming/roc-pulse#23 is a good start for gathering requirements for public API.
E.g. when multiple GCC versions are installed, it should be possible to select specific version.
Determine alignment requirements for OpenFEC and check that buffers are properly aligned.
When alignment requirements are not met, OpenFEC encoder segfaults. (Misaligned packets were fixed in 1977737).
Resampler frame size is configured at compile time. Does it indirectly depend on other parameters, like packet size and player frame size?
If so, using non-default values for those parameters may break resampler.
Currently, output latency is handled directly in Server
. We should move it to new pipeline component, DelayedWriter
.
roc-recv
: add --oneshot
option; if it's set, exit when first client disconnectsroc-send
: fix EOF handling when sending filesroc-send
: produce readable error message when sox can't open input fileImplement some sort of session negotiation.
At a minimum, we need to be able to transfer session parameters from the sender to the receiver. The parameters should include rtpmap (dynamic payload types) and fssi (FEC-scheme specific information).
Why we need this:
The session description should be in the SDP format because this is required by various RTP-related RFCs, e.g. FECFRAME or Opus for RTP.
As for the control protocol, we can use RTSP/1.0 or SIP.
We should also add a session control port to the API, command-line tools, and PulseAudio modules.
Test sender and receiver.
Splitter
and fec::Encoder
fec::Decoder
IPacketReader
instead if single IAudioPacketReader
IAudioPacketReader
interfaceIAudioPacket
in Streamer
Please :)
Example:
pactl list sources | grep Name: | grep monitor
Name: alsa_output.pci-0000_01_00.1.hdmi-stereo.monitor
Name: alsa_output.pci-0000_00_14.2.analog-stereo.monitor
roc-send -vv -t pulseaudio -i alsa_output.pci-0000_01_00.1.hdmi-stereo.monitor \
-s 127.0.0.1:10001 -r 127.0.0.1:10002
Three components are aware of sample rate:
sndio::Reader
sndio::Writer
audio::TimedWriter
, used in pipeline::Server
and pipeline::Client
What we need:
--rate
option for roc-send
and roc-recv
IPacket
Splitter
Watchdog
See also #51.
Move output latency from SampleBufferQueue
to Server
.
SampleBufferQueue
output_latency / samples_per_tick
zero buffers before first tick in serverSee also #44.
First, we need https://travis-ci.org/ for various linux distros and mac os.
Docker:
Linux distros:
Platforms:
Compilers:
Variants:
--with-3rdparty=all
Later, we can also use travis for MacOS and http://www.appveyor.com/ for windows builds.
Add roc-resample tool:
In current implementation, the following encoder/decoder parameters are configured at compile time:
They are:
ROC_CONFIG_DEFAULT_FEC_BLOCK_DATA_PACKETS
ROC_CONFIG_DEFAULT_FEC_BLOCK_REDUNDANT_PACKETS
ROC_CONFIG_DEFAULT_PACKET_SIZE
Moreover, if server and client are built with different parameters, server may behave incorrectly.
What we need instead:
ROC_CONFIG_
values just as defaults;ServerConfig
and ClientConfig
in roc_pipeline
;Our integration tests with random packet loss of 1% fail from time to time. I guess it's because sometimes, there are to much lost packets per block.
I think we should use a fixed number of lost packets per block in such tests.
Currently, there are only separate unit tests for client and server.
I've found two alternative groups of RFCs:
FEC Framework (FECFRAME)
Supports RTP and externally defined FEC schemes.
RFC 6816: LDPC FEC Scheme for FEC Framework
LDPC-Staircase from OpenFEC implements this scheme.
RFC 6816 inherits Section 6: FEC Scheme Specification and Section 5.7: PRNG from this RFC.
There is also Raptor(Q) support for FEC Framework:
Generic FEC
Finally, these RFCs are mentioned by both FEC Framework and Generic FEC:
audio::IRenderer
audio::Renderer
into pipeline::Session
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.