webrtc-for-the-curious/webrtc-for-the-curious

WebRTC for the Curious: Go beyond the APIs

Welcome to WebRTC for the Curious!

What is this book.

Start reading the book directly on GitHub or at webrtcforthecurious.com

When we announced the book to social media we used the following copy

Title: WebRTC for the Curious: Go beyond the APIs

Subject: The WebRTC book that explains everything.
WebRTC is a real-time communication framework that makes it easy
to build real-time interactions for web and mobile devices.

You will learn about the WebRTC specification and how all the
protocols work in depth, not just a tour of the APIs.
The book is completely Open Source and available at
https://webrtcforthecurious.com and
https://github.com/webrtc-for-the-curious/webrtc-for-the-curious

Learn the full details of ICE, SCTP, DTLS, SRTP, and how they work
together to make up the WebRTC stack.

Hear how WebRTC implementers debug issues with
the tools of the trade.

Listen to interviews with the authors of foundational
WebRTC tech! Hear the motivations and design details
that pre-dated WebRTC by 20 years.

Explore the cutting edge of what people are building with
WebRTC. Learn about interesting use cases and how real-world
applications get designed, tested and implemented in production.

Written by developers who have written all of this
from scratch. We learned it the hard way, now we want
to share it with you!

This book is vendor agnostic and multiple Open Source projects
and companies are involved. We would love to have you involved!

Running/Deploying

• Clone this repo and it's submodules git clone --recursive https://github.com/webrtc-for-the-curious/webrtc-for-the-curious.git
• Download the extended version of hugo
• Run hugo server

Contributing

We would love contributions! We are open to suggestions about the content as well. This is a community-owned project, and we want to hear what is important to you. Below is a rough guide to where each chapter stands.

Contributing from your browser (via Codespaces!)

We support Github Codespaces so helping out is as easy as starting up a Codespace from this repo.

Once started, click "Run Live Docs Server" in the statusbar to start the Hugo server. Note that Hugo's autoreload doesn't trigger a browser refresh from Codespaces yet due to missing proxying of WebSockets traffic. Reload a page manually to rerender it.

License

This book is available under the CC0 license. The authors have waived all their copyright and related rights in their works to the fullest extent allowed by law. You May use this work however you want and no attribution is required.

The only intention of this book is to make the world a better place. WebRTC is a wonderful technology but is difficult to use. This book is vendor agnostic, and we have tried to remove any conflicts of interest.

webrtc-for-the-curious's Issues

Write about RTP Payload Headers

Write how RTP header is different the RTP Payload header

Add HTML Generate to GithubPages

I bought webrtcforthecurious.com for the next 3 years, will point to GitHub pages.

Personally I prefer hugo, but don't feel strongly.

Typo in FAQ about maximum number of DataChannels

In the FAQ section of the book, the question How many DataChannels can I have? is answered with 65536 channels as stream identifier has 16 bits. You can close and open a new one at any time.

However, in chapter 7 Data Communication, it is stated that the maximum number of DataChannels is 65,534, and upon further research, I confirmed that it is the correct number. Hence, the answer in the FAQ needs to be updated slightly.

PS. Thank you so much for taking the time to write such an invaluable resource on the ins and outs of WebRTC. It has helped me a lot and I am looking forward to reading the missing sections, especially the part on renegotiation.

Feedback from tumetab1

• When you mention a protocol can clarify on what ISO/Stack it’s? I have no idea if ICE is application protocols or connection and that really would me be understand the first chapter.
• Just add something like “ICE (at the same stack level as TCP/TLS/HTTP”.

WebRTC transport 101

Would be great to have few words talking about "under the hood" details of various WebRTC systems. For example, what happens in browser (on both ends) before onTrack callback is called ? Is that different when those two are connected through SFU ?

Does onstatechange actually mean onconnectionstatechange?

https://github.com/webrtc-for-the-curious/webrtc-for-the-curious/blob/master/content/docs/01-what-why-and-how.md#onstatechange

In How does WebRTC (the API) work section in the above URL, it mentions onstatechange.
Although threre's onconnectionstatechange property in RTCPeerConnection, it seems onstatechange property doesn't exist.
https://developer.mozilla.org/en-US/docs/Web/API/RTCPeerConnection/onconnectionstatechange

Does the document want to say onconnectionstatechange instead of onstatechange?

[Propose] Add JSEP reference.

Hello, members.

WebRTC only uses some SDP keys
says.

Not all key values defined by the Session Description Protocol are used by WebRTC.

I would like to know, what document defines those usages. It seems that the document is RFC 8829: JavaScript Session Establishment Protocol (JSEP), Correct?

If so, what do you think to add a description?

Not all key values defined by the Session Description Protocol are used by WebRTC. It is defined in RFC 8829

If it is OK, I'll create a PR. Thanks.

diff --git a/content/docs/02-signaling.md b/content/docs/02-signaling.md
index ce8c55e..20306ad 100644
--- a/content/docs/02-signaling.md
+++ b/content/docs/02-signaling.md
@@ -39,7 +39,7 @@ a=second-value
 You have two lines. Each with the key `a`. The first line has the value `my-sdp-value`, the second line has the value `second-value`.

 ### WebRTC only uses some SDP keys
-Not all key values defined by the Session Description Protocol are used by WebRTC. The following seven keys are the only ones you need to understand right now:
+Not all key values defined by the Session Description Protocol are used by WebRTC. It is defined in [RFC 8829](https://datatracker.ietf.org/doc/html/rfc8829). The following seven keys are the only ones you need to understand right now:

 * `v` - Version, should be equal to `0`.
 * `o` - Origin, contains a unique ID useful for renegotiations.

ePub Version

Maybe we can make an ePub copy of readers like iBooks?

Chapter on congestion control

Not sure if this is going to be part of any chapter, but I think we should have a dedicated chapter on this.

Hamburger menu invisible in dark mode on mobile

Chrome on Android here.

The sidebar is invisible, and so the book seems to consist of just the Introduction landing page.

And yes, please use pandoc to provide EPUB downloads, as mentioned in another issue.

Header extensions

Hi,

Firstly thank you for the initiative, i have a request, could there be a few paragraphs on RTP header extensions and stuff like RTX.

Also if you could squeeze in a chapter on building native webrtc it would be a big help, it was especially confusing to me when i was just starting out.

Thanks.

accepting PRs?

;)

PDF/EPUB Version

I wanted to read this book on my reader, but then realized there was no PDF/EPUB version. Is it planned?

TLS is used to secure communication over TCP, not just HTTP

Generate contributors page

I think we should generate a contributors page based on the git history. Either using something like https://github.com/marketplace/actions/contribute-list or https://github.com/marketplace/actions/auto-add-contributors, or just generate it from the git history.

If needed, we could add a .mailmap to merge or fix any missing names.

Broken path to image

https://webrtcforthecurious.com/docs/06-media-communication/ is supposed to embed https://webrtcforthecurious.com/images/06-frame-types.png but instead links to https://webrtcforthecurious.com/docs/images/06-frame-types.png

I would have provided a pull request but since the image appears correctly in https://github.com/webrtc-for-the-curious/webrtc-for-the-curious/blob/master/content/docs/06-media-communication.md, I figured this may need more contextual awareness than I have

Suggestion about candidate description in connection chapter

In this phase Candidate Gathering

These candidates are also distributed via the Session Description

When I use WebRTC API in browser, the ice candidates are sending by signaling method like websocket instead of session description.
The create offer / answer is separated from ice candidate exchanging.

Based on MDN
https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API/Connectivity#the_entire_exchange_in_a_complicated_diagram

Feedback from jbaudanza

This is great! I just spent the weekend researching WebRTC for a language learning project I'm working on. I have a few questions about SFUs. Maybe someone here can help.

1. SFUs seem to be the clear winner for multi-party calls. Do they provide any advantage for 1-on-1 calls over peer'd connections? It seems like using an SFU in this case would double the number of connections, and hence double the connection issues.

2. Do clients communicate with SFUs by establishing a normal RTCPeerConnection like they would with another peer? Or is there some other video sharing protocol for SFUs that I don't know about?

Make sure pandoc handles `<sub>`

I used <sub> in #157
for "T_observe"

Suggestions regarding the "connecting" page

Hi,

Please find below some random topics that would be great to detail for the connection page:

• STUN requests are sent even when not using a STUN or TURN servers. For example if one of a 2 peers communication, if one is not behind a NAT then no need a STUN server but STUN binding requests will still be made.
• As a test how one could configure its router to be one of the 4 type of NAT (full-cone etc...) , i.e. what option to search in the router configuration. Are there simple tools we can use to determine what type of NAT we are behind ?
• The page does not say that TURN is an extension of STUN, so it is not something orthogonal.
• A server like coturn can do both STUN and TURN server. The page could say that one could deploy just one server to do both.
• Maybe also a section on SFU, and what are the implication for STUN and TURN. It is not a STUN or TURN server but if one SFU is on the cloud with a public IP then no need for a STUN server. But still need a TURN server for the peers behind a symmetric NAT or router blocking udp entirely (so requiring to use tcp)
• Would be great to provide a typical example where one will be likely behind a symmetric NAT. For example what kind of company or places (hotels?) are likely to use a symmetric NAT.

Many thanks!

epub can not be read on iOS Books app

only the summary is visible; turning pages to the first actual content page or clicking on the 1st chapter ("Who is this book for") shows this:

the error is:

This page contains the following errors:

error on line 11 at column 92: Attribute data-number redefined

below is a rendering of the page up to the first error

indeed I opened the epub in sigil and:

are you using pandoc 2.8 or 2.9 ? it seems the bug is fixed in later releases: jgm/pandoc#5986

Index layout seems wrong.

01-what-why-and-how.md for example:
Currently:

• h1 What is WebRTC
  • h2 Why should I learn WebRTC
  • ...

Expected:

• h1 What-why-and-how
  • h2 What is WebRTC
  • h2 Why should I learn WebRTC
  • ...

Add a link to the WFTC Matrix channel

Sean and I made a channel on Matrix for people to openly discuss WebRTC questions - inspired by the book

The room is here - which links to https://matrix.to/#/!jQIZWJSvVrccJkAcJQ:commcon.xyz?via=matrix.org&via=nimblea.pe&via=bordum.dk

Where should this live? README? As part of the book in a chapter that talks about discuss-webrtc etc etc?

Bug: [Security 101]-[Nonce]-[By using a nonce you can get different input]

original:

A nonce is an additional input to a cipher. This is used so that you can get different output from the cipher, even if you are encrypting the same message multiple times.

If you encrypt the same message 10 times, the cipher will give you the same ciphertext 10 times. By using a nonce you can get different input, while still using the same key. It is important you use a different nonce for each message! Otherwise, it negates much of the value.

expectation:

By using a nonce you can get different output.

Website is down webrtcforthecurious.com

I am trying to access website https://webrtcforthecurious.com/
But it shows:

Feedback from a friend

Got this feedback from a friend, creating here will come back later

 - "Because WebRTC has these attributes you get these advantages over traditional Client/Server technology." -> what attributes? I'm assuming advantages are the ones listed below?

- "The process described above is ICE. Another protocol that pre-dates WebRTC." -> where did you describe the process? you could probably just take this line out instead and say that "WebRTC uses ICE to establish its peer to peer connections"

- "Determining NAT Type #" -> is the STUN server able to determine our NAT mapping at all? or is that only determined via checking for connectivity with the other peer. 

- "LIFETIME" - > I would also add you can destroy by setting lifetime to 0 through a refresh request

- "This means it should send a STUN Binding Request to the TURN Server." -> What happens in the case the TURN server doesnt support STUN?

- "One TURN Allocations for Communication #" -> having 'UDP Client' represent the peer is a bit confusing..since couldn't TURN also be using a UDP client? In these diagrams I usually like the word peer and TURN client. Should there also be an arrow to show some sort of processing or traffic going between server and relayed address on the TURN server?

[Feedback] Not in the same network

Thank you for creating great documents. It's very useful.

https://webrtcforthecurious.com/docs/03-connecting/#not-in-the-same-network

This section says.

For the hosts in the same network it is very easy to connect. Communication between 192.168.0.1 -> 192.168.0.2 is easy to do! These two hosts can connect to each other without any outside help.

But this image doesn't show 192.168.0.2

Current

Correct image?

Best regards.

Topics proposals and some questions

Thank you for the book!

Would be great if you could mention, at least briefly:

1. Where does Video/Audio processing happen? Ex. If I want to analyze video in real-time or apply certain filters, where would this component sit in the flow and how this affects everything (not sure if p2p works with this approach...)?
2. Would be great to have a single terminology index
3. System and flow diagrams would help a lot, ex. for standard cases, like a video group chat.

I would also appreciate if you could describe what WebRTC agent is, technically.

Thanks!!
D

Chapter Ownership

These aren’t hard assignments! Maybe just a good start to find out people’s availability/interest. Totally understand if people don’t actually have the bandwidth. A rough draft is also better then nothing!

• Data Communication - @enobufs
• Media Communication - @zhuker
• Applied WebRTC - @leewardbound

@tQsW what interests you the most? If you like networking that chapter still needs massive improvements :)

But w/e people feel passionate about is what they should do. That is how you get quality IMO.

SEO/Outreach

We need to do some basic SEO so this shows up in a search engine.

I plan on putting this in the Pion README.md! My goal is that we get this book mature enough that the Open Source community can take shared ownership of it.

With that in mind I would love to get in contact with.

• GStreamer
• mediasoup
• Janus
• Jitsi
• Mozilla
• Google
• Kurento
• SimplePeer
• Muaz Khan
• WebRTC Hacks
• wrtc
• aiortc

Would love to see if helping users understand stuff deeper is actually helpful. If it is what they need to convince them to get involved.

Brand Guide

• Font
• Color Scheme

Glossary page

I think we should have dedicated glossary to explain some of the terms in the book

[Feedback] Media Descriptions in a Session Description

Thank you for creating this document.

https://webrtcforthecurious.com/docs/02-signaling/#media-descriptions-in-a-session-description

m=<media> <port> <proto> <fmt> ... description may help to find fmt place.
It was a little hard to find fmt place for me. 😄

For example,

• m - Media Description, Syntax: m=<media> <port> <proto> <fmt> ... described in detail below

Best regards

Chapter on Data Communication has duplicate answer

Please see the screenshot，the answer to Question: How do I know if I am sending too much？ is incorrect.

The FAQ section doesn't look right in the PDF

Additional tags get included:

{{<details “Why does WebRTC use UDP?”>}} NAT Traversal requires UDP.
Without NAT Traversal establishing a P2P connection wouldn’t be possible.
UDP doesn’t provide “guaranteed delivery” like TCP, so WebRTC provides it at
the user level.
See [Connecting]({{< ref “03-connecting” >}}) for more info. {{
}}

Illustrations

• Front Cover
• Each Chapter Start

All technical diagrams are going to user mermaid.

I am trying to access my Work Microsoft365 account however several of the email accounts that I linked to my main Work account have came up showing not there as well as my phone number associated with it. Please help me recover a information as simply and QuickTime as possible

Edit page footer link is broken

Clicking the edit button gives a 404, content/ is duplicated. The link at the end of Applied WebRTC for example:

https://github.com/webrtc-for-the-curious/webrtc-for-the-curious/edit/master/content/content/docs/08-applied-webrtc.md

Should be:

https://github.com/webrtc-for-the-curious/webrtc-for-the-curious/edit/master/content/docs/08-applied-webrtc.md

Elevator Pitch

I am going to post on Twitter/HN/Reddit/discuss-webrtc (and anywhere else you suggest) Thursday.
My paternity leave is done this week so I want to get this out there while I still have time!

This is the elevator pitch. Please feel free to edit the issue directly or suggest things :)
Depending on where I post i will shorten/edit things.

Title: WebRTC for the Curious: Go beyond the APIs

Subject: The WebRTC book that explains everything.
WebRTC is a real-time communication framework that makes it easy
to build real-time interactions for web and mobile devices.

You will learn about the WebRTC specification and how all the
protocols work in depth, not just a tour of the APIs.
The book is completely Open Source and available at
https://webrtcforthecurious.com and
https://github.com/webrtc-for-the-curious/webrtc-for-the-curious

Learn the full details of ICE, SCTP, DTLS, SRTP, and how they work
together to make up the WebRTC stack.

Hear how WebRTC implementers debug issues with
the tools of the trade.

Listen to interviews with the authors of foundational
WebRTC tech! Hear the motivations and design details
that pre-dated WebRTC by 20 years.

Explore the cutting edge of what people are building with
WebRTC. Learn about interesting use cases and how real-world
applications get designed, tested and implemented in production.

Written by developers who have written all of this
from scratch. We learned it the hard way, now we want
to share it with you!

This book is vendor agnostic and multiple Open Source projects
and companies are involved. We would love to have you involved!

Metrics

This could be a little advanced, but since it is part of WebRTC, we should also include information about the stats and how each of them could be useful. More of a generic view I would say without going into the details of every stat webrtc provides

case inconsistency: addicecandidate versus addIceCandidate

TIny thing... but in section "How does WebRTC (the API) work", lowercase addicecandidate and mixed case addIceCandidate are both used. Should be mixed case

Book Pitch/Ethos

These are the guiding principles/ideas I have been working off of. Feel free to add more and discuss :)

The book is designed for multiple reads

We should write the book so that you can get value even if you don’t understand everything. Each chapter should be self contained and ramp up in difficulty. If you don’t understand ICE the RTP section should still be useful.

• What is being solved (Packet loss)
• How is it solved (NACK)
• How do they actually work (Deep dive)

Target Audiences

• Web Developers who don’t even know what WebRTC solves
• Someone who builds with WebRTC but wants to know more
• Established Developer who needs help debugging
• WebRTC implementor who needs a rough idea on one piece.

Communicate our love of RTC

This is hard to do, but I hope we can genuinely get people excited. I think showing history and use cases will help a lot.

Vendor Agnostic

It will be important for getting others involved. I hope we can make something that other WebRTC projects will feel comfortable sharing with their community.

Interested in Adding external links to

Are you interested in adding any links to external resources for things like this?

Potential Updates

• NAT Traversal
• Available in Browsers - I don't know how up to date this site it
• Open Standards - ICE, STUN, RTP, etc. I'd add links to RFCs here
• Multiple Implementation - pion/webrtc, - References to prominent libraries in different languages (May go out of date over time, but would be helpful -

Roadmap to v1.0

My goal is to have the book complete by Summer. At that time I am going to look at finding a publisher or self publishing. The goals is to put a physical copy in peoples hands. These are all the things that need to happen before then.

What is the goal of the book?

The goal of the book is to to teach developers how to learn about WebRTC. The topic is too large for one book to describe fully. The book aims to introduce all the concepts need for developers to go learn on their own. We then should recommend books that focus on the single topic. The book also will not describe any WebRTC software in particular. This brings down the amount of developers that will read it. If we only focus on one WebRTC software it will miss its full potential.

Do the chapters cover all the high level concepts?

• What is WebRTC. A bundling of existing protocols, how they work together (chapter 1
• Why does WebRTC need signalling, how does it work (chapter 2)
• Why does WebRTC need ICE, how does it work (chapter 3)
• Why does WebRTC need security, how does it work (chapter 4)
• Network Fundamental, precursor to Media/Data Transport (chapter 5)
• Why does WebRTC need RTP, how does it work (Chapter 6)
• Why does WebRTC need SCTP, how does it work (Chapter 7)
• What can you build with WebRTC (Chapter 8)
• Debugging WebRTC, tools and thinking like a WebRTC dev (chapter 9)
• History of WebRTC, how WebRTC is a bundling of existing protocols (chapter 10)
• FAQ, common questions redirects people to existing chapters (chapter 11)

Yes I believe so. There are no other topics I would like to cover.

Is Chapter 1 Complete?

Yes, needs review.

Is Chapter 2 Complete?

Yes, needs review.

• Simulcast
• Transceiver Matching
• Renegotiation

Is Chapter 3 Complete?

Yes, needs review.

• ICE isn't fully documented

Is Chapter 4 Complete?

Yes, needs review.

I would also like to recommend Serious Cryptography: A Practical Introduction to Modern Encryption for users who wish to learn more.

Is Chapter 5 Complete?

Yes, needs review.

Is Chapter 6 Complete?

Yes, needs review

• RTCP wire protocol examples
• RTP Header extensions need explained

Is Chapter 7 Complete?

Yes, needs review.

• More state machine questions
  • How does a user message get sent?
  • TSN and retransmission
  • Congestion avoidance
  • Selective ACK
  • Fast retransmission/recovery
  • Partial Reliability

Is Chapter 8 Complete?

No needs extensive work/outline here

Is Chapter 9 Complete?

No needs extensive work/outline here

Is Chapter 10 Complete?

We still need an interviewee for

• ICE, STUN TURN
• SCTP
• DTLS/SRTP
• WebRTC

Is Chapter 11 Complete?

No needs extensive work/outline here

Layout and formats

• Replace Mermaid Graphs with png files
• Cover image epub
• Cover image pdf
• Generate mobi-file
• Generate translated docs

Copyediting

• Redraw illustrations using draw.io

Publishing

I would like to publish and let the publisher keep more/proceeds go to charity. It is more important to me that the book stays open and gets in as many hands as possible.

• No Starch Press said they want something more applied.
• Pragmatic Bookshelf liked the book, but was apprehensive about the idea of it being open. Asked to come back when complete.
• Packt Publishing haven't approached yet.
• Send an email to Fabien Sanglard and ask his advice/opinions.
• Contact Bill Kennedy, for advice/opinions

Graphs in PDF not rendering

Example from page 8

Amazing resource btw! Even without the graphs ;)
Got the pdf from https://webrtcforthecurious.com/docs/webrtc-for-the-curious.pdf

"Generating a Bandwidth Estimate" glosses over important elements of bandwidth estimation

Regarding: "Generating a Bandwidth Estimate" in section 6.

While packet loss is one indicator of network congestion, Google Congestion Control uses another measure that may have a much larger impact on advertised available bitrates.

Without needing to go too far into the weeds, I think it's worth calling out that GCC's usage of a Kalman filter is to make accurate estimations of inter-packet arrival delays. GCC measures the differences in arrival times of each frame, and from that variation infers whether or not the network is congested. Increases in inter-packet arrival times (after controlling for the variation in frame size in bytes) suggest that the network is increasingly queuing up packets on a congested link. If that inferred congestion increases sufficiently (sufficiently-large time increases over a long-enough measurement window), GCC will reduce the advertised incoming bitrate.

If that is too much detail to add for this section, that's alright. It may suffice to add a superficial description about how GCC attempts to estimate network congestion and may reduce incoming bitrate advertisements even in the absence of (appreciable) packet loss.

Feedback about the book

Hi,
First of all, It's a great book and is useful for someone without prior knowledge of webRTC like me.
I am looking forward to furthermore content.

In Connecting After the definition of NAT and Mapping strategies, It became too theoretical and a little hard to understand. Some existing examples helped a lot. I think adding a few more examples would make it more concrete and understandable.

I think explanations about scenarios that can happen for existing peer connections, and their effect on them might also be useful, like the need for ICE Trickling and renegotiation.

Add section about RFC 8888 - RTCP Feedback for Congestion Control

Add a section about "RTCP Feedback for Congestion Control" detailed in RFC 8888 to 06-media-communication.md.

Incomprehensible sentence, please explain

This is the part I am really interested in and I can't wrap my head around this sentence. Please help!

Look here https://webrtcforthecurious.com/docs/03-connecting/#turn in the second paragraph
"TURN uses a dedicated server. This server acts as a proxy for a client. The client connects to a TURN Server and creates an Allocation. By creating an Allocation a client gets a temporary IP/Port/Protocol that can send into to get traffic back to the client. This new listener is known as the Relayed Transport Address. Think of it as a forwarding address, you give this out so others can send you traffic via TURN! For each peer you give the Relay Transport Address to, you must create a Permission to allow communication with you."

The bold part seems like an invalid sentence and I can't understand what the author meant.

Operational pointers re STUN/TURN/ICE

A couple of points:

1. If the end-points can deduce peer reflexive addresses and do Trickle ICE, there is no need to run a STUN server (of course STUN protocol is needed). It may shave off some time during ICE procedure.
2. Some local networks may block many ports, it is preferable that TURN server listens at port 443 using turns scheme.