fippo/rtcstats-server

Server for https://github.com/fippo/rtcstats

rtcstats-server's Issues

detect micro-glitches

i've had a horrible call experience today (which turned out to be related to a broken battery and running low on that). Basically it glitched very, very often.

This is detectable in getStats. If packetsReceived or bytesReceived for a track stay the same despite a 1s difference in time that is a good indicator.

googrtt needs to be measured for the active connection

currently it is not

undefined in getstats

               "undefined" : "2016-06-03T20:45:52.297Z",

This oddly looks as if the timestamp is set wrong

document export to elasticsearch

@dagingaa just figured out how to do this and I think @ggarber might be interested.

move feature generation to worker process

as noted in #106 (comment) -- it is a good idea to move feature generation (which can take quite a bit of time) to workers

investigate chrome metrics

things like https://www.chromestatus.com/metrics/feature/timeline/popularity/1042

count googAdaptionChanges

they show bandwidth/cpu adaption. Lives on ssrc report currently

googAccelerateRate

https://code.google.com/p/chromium/codesearch#chromium/src/third_party/webrtc/media/base/mediachannel.h&q=accelerate_rate&sq=package:chromium&type=cs&l=646
fraction of data removed sounds bad. I just saw something 29%. Pretty sure this results in bad audio.
I want a mean, max, std dev of this

detect audio PLC kicking in

seems earlier than packet loss. googDecodingPLC

count number of ice failures and restarts

work with spec-stats

we're mostly relying on the google stats, we should be working with spec-stats more. One day...

turn type used

we don't have a feature telling us what turn type is used. Basically a combination of the type pref code in https://github.com/opentok/rtcstats-server/blob/develop/features.js#L26 and the 'figure out connected pair' one in https://github.com/opentok/rtcstats-server/blob/develop/features.js#L1055

receivingvideo10s is broken

https://github.com/opentok/rtcstats-server/blob/develop/features.js#L787-L808
-- this assumes a getStats frequency of once per second. Should instead measure the time.

write peerconnections to different files

currently, all data associated with a session is written to a single file. Splitting this up into one file for session metadata and another file for each peerconnection would allow processing a peerconnection once its done and not when the client closes the page.
Increases realtimeness and hopefully also reduces disk usage.

infinity values

PC_0 FEATURE statsMeanReceivingBitrate => Infinity
PC_0 FEATURE statsMeanSendingBitrate => Infinity

calculate delay between SLD/SRD

and SRD/addIceCandidate

"statsMeanRoundTripTime":null in Firefox

this is null in Firefox, causing issues with saving data

determine whether the remote side is firefox or edge

by looking at the SDP. For Firefox this means looking for SDPARTA, for edge thisisadapterortc.

This will be helpful classifying ice failures.

detect all-zero jitter

to prevent things like http://logs.glob.uno/?c=mozilla%23media#c242496 :-(

measure time between getusermedia and getusermediasuccess / failure

this allows making a guess whether permission has been granted. Arguably, that could be also determined by checking whether device labels pop up in enumerateDevices before getUserMedia.

number of different host candidates in onicecandidate

delta storage

getStats is currently stored as a full report. It would be much more efficient to store the same delta value that is transmitted.

crashes in zlib.js

the major remaining crash is in zlib.js line 623 which is this code:

      var newReq = self._handle.write(flushFlag,
                                      chunk,
                                      inOff,
                                      availInBefore,
                                      self._buffer,
                                      self._offset,
                                      self._chunkSize);

self._handle is null:

zlib.js:623
      var newReq = self._handle.write(flushFlag,
                               ^

TypeError: Cannot read property 'write' of null
    at Zlib.callback (zlib.js:623:32)

It is in the process that handles websockets so I suspect the websocket closes in the middle of things. Sounds like a bug in node-core...

changing dynamodb table names on the fly.

the size of our dynamodb table makes exports to redshift very hard. Also, we re-export the whole DB every time.

What we should do instead is to generate the tablename based on the current data. That assumes that the tables are created in advance but that is doable.

Tables can also be archived to S3 when they are no longer needed for read access.

WDYT @ggarber?

extractTrack should not rely on onaddstream

https://github.com/opentok/rtcstats-server/blob/develop/features.js#L189-L220
relies on onaddstream. This caused a decrease in the number of records for some of our quality metrics.

Action:

1. fix this
2. shake fist at @henbos

security: valid origins

how do we determine that a origin is authorized to send us data?

scale it

the current model is to keep stuff in memory until the client closes the connection. That won't scale...

positive/negative googEchoCancellationDelayMedia

seen it go from pos to neg. Does that mean there is echo?

more ws woes

/home/ubuntu/rtcstats-server/node_modules/ws/lib/Receiver.js:386
    default: srcBuffer.copy(dstBuffer, dstOffset, 0, length); break;
                       ^

RangeError: out of range index
    at RangeError (native)
    at fastCopy (/home/ubuntu/rtcstats-server/node_modules/ws/lib/Receiver.js:386:24)
    at Receiver.add (/home/ubuntu/rtcstats-server/node_modules/ws/lib/Receiver.js:86:3)
    at Socket.realHandler (/home/ubuntu/rtcstats-server/node_modules/ws/lib/WebSocket.js:800:20)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:188:7)
    at readableAddChunk (_stream_readable.js:177:18)
    at Socket.Readable.push (_stream_readable.js:135:10)
    at TCP.onread (net.js:542:20)

seems to be websockets/ws#778 even though that is using ws 1.x whereas we are still stuck on 0.8

location features

quite similar to what Eric did with distance to datacenter... roughly:

var maxmind = require('maxmind');

var cityLookup = maxmind.open('./GeoLite2-City.mmdb');
var city = cityLookup.get('2.150.33.235');
console.log(city.location);
console.log(city.country.iso_code);
console.log(city.continent.code);

which gives us location (lon/lat), county, continent and timezone

detect higher-than-requested framerate

mostly linux bugs I think

dont obfuscate relay addresses in getStats

               "Cand-+zj+Zh1T" : {
                  "id" : "Cand-+zj+Zh1T",
                  "type" : "remotecandidate",
                  "portNumber" : 50144,
                  "priority" : 25108479,
                  "transport" : "udp",
                  "timestamp" : 1465744899781,
                  "candidateType" : "relayed",
                  "ipAddress" : "0.0.0.0"
               },

see #61 for why we don't need to obfuscate an ip we own.

chrome 51 broke stuff

I was looking into the audiosendgoogrttmean feature and it has disappeared in Chrome >= 52. This is because https://bugs.chromium.org/p/webrtc/issues/detail?id=3377 has been fixed.

Those stats still appear but apparently not in the first report. For video this seems to be different?!
This means they're undefined and bad things happen in https://github.com/opentok/rtcstats-server/blob/develop/features.js#L1583-L1584

Solution is probably to filter undefined and then calculate instead of throwing away things if the first entry is undefined

feature: candidate types after ice restart

'cause https://bugzilla.mozilla.org/show_bug.cgi?id=1295552
-- also this will let me detect if credentials expired.

Do not log IPs to make Antonio happy

extract used audio/video codec

from getStats for Chrome, SDP for Firefox

feature for frame(width/height) trashing

defined as max(number of framewidth changes per interval)

googCpuLimitedResolution + googBandwidthLimitedResolution percentage

i want a feature that calculates the percentage of googCpuLimitedResolution and googBandwidthLimitedResolution being true.

retroactively run features

basically as follows:

• select connection ids which don't have that feature in redshift
• fetch data from s3
• run feature
• update dynamodb item

Ability to start/end session outside opening/closing tab

We would love for there to be a way to start/end a session outside of opening and closing the tab. This is for example useful if you have a lot of back to back calls, and don't open a new browser window for every call. And if the user forgets to close the tab, we can end the session programatically and dump it to rtcstats at the logical end of the call, rather than closing the tab.

add a LICENSE.md

@ggarber can you add a LICENSE.md please? MIT?

ice restarts need consideration

I just saw a case where there was no remote relay candidate before the failure. There was one after the ice restart but that was counted as hadremoterelaycandidate => true which lead me to investigate this as "what the heck" instead of shrugging it off as "oh yeah, no remote relay. not gonna work"

setup instructions

are missing. Making a list here.

• what is the primary key of the dynamodb table? 'Item'?
• how is export to redshift done
• what needs to be configured for passport?

detect b=AS setting

inconsistent internal use of .timestamp

sometimes its a unix timestamp, sometimes its a Date object.
The solution is probably to make .timestamp a unix timestamp everywhere and have .time as the equivalent date object.

needs a better plan for ssrc-stat extraction

extracting stats for a ssrc needs a better plan.

Basically:

• extract local streams from addStream, remote streams from onaddstream
• extract audio/video track
• extract ssrc stats for track

no gum calls or peerconnections made

this should be handled -- makes no sense to serialize something to the db then

multistream handling

per discussion with @ggarber it is probably easiest to just make one row per stream/track

minFrameWidth is -1

I saw a minGoogFrameHeightReceived of -1 -- that is bogus obviously and should be filtered.

use presto?

https://prestodb.io/
when @baddn showed me this it seemed very similar to the current workflow. But better.