rangho/webrtc-delay-estimation

Delay estimation logic extracted from WebRTC

My name is RangHo Lee (and no, that's not my real name), an undergraduate student majoring in Computer Science and Engineering and English Literature and Linguistics at Sogang University! I love learning and trying new things, and of course, building stuff.

Do feel free to reach me out via:

🙋 More about me! I...

• 🌐 ...speak Korean and English semi-competently and currently learning German, Japanese and Russian.
• 📝 ...sometimes write articles on rangho.postype.com (in Korean, though).
  • It's not a tech blog; it's more like a trashcan where I stash my gibberish.
• 🎙️ ...like speaking in public about things I am interested in, such as:
  • this stuff about Linux customization (Korean),
  • this stuff about prototype-based object-oriented programming (also Korean),
  • this stuff about WebAssembly in general (English),
  • and this stuff about mixing Rust and Python (also English).
• ✨ Fun Fact: ...used to learn Latin in high school and still in love with the language!

🧰 Languages and tools!

Here are the languages I have experience with!

Languages

Editors

Operating systems

📊 Some statistics!

webrtc-delay-estimation's Issues

WebRTC delay estimator not working on the real time data.

I have successfully calculated the dalay using the wave files but when I try to calculate the delay with real time data then it doesnt give me any delay estimate but only a NoEstimateAvailableError();
Can you tell why this happens?

Unable to compile on Windows

Attempting to compile this project using Visual Studio on Windows fails with syntax errors in logging.cc. Tried both VS2019 CMake project configuration and CMake-generated vcxproj projects.

Since there are large amount of error messages from WinSock2.h, it is probably caused by the order of Windows header files.

Deleting #include <windows.h> from logging.cc was able to reduce the amount of error messages significantly.

Delay estimation result looks wrong

Although the main.cc source code follows the logic of WebRTC's delay estimation unit test, the number of estimated delay still looks very off. Below is the source which main.cc is based on.

// Verifies that the delay estimator produces correct delay for artificially
// delayed signals.
TEST(EchoPathDelayEstimator, DelayEstimation) {
  constexpr size_t kNumRenderChannels = 1;
  constexpr size_t kNumCaptureChannels = 1;
  constexpr int kSampleRateHz = 48000;
  constexpr size_t kNumBands = NumBandsForRate(kSampleRateHz);

  Random random_generator(42U);
  std::vector<std::vector<std::vector<float>>> render(
      kNumBands, std::vector<std::vector<float>>(
                     kNumRenderChannels, std::vector<float>(kBlockSize)));
  std::vector<std::vector<float>> capture(kNumCaptureChannels,
                                          std::vector<float>(kBlockSize));
  ApmDataDumper data_dumper(0);
  constexpr size_t kDownSamplingFactors[] = {2, 4, 8};
  for (auto down_sampling_factor : kDownSamplingFactors) {
    EchoCanceller3Config config;
    config.delay.down_sampling_factor = down_sampling_factor;
    config.delay.num_filters = 10;
    for (size_t delay_samples : {30, 64, 150, 200, 800, 4000}) {
      SCOPED_TRACE(ProduceDebugText(delay_samples, down_sampling_factor));
      std::unique_ptr<RenderDelayBuffer> render_delay_buffer(
          RenderDelayBuffer::Create(config, kSampleRateHz, kNumRenderChannels));
      DelayBuffer<float> signal_delay_buffer(delay_samples);
      EchoPathDelayEstimator estimator(&data_dumper, config,
                                       kNumCaptureChannels);

      absl::optional<DelayEstimate> estimated_delay_samples;
      for (size_t k = 0; k < (500 + (delay_samples) / kBlockSize); ++k) {
        RandomizeSampleVector(&random_generator, render[0][0]);
        signal_delay_buffer.Delay(render[0][0], capture[0]);
        render_delay_buffer->Insert(render);

        if (k == 0) {
          render_delay_buffer->Reset();
        }

        render_delay_buffer->PrepareCaptureProcessing();

        auto estimate = estimator.EstimateDelay(
            render_delay_buffer->GetDownsampledRenderBuffer(), capture);

        if (estimate) {
          estimated_delay_samples = estimate;
        }
      }

      if (estimated_delay_samples) {
        // Allow estimated delay to be off by one sample in the down-sampled
        // domain.
        size_t delay_ds = delay_samples / down_sampling_factor;
        size_t estimated_delay_ds =
            estimated_delay_samples->delay / down_sampling_factor;
        EXPECT_NEAR(delay_ds, estimated_delay_ds, 1);
      } else {
        ADD_FAILURE();
      }
    }
  }
}

Maybe reproducing this unit test helps to pinpoint the issue? Not sure if this unit test even passes in the WebRTC repository in the first place.

Block size is a constant in WebRTC, thus should not be changed

Now, we accept --block as a command line argument. Block size is already defined in WebRTC, so attempting to use a different value other than 64 would cause a critical assertion failure.

webrtc-delay-estimation/src/main.cc

Lines 44 to 45 in bc00b23

	("b,block-size", "Block size to use when recognizing delay.",
	cxxopts::value(block_size)->default_value(default_block_size))

Build fails on Windows for not being able to find libraries

On Windows with Visual Studio 2019, build fails as CMake is unable to find REQUIRED libraries.

Attempt to run the program on Linux crashes with SIGILL

Backtrace looks like this:

#0  webrtc::AlignmentMixer::AlignmentMixer (this=0x5555556cd550, num_channels=1, downmix=false, adaptive_selection=true, activity_power_threshold=10000, prefer_first_two_channels=true)
    at /home/rangho/Documents/Development/RangHo/webrtc-delay-estimation/src/alignment_mixer.cc:54
#1  0x000055555557a3e5 in webrtc::AlignmentMixer::AlignmentMixer (this=0x5555556cd550, num_channels=1, config=...) at /home/rangho/Documents/Development/RangHo/webrtc-delay-estimation/src/alignment_mixer.cc:46
#2  0x000055555559119a in webrtc::(anonymous namespace)::RenderDelayBufferImpl::RenderDelayBufferImpl (this=0x5555556cd220, config=..., sample_rate_hz=16000, num_render_channels=1)
    at /home/rangho/Documents/Development/RangHo/webrtc-delay-estimation/src/render_delay_buffer.cc:160
#3  0x0000555555595841 in webrtc::RenderDelayBuffer::Create (config=..., sample_rate_hz=16000, num_render_channels=1) at /home/rangho/Documents/Development/RangHo/webrtc-delay-estimation/src/render_delay_buffer.cc:520
#4  0x0000555555563452 in main (argc=3, argv=0x7fffffffdfb8) at /home/rangho/Documents/Development/RangHo/webrtc-delay-estimation/src/main.cc:94

Unable to understand platform-dependent assembly instruction seems to be the issue here. Maybe SSE2?