mprib/multiwebcam

concurrent frame capture with USB webcams

📷📷📷 MULTIWEBCAM 📷📷📷

Concurrent webcam recording to bootstrap low-cost/early-stage computer vision projects

mwc_sample.mp4

Introduction

I needed a cheap way to record concurrent frames while prototyping a computer vision project (caliscope). Extreme precision was less important than getting something reasonable with a minimal budget. When conscientiously managed, USB webcams controlled via OpenCV can perform surprisingly well at this task. I have spun this functionality off into its own package to create a clear separation of concerns between data capture and data processing, while hopefully creating a simpler package that others might find useful.

If MultiWebCam (MWC) is close to what you need but not quite, please feel free to raise an issue and I'll see if I can incorporate your use case. These are the core functions that are currently implemented:

• Record concurrent frames from multiple webcams
• Record from single webcams to pull single camera calibration video.
• "time-align" frames in real time to understand dropped frame rate
• include frame-by-frame time stamp history to facilitate off-line processing
• easy adjustment of the following parameters:
  • resolution
  • exposure
  • target fps

Quick Start

Basic pip install

You can install MWC into your python environment with pip install multiwebcam and then launch it from the command line with

mwc

Note that this has primarily been tested on Windows 10, infrequently on MacOS, and will not work on Linux as far as I can tell ☹️. If someone is familiar with getting USB cameras working through OpenCV on Linux, I'm all ears.

Editable Install Using Poetry

If you prefer to contribute to MWC or want to install it in editable mode, follow these steps using Poetry:

Clone the Repository:

git clone https://github.com/mprib/multiwebcam.git
cd multiwebcam

Install Poetry:

pip install poetry

Set Up the Environment:

poetry install

By running poetry install, you'll install all dependencies and also set up the multiwebcam package in editable mode. Any changes you make to the code will be reflected in your environment.

Capturing Data

1. Launch MultiWebCam from the command line:

mwc

2. Make sure that the USB cameras you want to use are currently plugged in when you launch the new session.

3. Choose a new project directory through the File menu. MWC will attempt to connect to the cameras currently plugged in and will create a recording_config.toml file in the project directory.

4. From the Mode menu you can select single camera to change camera settings (such as resolution and exposure).

5. On the MultCamera mode you can set the target fps to achieve a desired dropped frame rate

6. Record videos

Checking Against System Clock

To provide a check of the accuracy of the time stamps, you can launch a widget that displays the perf_counter from the system by running from the command line:

mwc clock

Cross checking the frames with the recorded time stamp value can provide a sense of the temporal accuracy of the recording.

multiwebcam's Issues

Autoconnect on session launch

Don't require the user to explicitly connect and disconnect

only subscribe to active stream during single camera mode

Given the rate of display refresh in the single camera mode, the CPU can get overburdened by the background cameras. If they aren't the active stream at the moment, just let them rest...This will be relevant when attempting to record single camera streams...

Using with OPENCV Python

Is there any interface so that I can utilize this library with opencv python to get synchronized image from two USB camera for stereo.

Logger can be closed

pyside clock displays current time

For assessing the accuracy of the synchronization my plan is to have a single clock widget that will display the perfcounter time. This could run concurrent with the recording so that when all of the cameras are pointed back toward the computer monitor they should be able to grab a frame that aligns with the time stamp...

create single cam video recorder object

merge settings into group box on multirecord

The rendered rate should probably be in the settings box...I now there is a risk of confusion.

You should also add a tool tip explaining that parameter

Open workspace directory from menu

Make it easy to check on the contents of what is being recorded

User can reload session

If you make some changes to the config (such as ignoring a camera) you should be able to just reload the session rather than closing out and relaunching...

Resolution drop down needs to reflect current resolution

enable default filepaths to align with calibration

This would mean that single camera recordings will go to:
workspace_dir/calibration/intrinsic

and multiple recordings will got to
workspace_dir/calibration/extrinsic

When disabled, then all recordings go into
workspace_dir/calibration/extrinsic

User can modify rendered rate

Single camera recording button added

This should block the rest of the GUI from getting used....

assign backend via config file

multirecording widget has scroll area

FPS target needs to always reflect current target

Upon launching new session cannot switch to single cam mode

Attempts to switch to a new single cam mode after launching a new project will cause things to freeze. If I close and reload, then everything works as expected. Potentially there is an issue with the setup of the single streams, so start by taking a look there...

Change backend for camera via GUI