freemocap/skellycam

The camera back-end for the `freemocap` project - or - An easy and efficient way to connect to one or more cameras and record synchronized videos💀📸

The FreeMoCap Project

A free-and-open-source, hardware-and-software-agnostic, minimal-cost, research-grade, motion capture system and platform for decentralized scientific research, education, and training

2022-03-09_FreeMoCap_Clips_SHORT_compressed.mp4

--

QUICKSTART

0. Create a a Python 3.9 through 3.11 environment (python3.11 recommended)¶

1. Install software via pip:

pip install freemocap

2. Launch the GUI by entering the command:

freemocap

3. A GUI should pop up that looks like this:

4. Have fun! It might break! Work in Progress lol

5. Join the Discord and let us know how it went!

---

Install/run from source code (i.e. the code in this repo)

Open an Anaconda-enabled command prompt (or your preferred method of environmnet management) and enter the following commands:

1. Create a Python environment (Recommended version is python3.11)

conda create -n freemocap-env python=3.11

2. Activate that newly created environment

conda activate freemocap-env

3. Clone the repository

git clone https://github.com/freemocap/freemocap

4. Navigate into the newly cloned/downloaded freemocap folder

cd freemocap

5. Install the package via the pyproject.toml file

pip install -e .

6. Launch the GUI (via the freemocap.__main__.py entry point)

python -m freemocap

A GUI should pop up!

---

Documentation

Our documenation is hosted at: https://freemocap.github.io/documentation/index_md.html

That site is built using writerside from this repository: https://github.com/freemocap/documentation

---

Contribution Guidelines

Please read our contribution doc: CONTRIBUTING.md

Related

Maintainers

• Jon Matthis
• Endurance Idehen

License

This project is licensed under the APGL License - see the LICENSE file for details.

If the AGPL does not work for your needs, we are happy to discuss terms to license this software to you with a different agreement at a price point that increases exponentially as you move spiritually away from the AGPL

skellycam's Issues

Method of synchronisation

Just want to check, as it doesn't specifically mention it in the docs: Is it the audio stream being used to synchronise the recordings? Currently I am recording with one camera that doesn't have audio, but I am still able to record videos - I guess that should throw an error if it is based on the audio stream?

shared memory

Move `camera_group` to a process to fix queue filling

[TO DO] - make default camera `show` method close/killable

currently if you run skellycam/examples/example1_single_camera_connection.py it results in a cv2.imshow window that can't be closed

we should:

• change the default Camera().show method to be killable with ESC
• alter the window title to something like f"Camera {camera_id} - Press ESC to close"

[BUG] - One of the camera processes will sometimes fail to start

I tracked it down to the actual cv2.VideoCapture part - something will fail with one of the cameras and the process will hang (or maybe die)?

(optionally) Draw camera name and stats on top of image to save space in GUI

The creation of a matplotlib gui inside the qt gui is causing freemocap to fail on a macbook

More details are given in this issue in the frremocap repo: (freemocap issue)[https://github.com/freemocap/freemocap/issues/353#issue-1639834016]

I'm making a duplicate issue here because this is likely where the change is going to happen.

Implement "Shared Memory" strategy

Replace use of queues to share image data from camera process to shared_memory solution similar to the one in this article -

https://medium.com/analytics-vidhya/using-numpy-efficiently-between-processes-1bee17dcb01

https://github.com/bml1g12/benchmarking_multiprocessing_np_array_comms

Add 'recording name tag' to video controller group box

Older Logitech Reading at Reduced Frame Rate

Summary

I began using skellycam with a three camera setup prior to working on a synchronizer class. I'm using two different types of webcams:

• One old Logitech HD 720p (Camera 2)
• Two more recent EMEET HD 1080P (Cameras 0 and 1)

Additionally, I'm running:

• Windows 10
• CPU: i5

Based on the timestamp diagnostic plots, the older Logitech is reading at a reduced frame rate (not 30 fps...closer to half that).

Impact on Synchronizer

This manifests in about half the frames dropping in the synchronized bundles. I wanted to be explicit that this is behavior that I'm seeing associated with my recent PR (which doesn't look great for the PR), but inspection of all the individual frame_times and diagnostic plots appears to confirm to me that this rooted in the frame_payloads coming out of the camera group.

Next Steps for Me

I'm going to see if I can get faster frame rates reading from this camera on calicam (I know I"m getting drops at higher resolutions, but not sure if it's 50%). If not then I guess it's just a hardware limitation for one cheap webcam. I just wanted to make sure I documented this before it got lost in the shuffle and I'll delete if I can duplicate the limitation on my own code.