tlancon/barbellcv

Track your barbell using computer vision for velocity-based insights in your training.

|||---barbell-cv---|||

Get started with velocity-based training with nothing but a laptop, a webcam, and a high contrast marker on your barbell.

Quick Start

1. Download this repository as a .zip file, then unzip it where you like.

2. Go to that directory using CMD (Windows) or the terminal (Mac/Linux.

3. Make sure the correct dependencies are installed using pip:

   python -m pip install -r requirements.txt
   

4. Run the program:

   python main.pyw
   

5. Preview your webcam using the "Preview" button, and rotate it if needed using the adjacent dropdown. Press Enter to escape the preview.

6. Select the color of your barbell marker interactively.

   • Press the "Select Color" button.
   • Drag your mouse over the marker in the popup (make sure to move the marker around and select it under varying light conditions and angles).
   • Press Enter when the marker is tracked satisfactorily.

7. Select the exercise you want to do.

   • Add exercises that you find missing to the /resources/lifts.json file.

8. Input the weight for the set in lbs or kgs.

9. Press "Log Set" and wait for the webcam preview to show before lifting.

10. After lifting, press the Enter key to complete the set.

11. The results for the set are shown.

    

12. Optionally reclassify phases of lifts for each rep.

    • For example, three reps will likely be found for a single clean and jerk. Individually label the concentric phases of the clean, then the front squat, then the jerk separately using the dropdown in row 1 of the table.
    • Also mark false positive reps as FALSE, failed reps as FAILED, or incorrectly detected reps for which the entire ROM was not found as PARTIAL.
    • Below is an example of a set of front squat triples where the unracking of the bar was detected as rep 1. The lifter was able to reclassify this false positive rep as FALSE, erasing that incorrect "rep" from the set.

    

    

Instructions

More in-depth instructions coming soon after testing or available on request.

Limitations

For the video analysis to work correctly on Windows 10 you may need to install the basic K-Lite Codec Pack.

As I test this setup more I will learn more about its accuracy and what the limitations are, then update this section. Suffice it to say that all results should be taken with a grain of salt until verified.

If you install this program and use it for your training, I'd love to hear your feedback. For any bugs or suggestions please open an issue here.

barbellcv's Issues

peak velocity, average velocity not logged in rep_stats in db

Add Help menu item that opens simple docs

Colorize motion plot based on time or rep number

Change to scatterplot?

Use minimum Y ROM to filter false peaks

Make optional in find_peaks function

Make popup UI to customize pass/fail criteria for each lift type

Currently you have to know the keys for set_stats and manually edit the JSON

Make instructions for use in readme

Make splash screen to show program is loading

To find cameras, barbellcv must activate them. And there is a sleep function inside the camera initiation function. So it takes a while to load! Display this to the user.

Make setting to control pass/fail criteria of each exercise

Lifter may want to monitor different aspects of each lift to determine whether it was good/bad or pass/fail

For example, I would like to indicate that a snatch fails if it's below a certain peak velocity, but I want to measure squats by the time of concentric motion

This needs to react to rep reassignment when #8 is implemented

Add pass/fail, pf_metric, and pf_condition to rep table in DB

Move columns to rows in log interface

Move build_filepaths into utilities.py

Requires logging set/rep data to database so filename isn't required to store exercise name and weight

Add Y/V vs T plot to log interface

Add ability to flag false or failed reps in log interface

Log actual N reps

Currently number of reps includes false reps

Add record in DB for the actual number of reps

Peaks "chopped" horizontally

Due to heavy median filtering of original Y signal with a kernel of 7

Auto-adjust Y position of labels for each rep in timeline plot

Show analysis propress bar popup

Add ability to open old sets

Export DB to Excel/csv

Keep set/rep history in SQLite db

Move settings from json to ini

Use configparser from std lib

Record-save-analyze instead of record-analyze-save

Recording the video up front, then reopening it to analyze it may yield better results and higher frame rates. Currently the video is analyzed AS its recorded and for faster movements (the very thing I want to track) some frames are missed.

Using a workflow to save the video, open it, then analyze it also opens up a more flexible UI. Instead of needing to record to perform an analysis, any video from any source can be opened and analyzed or reanalyzed. If someone wants to use their phone to upload directly to a cloud storage (i.e. OneDrive), then open the lift from there, this should enable that.

Add action in menu to find cameras

Currently if a camera is attached after barbellcv starts, the lifter must restart the software to be able to access it. Need an action in the UI to send a signal to the camera finding method to circumvent this

Move save settings to menu action

Add ability to label exercises in log interface correctly

Each rep in the table should be able to be correctly labeled AFTER the lift

Useful for situations where one rep is comprised of many individual movements

Ex.1:
The user logs a set of 2 clean and jerks. The rep finder will most likely report 6 separate reps:

1. Clean
2. Front squat
3. Jerk
4. Clean
5. Front squat
6. Jerk

The history database should remember this set as clean and jerks, but each rep should be remembered as the individual movement they are so that when the lifter reflects on their history they aren't comparing velocities/ROMS of intermixed movements.

Another example would be snatches, where the snatch and overhead squat are detected separately.

Somehow the database needs to respond to two different queries:

1. The lifter wants to see their history of jerks, but only when NOT performed as part of a clean and jerk or complex
2. The lifter wants to see a history of the jerk portion of clean and jerks

Round to 1 decimal in lb/kg spinboxes

Add ability to cancel rep during capture

log_set reacts to Escape key

Add horizontal lines to bar path plot showing peak power Y

Refactor into smaller files

Ideally:

barbellcv.py has logic to run the app and parse commands from the command line
log.py contains the UI for logging
analyze.py contains the UI for reviewing/editing lift history
helpers.py has self-contained functions that can be used throughout

Force 1:1 motion plot aspect ratio

Add action in menu to clear saved videos

Disable auto unit conversion by pyqtgraph

What do the plots look like if this is done? Worked before in initial testing

Tracing on video is upside down

Highlight each rep in timeline plot

Update requirements to explicitly include PyQtWebEngine

Since 45234p5u3245872458342 years ago when I made this, PyQt5 has broken out the web widgets I used for rendering the HTML documentation to its own package in pip (PyQtWebEngine). Add this to requirements.txt.

Also, update installation instructions in README.md to run the software using python main.pyw instead of python main.py.

Massive peak velocities reported erroneously

Sometimes peak velocities are reported upwards of 5 - 8 m/s.

Currently the bar marker detection find the largest contour within the color range in every frame it can, and plots that contour as part of the bar path.

It's not uncommon for the marker to "blur" between some frames due to video compression, and for the blurred color to be outside of the valid color range that the user calibrates. There is also no way to expect the user to be able to calibrate this. Additionally, this normally occurs during a critical time in the concentric phase of the rep.

On top of that, it's also not uncommon for noisy pixels to present themselves for individual frames.

When these coincide, you end up with a marker that disappears during the concentric phase - when measuring velocity is most important - and for a tiny false "marker" to show up somewhere randomly in the frame.

Since barbellcv accepts the largest contour it finds, that means that the bar path will jump from the marker to that noise then back to the bar in a very short time, making it look like the bar traveled very far in a short period of time.

Ideas to get around this:

1. Use a "calibration" period when the set is first logged. A progress bar shows the lifter that they should leave the barbell relatively still for the first ~20 - 30 frames (1 - 2 seconds). After this, the marker radius is known, and any contours outside of a range of this radius are rejected, skipping frames where the marker is not found.
2. After the calibration period described above, simply record a video with no live tracking. Then post-process the video with a tracking algorithm that finds a marker in every successive frame based on similarity metrics instead of color ranges. This delays feedback to the user for some time (maybe up to 10 - 15 seconds for sets of 8+).

Make save button for settings

Need to be able to save without exiting

Fix tab order in interface