spiideo/soccersegcal

Soccer pitch segmentation and camera calibration in two steps. Step 1, pixelwise segmentation of an broacast image of a soccer game into six different clases defined by the line markings. Step 2, a differential-rendering optimizer that tries to estimate camera parameters from such segementations.Trained on SoccerNet.

Soccer Pitch Segmentation and Camera Calibration

Official implementation of Spiideo's contribution to the 2023 SoccerNet Camera Calibration challange.

It includes a modified version of the SoccerNet Camera Calibration Development Kit in sncalib.

Install

Install pytorch3d following it's installation instaructions, for example

python -mpip install --no-cache-dir pytorch3d -f https://dl.fbaipublicfiles.com/pytorch3d/packaging/wheels/py38_cu113_pyt1120/download.html

Install remaining requirements and setup python to run the modules from the checked out source

python -mpip install -r requirements.txt
python setup.py develop --user

Data

Download the SoccerNet into data/SoccerNet/calibration-2023

from SoccerNet.Downloader import SoccerNetDownloader as SNdl
soccerNetDownloader = SNdl(LocalDirectory="data/SoccerNet/calibration-2023")
soccerNetDownloader.downloadDataTask(task="calibration-2023", split=["train", "valid", "test", "challenge"])

Run the dataloader to display the images and generated segmentations:

python soccersegcal/dataloader.py

Train Segmentation

Train the segmentation model (add --help to se availible options):

python soccersegcal/train.py

To monitor training progress, compare different runs and get hold of the resulting checkpoint.ckpt:

mlflow ui

The checkpoint can also be found by digging through the mlruns dir.

Estimate Cameras

To use the trained segmentation model to estimate camera parameters for the first two samples (index 0 and 1) in the validation set while visualizing the optimization:

python soccersegcal/estimate_cameras.py -c path/to/segmentation/checkpoint.ckpt -i [0,1] -s

To estimate all the cameras in the test set without visualisation (faster):

python soccersegcal/estimate_cameras.py -c path/to/segmentation/checkpoint.ckpt -p test

To se other options:

python soccersegcal/estimate_cameras.py --help

The estimated cameras will be saved in the cams_out directory. To run the SoccerNet evaluation on them:

python sncalib/evaluate_camera.py -s data/SoccerNet/calibration-2023/ --split test -p cams_out/

Pretrained weights

Pretrained weights can be downloaded from the table below. It also lists hyperparameters with non-default values.

Hyperparameters	Combined Metric	Accuracy@5	Completeness
epochs=27	0.53	52.95	99.96	snapshot.ckpt

soccersegcal's Issues

Model weights and hyperparameters

Hello,
I'm asking if you are planning to upload final weights of the learned model. It would be very helpful to me and definitely to other people trying out this model later on.
Also, could you please elaborate on hyperparameters of your best model? Which learning rate did you choose, how many epochs did the model require to reach the final result or what batch size was optimal for you.

Thank you!