m-bain/webvid

Large-scale text-video dataset. 10 million captioned short videos.

webvid's Issues

Size of 2.5M subset

Hi,
Just wondering what the size of the 2.5M subset is once downloaded?

How to resume downloading dataset from the broken page?

such as from page 003151_003200

thank you.

8 videos missing

Hi, thanks for the amazing work! I download the videos from CSV files, but I find 8 dataset URLs are 403 Forbidden, Can you help me with this problem?

https://ak.picdn.net/shutterstock/videos/4422923/preview/stock-footage-stockholm-may-amusement-park-from-canal-in-stockholm-on-may-in-stockholm-sweden.mp4
https://ak.picdn.net/shutterstock/videos/3741062/preview/stock-footage-beautiful-ocean-coastline-landscapes-of-volcano-and-lava-field-with-animals-at-rancho-palos-verdes.mp4
https://ak.picdn.net/shutterstock/videos/10681772/preview/stock-footage-united-states-s-animation-of-mop-spreading-bacteria-animation-of-nurses-nurse-disappears.mp4
https://ak.picdn.net/shutterstock/videos/6315860/preview/stock-footage-panoramic-view-to-the-ocean-at-sunrise-time-in-cancun-mexico.mp4
https://ak.picdn.net/shutterstock/videos/3742148/preview/stock-footage-model-posing-for-photographer-at-volcanic-lava-field-at-ocean-side-or-palos-verdes-ca.mp4
https://ak.picdn.net/shutterstock/videos/3742151/preview/stock-footage-bikini-photo-shoot-at-stony-pint-park-ca.mp4
https://ak.picdn.net/shutterstock/videos/2416907/preview/stock-footage-paper-latterns-under-the-tent-full-hd-video.mp4
https://ak.picdn.net/shutterstock/videos/7412620/preview/stock-footage-black-and-white-cat-in-the-grass-defending-herself.mp4

About some of the video durations being incorrectly labeled.

Seven of the WebVid-2.5M training sets are longer than one hour, but when I found the specific videos to watch, I found that the actual length of the videos was not consistent with the label.
For example:

25677788, Sailboats on the Horizon, 90507090, 501 _090550 PT01H02M37S, https://ak.picdn.net/shutterstock/videos/25677788/preview/stock-footage-sailboat s-on-the-horizon.mp4

The real duration of the video is 11S, which is not consistent with the label. Several other tags have the same problem.

results_2M_val.csv download is closed，how to get it

(base) [wangxi@v100-4 webvid]$ wget -nc http://www.robots.ox.ac.uk/~maxbain/webvid/results_2M_val.csv
--2024-02-27 10:49:36-- http://www.robots.ox.ac.uk/~maxbain/webvid/results_2M_val.csv
Resolving www.robots.ox.ac.uk (www.robots.ox.ac.uk)... 129.67.94.2
Connecting to www.robots.ox.ac.uk (www.robots.ox.ac.uk)|129.67.94.2|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://www.robots.ox.ac.uk/~maxbain/webvid/results_2M_val.csv [following]
--2024-02-27 10:49:43-- https://www.robots.ox.ac.uk/~maxbain/webvid/results_2M_val.csv
Connecting to www.robots.ox.ac.uk (www.robots.ox.ac.uk)|129.67.94.2|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-02-27 10:49:44 ERROR 404: Not Found.

(base) [wangxi@v100-4 webvid]$

No audio information?

Hey,
I saw some videos from webvid and found that these videos had no audio.
Does all of the webvid have no audio information?

Question about the page_dir.

Hi @m-bain
I have downloaded the webvid10M_val by your ‘results_10M_val.csv’ and 'download.py'. I would like to ask some questions about the page_dir.

1. How do you produce the page_dir for each videoid?
   Just like videoid%10?

2. What is the function of page_dir?
   To avoid the limitation of the maximum number of files per directory?

3. What file order should I use for pre-training?
   Follow the page_dir from small to big?
   Follow the results_10M_train.csv from the first row to the last row?

Thank you:)

Questions about the dataset storage.

Hi @m-bain

1. How large are the WebVid2.5M and WebVid10M respectively? If it is convenient, please show video-storage and text-storage.
2. Have you finished pretraining FIT (or other model) on WebVid10M? If yes, how long does it take?

Need a NSFW-filter?

About the correlation between 2.5m subset and 10m dataset.

Hi @m-bain, I'm wondering whether the csv file of 2.5m subset is DIRECTLY sampled from the complete 10m csv file, meaning that the videoid and page_dir should be identical for the same video in both csv files (10m and 2.5m version) without other ordering issues?

some records of this two csv show NaN in ‘page_dir’ field

does it right? or does something bad happen when I download these CSV files?

Original video links

Hi,

Thanks for the great work.
Since webvid has no audios for most of the videos, I'm wondering if it's possible to release original vidoe links so that we can download on our own to get audios?
Thank you very much!

Time to download 2.5M videos

Hi,

How much ETA does it take to download 2.5M videos using your script?

Thank you.

dataset url is 403 Forbidden

dataset url is 403 Forbidden, do you have any idea to solve it, thx!

Webvid-2.5M dataset download

We are researching a project, and the Webvid-2.5M dataset is very important to us. We found that the link is no longer working. Could you please provide us with a new link?

All video/ gifs seems to have the "shutterstock" watermark at the same location.

Is there a download limit imposed by the video source website?

I run this commad:

video2dataset --url_list="results_2M_train.csv" \
        --input_format="csv" \
        --output-format="webdataset" \
	--output_folder="test" \
        --url_col="contentUrl" \
        --caption_col="name" \
        --save_additional_columns='[videoid,page_idx,page_dir,duration]' \
        --enable_wandb=False \
	--config=default \

At first, the download process went smoothly, and I successfully downloaded 96 .tar files, totaling about 200GB. Then, error messages started appearing.

HTTPSConnectionPool(host='ak.picdn.net', port=443): Read timed out.

I switched to a different computer and attempted to download again, but encountered the same errors after downloading around 200GB.
Could this be due to a download limit imposed by the video source website?
How should I resolve this issue?
@m-bain

Script to download the videos

Just saw you released a download script. FWIW, this is what I used to download the 2M version, just wanted to share it. I think it's simpler (it uses csvkit and parallel) but maybe it has fewer features:

csvcut -c videoid,contentUrl results_2M_val.csv \
  | sed 1d \
  | parallel --resume -v --joblog val.log --bar -j 8 'video_id=$(echo {} | cut -d , -f 1) && url=$(echo {} | cut -d , -f 2) && extension=${url##*.} && wget --no-clobber -qO "videos/val/$video_id.$extension" $url'

It downloads with 8 jobs in parallel (the flag -j 8).

Inquiry about Video Caption Generation in the WebVid Dataset

Hello，Thanks for sharing the data. Could you please tell me the method used to generate the video captions within the WebVid dataset. Please provide some insights into whether the captions were:

1. Generated by a deep learning model, and if so, which model was used?
2. Manually written by human annotators, and if so, what guidelines were they provided with?
3. Scraped from the internet, and if so, what was the process for ensuring the relevance and quality of the captions?

Issues while installing mpi4py on linux based cluster

Hi, I am trying to run a pip install mpi4py but it throws off with an error. The exact error is here:

Defaulting to user installation because normal site-packages is not writeable
Collecting mpi4py
  Downloading mpi4py-3.1.4.tar.gz (2.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.5/2.5 MB 18.3 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: mpi4py
  Building wheel for mpi4py (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for mpi4py (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [156 lines of output]
      running bdist_wheel
      running build
      running build_src
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-39
      creating build/lib.linux-x86_64-cpython-39/mpi4py
      copying src/mpi4py/run.py -> build/lib.linux-x86_64-cpython-39/mpi4py
      copying src/mpi4py/__init__.py -> build/lib.linux-x86_64-cpython-39/mpi4py
      copying src/mpi4py/bench.py -> build/lib.linux-x86_64-cpython-39/mpi4py
      copying src/mpi4py/__main__.py -> build/lib.linux-x86_64-cpython-39/mpi4py
      creating build/lib.linux-x86_64-cpython-39/mpi4py/futures
      copying src/mpi4py/futures/server.py -> build/lib.linux-x86_64-cpython-39/mpi4py/futures
      copying src/mpi4py/futures/_base.py -> build/lib.linux-x86_64-cpython-39/mpi4py/futures
      copying src/mpi4py/futures/_lib.py -> build/lib.linux-x86_64-cpython-39/mpi4py/futures
      copying src/mpi4py/futures/__init__.py -> build/lib.linux-x86_64-cpython-39/mpi4py/futures
      copying src/mpi4py/futures/_core.py -> build/lib.linux-x86_64-cpython-39/mpi4py/futures
      copying src/mpi4py/futures/aplus.py -> build/lib.linux-x86_64-cpython-39/mpi4py/futures
      copying src/mpi4py/futures/pool.py -> build/lib.linux-x86_64-cpython-39/mpi4py/futures
      copying src/mpi4py/futures/__main__.py -> build/lib.linux-x86_64-cpython-39/mpi4py/futures
      creating build/lib.linux-x86_64-cpython-39/mpi4py/util
      copying src/mpi4py/util/__init__.py -> build/lib.linux-x86_64-cpython-39/mpi4py/util
      copying src/mpi4py/util/pkl5.py -> build/lib.linux-x86_64-cpython-39/mpi4py/util
      copying src/mpi4py/util/dtlib.py -> build/lib.linux-x86_64-cpython-39/mpi4py/util
      copying src/mpi4py/py.typed -> build/lib.linux-x86_64-cpython-39/mpi4py
      copying src/mpi4py/dl.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py
      copying src/mpi4py/run.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py
      copying src/mpi4py/__main__.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py
      copying src/mpi4py/__init__.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py
      copying src/mpi4py/bench.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py
      copying src/mpi4py/MPI.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py
      copying src/mpi4py/MPI.pxd -> build/lib.linux-x86_64-cpython-39/mpi4py
      copying src/mpi4py/__init__.pxd -> build/lib.linux-x86_64-cpython-39/mpi4py
      copying src/mpi4py/libmpi.pxd -> build/lib.linux-x86_64-cpython-39/mpi4py
      creating build/lib.linux-x86_64-cpython-39/mpi4py/include
      creating build/lib.linux-x86_64-cpython-39/mpi4py/include/mpi4py
      copying src/mpi4py/include/mpi4py/mpi4py.h -> build/lib.linux-x86_64-cpython-39/mpi4py/include/mpi4py
      copying src/mpi4py/include/mpi4py/mpi4py.MPI.h -> build/lib.linux-x86_64-cpython-39/mpi4py/include/mpi4py
      copying src/mpi4py/include/mpi4py/mpi4py.MPI_api.h -> build/lib.linux-x86_64-cpython-39/mpi4py/include/mpi4py
      copying src/mpi4py/include/mpi4py/mpi4py.i -> build/lib.linux-x86_64-cpython-39/mpi4py/include/mpi4py
      copying src/mpi4py/include/mpi4py/mpi.pxi -> build/lib.linux-x86_64-cpython-39/mpi4py/include/mpi4py
      copying src/mpi4py/futures/_lib.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py/futures
      copying src/mpi4py/futures/pool.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py/futures
      copying src/mpi4py/futures/server.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py/futures
      copying src/mpi4py/futures/__main__.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py/futures
      copying src/mpi4py/futures/aplus.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py/futures
      copying src/mpi4py/futures/__init__.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py/futures
      copying src/mpi4py/futures/_core.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py/futures
      copying src/mpi4py/util/pkl5.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py/util
      copying src/mpi4py/util/__init__.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py/util
      copying src/mpi4py/util/dtlib.pyi -> build/lib.linux-x86_64-cpython-39/mpi4py/util
      running build_clib
      MPI configuration: [mpi] from 'mpi.cfg'
      MPI C compiler:    /share/apps/anaconda3-2022.05/bin/mpicc
      MPI C++ compiler:  /share/apps/anaconda3-2022.05/bin/mpicxx
      MPI F compiler:    /share/apps/anaconda3-2022.05/bin/mpifort
      MPI F90 compiler:  /share/apps/anaconda3-2022.05/bin/mpif90
      MPI F77 compiler:  /share/apps/anaconda3-2022.05/bin/mpif77
      checking for library 'lmpe' ...
      /share/apps/anaconda3-2022.05/bin/mpicc -pthread -B /share/apps/anaconda3-2022.05/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -I/share/apps/anaconda3-2022.05/include -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -fPIC -c _configtest.c -o _configtest.o
      /share/apps/anaconda3-2022.05/bin/mpicc: line 301: x86_64-conda_cos6-linux-gnu-cc: command not found
      failure.
      removing: _configtest.c _configtest.o
      building 'mpe' dylib library
      creating build/temp.linux-x86_64-cpython-39
      creating build/temp.linux-x86_64-cpython-39/src
      creating build/temp.linux-x86_64-cpython-39/src/lib-pmpi
      /share/apps/anaconda3-2022.05/bin/mpicc -pthread -B /share/apps/anaconda3-2022.05/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -I/share/apps/anaconda3-2022.05/include -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -fPIC -c src/lib-pmpi/mpe.c -o build/temp.linux-x86_64-cpython-39/src/lib-pmpi/mpe.o
      /share/apps/anaconda3-2022.05/bin/mpicc: line 301: x86_64-conda_cos6-linux-gnu-cc: command not found
      warning: build_clib: command '/share/apps/anaconda3-2022.05/bin/mpicc' failed with exit code 127

      warning: build_clib: building optional library "mpe" failed

      checking for library 'vt-mpi' ...
      /share/apps/anaconda3-2022.05/bin/mpicc -pthread -B /share/apps/anaconda3-2022.05/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -I/share/apps/anaconda3-2022.05/include -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -fPIC -c _configtest.c -o _configtest.o
      /share/apps/anaconda3-2022.05/bin/mpicc: line 301: x86_64-conda_cos6-linux-gnu-cc: command not found
      failure.
      removing: _configtest.c _configtest.o
      checking for library 'vt.mpi' ...
      /share/apps/anaconda3-2022.05/bin/mpicc -pthread -B /share/apps/anaconda3-2022.05/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -I/share/apps/anaconda3-2022.05/include -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -fPIC -c _configtest.c -o _configtest.o
      /share/apps/anaconda3-2022.05/bin/mpicc: line 301: x86_64-conda_cos6-linux-gnu-cc: command not found
      failure.
      removing: _configtest.c _configtest.o
      building 'vt' dylib library
      /share/apps/anaconda3-2022.05/bin/mpicc -pthread -B /share/apps/anaconda3-2022.05/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -I/share/apps/anaconda3-2022.05/include -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -fPIC -c src/lib-pmpi/vt.c -o build/temp.linux-x86_64-cpython-39/src/lib-pmpi/vt.o
      /share/apps/anaconda3-2022.05/bin/mpicc: line 301: x86_64-conda_cos6-linux-gnu-cc: command not found
      warning: build_clib: command '/share/apps/anaconda3-2022.05/bin/mpicc' failed with exit code 127

      warning: build_clib: building optional library "vt" failed

      checking for library 'vt-mpi' ...
      /share/apps/anaconda3-2022.05/bin/mpicc -pthread -B /share/apps/anaconda3-2022.05/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -I/share/apps/anaconda3-2022.05/include -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -fPIC -c _configtest.c -o _configtest.o
      /share/apps/anaconda3-2022.05/bin/mpicc: line 301: x86_64-conda_cos6-linux-gnu-cc: command not found
      failure.
      removing: _configtest.c _configtest.o
      checking for library 'vt.mpi' ...
      /share/apps/anaconda3-2022.05/bin/mpicc -pthread -B /share/apps/anaconda3-2022.05/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -I/share/apps/anaconda3-2022.05/include -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -fPIC -c _configtest.c -o _configtest.o
      /share/apps/anaconda3-2022.05/bin/mpicc: line 301: x86_64-conda_cos6-linux-gnu-cc: command not found
      failure.
      removing: _configtest.c _configtest.o
      building 'vt-mpi' dylib library
      /share/apps/anaconda3-2022.05/bin/mpicc -pthread -B /share/apps/anaconda3-2022.05/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -I/share/apps/anaconda3-2022.05/include -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -fPIC -c src/lib-pmpi/vt-mpi.c -o build/temp.linux-x86_64-cpython-39/src/lib-pmpi/vt-mpi.o
      /share/apps/anaconda3-2022.05/bin/mpicc: line 301: x86_64-conda_cos6-linux-gnu-cc: command not found
      warning: build_clib: command '/share/apps/anaconda3-2022.05/bin/mpicc' failed with exit code 127

      warning: build_clib: building optional library "vt-mpi" failed

      checking for library 'vt-hyb' ...
      /share/apps/anaconda3-2022.05/bin/mpicc -pthread -B /share/apps/anaconda3-2022.05/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -I/share/apps/anaconda3-2022.05/include -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -fPIC -c _configtest.c -o _configtest.o
      /share/apps/anaconda3-2022.05/bin/mpicc: line 301: x86_64-conda_cos6-linux-gnu-cc: command not found
      failure.
      removing: _configtest.c _configtest.o
      checking for library 'vt.ompi' ...
      /share/apps/anaconda3-2022.05/bin/mpicc -pthread -B /share/apps/anaconda3-2022.05/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -I/share/apps/anaconda3-2022.05/include -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -fPIC -c _configtest.c -o _configtest.o
      /share/apps/anaconda3-2022.05/bin/mpicc: line 301: x86_64-conda_cos6-linux-gnu-cc: command not found
      failure.
      removing: _configtest.c _configtest.o
      building 'vt-hyb' dylib library
      /share/apps/anaconda3-2022.05/bin/mpicc -pthread -B /share/apps/anaconda3-2022.05/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -I/share/apps/anaconda3-2022.05/include -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -fPIC -c src/lib-pmpi/vt-hyb.c -o build/temp.linux-x86_64-cpython-39/src/lib-pmpi/vt-hyb.o
      /share/apps/anaconda3-2022.05/bin/mpicc: line 301: x86_64-conda_cos6-linux-gnu-cc: command not found
      warning: build_clib: command '/share/apps/anaconda3-2022.05/bin/mpicc' failed with exit code 127

      warning: build_clib: building optional library "vt-hyb" failed

      running build_ext
      MPI configuration: [mpi] from 'mpi.cfg'
      MPI C compiler:    /share/apps/anaconda3-2022.05/bin/mpicc
      MPI C++ compiler:  /share/apps/anaconda3-2022.05/bin/mpicxx
      MPI F compiler:    /share/apps/anaconda3-2022.05/bin/mpifort
      MPI F90 compiler:  /share/apps/anaconda3-2022.05/bin/mpif90
      MPI F77 compiler:  /share/apps/anaconda3-2022.05/bin/mpif77
      checking for dlopen() availability ...
      checking for header 'dlfcn.h' ...
      gcc -pthread -B /share/apps/anaconda3-2022.05/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -I/share/apps/anaconda3-2022.05/include -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -fPIC -I/share/apps/anaconda3-2022.05/include/python3.9 -c _configtest.c -o _configtest.o
      success!
      removing: _configtest.c _configtest.o
      success!
      checking for library 'dl' ...
      gcc -pthread -B /share/apps/anaconda3-2022.05/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -I/share/apps/anaconda3-2022.05/include -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -fPIC -I/share/apps/anaconda3-2022.05/include/python3.9 -c _configtest.c -o _configtest.o
      gcc -pthread -B /share/apps/anaconda3-2022.05/compiler_compat _configtest.o -Lbuild/temp.linux-x86_64-cpython-39 -ldl -o _configtest
      success!
      removing: _configtest.c _configtest.o _configtest
      checking for function 'dlopen' ...
      gcc -pthread -B /share/apps/anaconda3-2022.05/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -I/share/apps/anaconda3-2022.05/include -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -fPIC -I/share/apps/anaconda3-2022.05/include/python3.9 -c _configtest.c -o _configtest.o
      gcc -pthread -B /share/apps/anaconda3-2022.05/compiler_compat _configtest.o -Lbuild/temp.linux-x86_64-cpython-39 -ldl -o _configtest
      success!
      removing: _configtest.c _configtest.o _configtest
      building 'mpi4py.dl' extension
      gcc -pthread -B /share/apps/anaconda3-2022.05/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -I/share/apps/anaconda3-2022.05/include -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -fPIC -DHAVE_DLFCN_H=1 -DHAVE_DLOPEN=1 -I/share/apps/anaconda3-2022.05/include/python3.9 -c src/dynload.c -o build/temp.linux-x86_64-cpython-39/src/dynload.o
      gcc -pthread -B /share/apps/anaconda3-2022.05/compiler_compat -shared -Wl,-rpath,/share/apps/anaconda3-2022.05/lib -Wl,-rpath-link,/share/apps/anaconda3-2022.05/lib -L/share/apps/anaconda3-2022.05/lib -L/share/apps/anaconda3-2022.05/lib -Wl,-rpath,/share/apps/anaconda3-2022.05/lib -Wl,-rpath-link,/share/apps/anaconda3-2022.05/lib -L/share/apps/anaconda3-2022.05/lib build/temp.linux-x86_64-cpython-39/src/dynload.o -Lbuild/temp.linux-x86_64-cpython-39 -ldl -o build/lib.linux-x86_64-cpython-39/mpi4py/dl.cpython-39-x86_64-linux-gnu.so
      checking for MPI compile and link ...
      /share/apps/anaconda3-2022.05/bin/mpicc -pthread -B /share/apps/anaconda3-2022.05/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -I/share/apps/anaconda3-2022.05/include -fPIC -O2 -isystem /share/apps/anaconda3-2022.05/include -fPIC -I/share/apps/anaconda3-2022.05/include/python3.9 -c _configtest.c -o _configtest.o
      /share/apps/anaconda3-2022.05/bin/mpicc: line 301: x86_64-conda_cos6-linux-gnu-cc: command not found
      failure.
      removing: _configtest.c _configtest.o
      error: Cannot compile MPI programs. Check your configuration!!!
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for mpi4py
Failed to build mpi4py
ERROR: Could not build wheels for mpi4py, which is required to install pyproject.toml-based projects

This however works on conda, but even if I can see the package on conda list, it doesn't show up when I try importing it from python. Is there a fix for this? Is this a python version issue? I'm using python 3.10.

May I know the total frames of the dataset please?

Hi, I want to know the total frames of the dataset, could you please give me any info about it ?