llsourcell/3d_pose_estimation

This is the code for "Machine Vision" By Siraj Raval on Youtube

Overview

This is the code for this video on Youtube by Siraj Raval.

3d-pose-baseline

This is the code for the paper

Julieta Martinez, Rayat Hossain, Javier Romero, James J. Little. A simple yet effective baseline for 3d human pose estimation. In ICCV, 2017. https://arxiv.org/pdf/1705.03098.pdf.

The code in this repository was mostly written by Julieta Martinez, Rayat Hossain and Javier Romero.

We provide a strong baseline for 3d human pose estimation that also sheds light on the challenges of current approaches. Our model is lightweight and we strive to make our code transparent, compact, and easy-to-understand.

Dependencies

• h5py
• tensorflow 1.0 or later

First of all

1. Watch our video: https://youtu.be/Hmi3Pd9x1BE
2. Clone this repository and get the data. We provide the Human3.6M dataset in 3d points, camera parameters to produce ground truth 2d detections, and Stacked Hourglass detections.

git clone https://github.com/una-dinosauria/3d-pose-baseline.git
cd 3d-pose-baseline
mkdir data
cd data
wget https://www.dropbox.com/s/e35qv3n6zlkouki/h36m.zip
unzip h36m.zip
rm h36m.zip
cd ..

Quick demo

For a quick demo, you can train for one epoch and visualize the results. To train, run

python src/predict_3dpose.py --camera_frame --residual --batch_norm --dropout 0.5 --max_norm --evaluateActionWise --use_sh --epochs 1

This should take about <5 minutes to complete on a GTX 1080, and give you around 75 mm of error on the test set.

Now, to visualize the results, simply run

python src/predict_3dpose.py --camera_frame --residual --batch_norm --dropout 0.5 --max_norm --evaluateActionWise --use_sh --epochs 1 --sample --load 24371

This will produce a visualization similar to this:

Training

To train a model with clean 2d detections, run:

python src/predict_3dpose.py --camera_frame --residual --batch_norm --dropout 0.5 --max_norm --evaluateActionWise

This corresponds to Table 2, bottom row. Ours (GT detections) (MA)

To train on Stacked Hourglass detections, run

python src/predict_3dpose.py --camera_frame --residual --batch_norm --dropout 0.5 --max_norm --evaluateActionWise --use_sh

This corresponds to Table 2, next-to-last row. Ours (SH detections) (MA)

On a GTX 1080 GPU, this takes <8 ms for forward+backward computation, and <6 ms for forward-only computation per batch of 64.

Pre-trained model

We also provide a model pre-trained on Stacked-Hourglass detections, available through google drive

To test the model, decompress the file at the top level of this project, and call

python src/predict_3dpose.py --camera_frame --residual --batch_norm --dropout 0.5 --max_norm --evaluateActionWise --use_sh --epochs 200 --sample --load 4874200

Citing

If you use our code, please cite our work

@inproceedings{martinez_2017_3dbaseline,
  title={A simple yet effective baseline for 3d human pose estimation},
  author={Martinez, Julieta and Hossain, Rayat and Romero, Javier and Little, James J.},
  booktitle={ICCV},
  year={2017}
}

License

MIT

Credits

Credits for this code go to una-dinosauria. I've merely created a wrapper to get people started.

3d_pose_estimation's Issues

`python src/predict_3dpose.py --camera_frame --residual --batch_norm --dropout 0.5 --max_norm --evaluateActionWise --use_sh --epochs 1`

got this error while running

W tensorflow/core/framework/op_kernel.cc:1401] OP_REQUIRES failed at save_restore_v2_ops.cc:109 : Not found: Failed to create a NewWriteableFile: experiments\All\dropout_0.5\epochs_200\lr_0.001\residual\depth_2\linear_size1024\batch_size_64\no_procrustes\maxnorm\batch_normalization\not_stacked_hourglass\predict_17\checkpoint-24371.data-00000-of-00001.tempstate14109936691770814557 : The system cannot find the path specified.
; No such process
Saving the model... Traceback (most recent call last):
File "C:\Users\Garima\Anaconda3\envs\fastai-3.7\lib\site-packages\tensorflow\python\client\session.py", line 1334, in _do_call
return fn(*args)
File "C:\Users\Garima\Anaconda3\envs\fastai-3.7\lib\site-packages\tensorflow\python\client\session.py", line 1319, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "C:\Users\Garima\Anaconda3\envs\fastai-3.7\lib\site-packages\tensorflow\python\client\session.py", line 1407, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.NotFoundError: Failed to create a NewWriteableFile: experiments\All\dropout_0.5\epochs_200\lr_0.001\residual\depth_2\linear_size1024\batch_size_64\no_procrustes\maxnorm\batch_normalization\not_stacked_hourglass\predict_17\checkpoint-24371.data-00000-of-00001.tempstate14109936691770814557 : The system cannot find the path specified.
; No such process
[[{{node save/SaveV2}}]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "src/predict_3dpose.py", line 537, in
tf.app.run()
File "C:\Users\Garima\Anaconda3\envs\fastai-3.7\lib\site-packages\tensorflow\python\platform\app.py", line 125, in run
_sys.exit(main(argv))
File "src/predict_3dpose.py", line 534, in main
train()
File "src/predict_3dpose.py", line 281, in train
model.saver.save(sess, os.path.join(train_dir, 'checkpoint'), global_step=current_step )
File "C:\Users\Garima\Anaconda3\envs\fastai-3.7\lib\site-packages\tensorflow\python\training\saver.py", line 1188, in save
raise exc
File "C:\Users\Garima\Anaconda3\envs\fastai-3.7\lib\site-packages\tensorflow\python\training\saver.py", line 1171, in save
{self.saver_def.filename_tensor_name: checkpoint_file})
File "C:\Users\Garima\Anaconda3\envs\fastai-3.7\lib\site-packages\tensorflow\python\client\session.py", line 929, in run
run_metadata_ptr)
File "C:\Users\Garima\Anaconda3\envs\fastai-3.7\lib\site-packages\tensorflow\python\client\session.py", line 1152, in _run
feed_dict_tensor, options, run_metadata)
File "C:\Users\Garima\Anaconda3\envs\fastai-3.7\lib\site-packages\tensorflow\python\client\session.py", line 1328, in _do_run
run_metadata)
File "C:\Users\Garima\Anaconda3\envs\fastai-3.7\lib\site-packages\tensorflow\python\client\session.py", line 1348, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.NotFoundError: Failed to create a NewWriteableFile: experiments\All\dropout_0.5\epochs_200\lr_0.001\residual\depth_2\linear_size1024\batch_size_64\no_procrustes\maxnorm\batch_normalization\not_stacked_hourglass\predict_17\checkpoint-24371.data-00000-of-00001.tempstate14109936691770814557 : The system cannot find the path specified.
; No such process
[[node save/SaveV2 (defined at D:\git_hub\3d-pose-baseline\src\linear_model.py:151) ]]

Caused by op 'save/SaveV2', defined at:
File "src/predict_3dpose.py", line 537, in
tf.app.run()
File "C:\Users\Garima\Anaconda3\envs\fastai-3.7\lib\site-packages\tensorflow\python\platform\app.py", line 125, in run
_sys.exit(main(argv))
File "src/predict_3dpose.py", line 534, in main
train()
File "src/predict_3dpose.py", line 171, in train
model = create_model( sess, actions, FLAGS.batch_size )
File "src/predict_3dpose.py", line 110, in create_model
dtype=tf.float16 if FLAGS.use_fp16 else tf.float32)
File "D:\git_hub\3d-pose-baseline\src\linear_model.py", line 151, in init
self.saver = tf.train.Saver( tf.global_variables(), max_to_keep=10 )
File "C:\Users\Garima\Anaconda3\envs\fastai-3.7\lib\site-packages\tensorflow\python\training\saver.py", line 832, in init
self.build()
File "C:\Users\Garima\Anaconda3\envs\fastai-3.7\lib\site-packages\tensorflow\python\training\saver.py", line 844, in build
self._build(self._filename, build_save=True, build_restore=True)
File "C:\Users\Garima\Anaconda3\envs\fastai-3.7\lib\site-packages\tensorflow\python\training\saver.py", line 881, in _build
build_save=build_save, build_restore=build_restore)
File "C:\Users\Garima\Anaconda3\envs\fastai-3.7\lib\site-packages\tensorflow\python\training\saver.py", line 510, in _build_internal
save_tensor = self._AddSaveOps(filename_tensor, saveables)
File "C:\Users\Garima\Anaconda3\envs\fastai-3.7\lib\site-packages\tensorflow\python\training\saver.py", line 210, in _AddSaveOps
save = self.save_op(filename_tensor, saveables)
File "C:\Users\Garima\Anaconda3\envs\fastai-3.7\lib\site-packages\tensorflow\python\training\saver.py", line 124, in save_op
tensors)
File "C:\Users\Garima\Anaconda3\envs\fastai-3.7\lib\site-packages\tensorflow\python\ops\gen_io_ops.py", line 1807, in save_v2
name=name)
File "C:\Users\Garima\Anaconda3\envs\fastai-3.7\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 788, in _apply_op_helper
op_def=op_def)
File "C:\Users\Garima\Anaconda3\envs\fastai-3.7\lib\site-packages\tensorflow\python\util\deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "C:\Users\Garima\Anaconda3\envs\fastai-3.7\lib\site-packages\tensorflow\python\framework\ops.py", line 3300, in create_op
op_def=op_def)
File "C:\Users\Garima\Anaconda3\envs\fastai-3.7\lib\site-packages\tensorflow\python\framework\ops.py", line 1801, in init
self._traceback = tf_stack.extract_stack()

NotFoundError (see above for traceback): Failed to create a NewWriteableFile: experiments\All\dropout_0.5\epochs_200\lr_0.001\residual\depth_2\linear_size1024\batch_size_64\no_procrustes\maxnorm\batch_normalization\not_stacked_hourglass\predict_17\checkpoint-24371.data-00000-of-00001.tempstate14109936691770814557 : The system cannot find the path specified.
; No such process
[[node save/SaveV2 (defined at D:\git_hub\3d-pose-baseline\src\linear_model.py:151) ]]

Thanks for your interest in our research!

If you have problems running our code, please include

1. Your operating system windows 10
2. Your tensorflow version 1.13.1
3. Your python version 3.7.2
4. The stack trace of the error that you see

This is research code, and its primary purpose is to reproduce the scientific results that you see in our paper.
If you are looking for ways to extend our work such as

• handling arbitrary images
• handling videos as input
• using other 2d detectors
• large-scale deployment
• handling missing data
• you want to make a startup out of this code

then pull requests are welcome, but it is very unlikely we will have the time to help you.

pre-trained available through google drive does no unzip properly

Thanks for your interest in our research!

If you have problems running our code, please include

1. Your operating system Windows 10
2. Your tensorflow version TF 1.5
3. Your python version 3.6
4. The stack trace of the error that you see there s no error: after unzipping the pretrained file, the folder is much much smaller than the zip file. Several big files inside the compressed archive are placed in the same directory and override each other during extraction.

This is research code, and its primary purpose is to reproduce the scientific results that you see in our paper.
If you are looking for ways to extend our work such as

• handling arbitrary images
• handling videos as input
• using other 2d detectors
• large-scale deployment
• handling missing data
• you want to make a startup out of this code

then pull requests are welcome, but it is very unlikely we will have the time to help you.

Error not able to visualize results

Hi,
I am getting the following error when I try to run the demo and visualize:
Instructions for updating:
Use tf.cast instead.
train_dir experiments\All\dropout_0.5\epochs_1\lr_0.001\residual\depth_2\linear_size1024\batch_size_64\no_procrustes\maxnorm\batch_normalization\use_stacked_hourglass\predict_17
Could not find checkpoint. Aborting.
Traceback (most recent call last):
File "src/predict_3dpose.py", line 537, in
tf.app.run()
File "C:\Users\k1749021\AppData\Local\Continuum\anaconda3\envs\Lift_Deep1\lib\site-packages\tensorflow\python\platform\app.py", line 125, in run
_sys.exit(main(argv))
File "src/predict_3dpose.py", line 532, in main
sample()
File "src/predict_3dpose.py", line 424, in sample
model = create_model(sess, actions, batch_size)
File "src/predict_3dpose.py", line 137, in create_model
raise( ValueError, "Checkpoint {0} does not seem to exist".format( ckpt.model_checkpoint_path ) )
AttributeError: 'NoneType' object has no attribute 'model_checkpoint_path'

The details requried are as below

1. Your operating system-Windows 10
2. Your tensorflow version-1,0
3. Your python version-3.7
4. The stack trace of the error that you see

Human3.6M link is not longer working

Human3.6M link is not longer working:
http://vision.imar.ro/human3.6m/description.php

ValueError, "Checkpoint {0} does not seem to exist".format( ckpt.model_checkpoint_path ) )

Hi,
Thanks for the code!!
I am trying to reproduce your results and get the hold of 3D reconstruction.
I am getting the following error when I tried to first train your model

and also still I tried further visualizing and i got these errors

Further Details:
OS: Windows 10
Tensor Flow version: Tensorflow GPU 1.12
Python Python 3

Thanks,
Mritula

Thanks for your interest in our research!

If you have problems running our code, please include

1. Your operating system
2. Your tensorflow version
3. Your python version
4. The stack trace of the error that you see

This is research code, and its primary purpose is to reproduce the scientific results that you see in our paper.
If you are looking for ways to extend our work such as

• handling arbitrary images
• handling videos as input
• using other 2d detectors
• large-scale deployment
• handling missing data
• you want to make a startup out of this code

then pull requests are welcome, but it is very unlikely we will have the time to help you.