michigancog/m-pact

A one stop shop for all of your activity recognition needs.

m-pact's Issues

Batch Size 1 - Variable Length

Hey,

I'm trying to train these models on a custom dataset that contains action videos of different length. I don't want to pad or trim video to a fixed length as dataset isn't much plus it's best if exact sequence without modification is given as an input. I guess changing the number-of-frames dim to None or something should fix this if I have a batch size of 1? Can you please tell me where the modifications would be required? I've looked into the models files but can't really make sense of everything.

Thanks

FIFOQueue is closed and has insufficient elements

I'm running into a OutOfRange error when running the following command. The error traceback is listed at the end.

python train.py --model c3d --numGpus 2 --dataset caltrain --load 0 --inputDims 10 --outputDims 4 --seqLength 1 --size 112  --expName example_1 --numClips 1 --clipLength 10 --clipOffset random --numVids 385 --split 1 --wd 0.0005 --lr 0.0001 --nEpochs 30 --baseDataPath data --fName trainlist --saveFreq 1 --verbose 1 --batchSize 10

I'm wondering why this might be, because I'm able to run training commands for the same dataset using resnet50 + LSTM and I3D. FYI this also fails when trying to train TSN. Is there some requirement on the data (such as minimum number of frames, resolutions, etc.) that C3D and TSN impose? I've seen online that this can usually be caused by incorrect paths or corrupt files but I'm confident both are correct as my other commands are working. Memory / GPU are also not issue, using a very beefy EC2 instance. Any help is greatly appreciated, thanks!

Here are the working commands:

# resnet50 + lstm on caltrain
python train.py  --model resnet  --dataset caltrain  --numGpus 2  --load 0  --size 224  --inputDims 10  --outputDims 4  --seqLength 10  --dropoutRate 0.5  --expName example_1  --numVids 385  --lr 0.01  --nEpochs 30  --baseDataPath data  --fName trainlist  --optChoice gd --batchSize 1
python test.py --model resnet --dataset caltrain --loadedDataset caltrain --train 0 --load 1 --inputDims 10 --outputDims 4 --seqLength 10 --size 224 --expName example_1 --numVids 40 --split 1 --baseDataPath data --fName testlist --freeze 1

# i3d on caltrain
python train.py --model i3d --inputDims 10 --outputDims 4 --dataset caltrain --load 0 --expName example_1 --numVids 385 --fName trainlist --seqLength 1 --size 224 --numGpus 2 --train 1 --split 1 --wd 0.0 --lr 0.01 --nEpochs 30 --baseDataPath data --saveFreq 1 --dropoutRate 0.5 --gradClipValue 100.0 --optChoice adam --batchSize 16
python test.py --model i3d --numGpus 1 --dataset caltrain --loadedDataset caltrain --train 0 --load 1 --inputDims 10 --outputDims 4 --seqLength 1 --size 224  --expName example_1 --numVids 40 --split 1 --baseDataPath data --fName testlist --verbose 1

Here is the error traceback:

Traceback (most recent call last):
  File "train.py", line 700, in <module>
    preproc_debugging   = args.preprocDebugging)
  File "train.py", line 525, in train
    learning_rate, model.get_track_variables()])
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 900, in run
    run_metadata_ptr)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1135, in _run
    feed_dict_tensor, options, run_metadata)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1316, in _do_run
    run_metadata)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1335, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.OutOfRangeError: FIFOQueue '_1_my_scope/fifo_queue' is closed and has insufficient elements (requested 20, current size 0)
         [[Node: my_scope/fifo_queue_DequeueMany = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_INT32, DT_STRING, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](my_scope/fifo_queue, my_scope/fifo_queue_DequeueMany/n)]]

Caused by op u'my_scope/fifo_queue_DequeueMany', defined at:
  File "train.py", line 700, in <module>
    preproc_debugging   = args.preprocDebugging)
  File "train.py", line 333, in train
    input_data_tensor, labels_tensor, names_tensor = load_dataset(model, num_gpus, batch_size, output_dims, input_dims, seq_length, size, data_path, dataset, istraining, clip_length, video_offset, clip_offset, num_clips, clip_stride, video_step, preproc_debugging, shuffle_seed, verbose)
  File "/mnt/user-home/M-PACT/utils/load_dataset_tfrecords.py", line 71, in load_dataset
    input_data_tensor, labels_tensor, names_tensor, video_step_tensor, alpha_tensor = clip_q.dequeue_many(num_gpus*batch_size)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/data_flow_ops.py", line 483, in dequeue_many
    self._queue_ref, n=n, component_types=self._dtypes, name=name)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_data_flow_ops.py", line 3480, in queue_dequeue_many_v2
    component_types=component_types, timeout_ms=timeout_ms, name=name)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 3392, in create_op
    op_def=op_def)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1718, in __init__
    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access

OutOfRangeError (see above for traceback): FIFOQueue '_1_my_scope/fifo_queue' is closed and has insufficient elements (requested 20, current size 0)
         [[Node: my_scope/fifo_queue_DequeueMany = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_INT32, DT_STRING, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](my_scope/fifo_queue, my_scope/fifo_queue_DequeueMany/n)]]

TSN loading npy weights error

Hello

I try to load pretrained weights of TSN saved in npy. But I found it not able to load. Error report :
Failed to load checkpoint information file
Failed loading checkpoint requested. Please check.

I guess the reason is npy format is not realy compatiable with tensorflow ckpt format ? How do you work around this ?

memory requirements?

Hi,

Just wonder what is the memory requirements for using M-PACT? I have Tesla p4 and 7G memory. When I run the train.py with ResNet50+LSTM I can see the memory goes up quickly, almost reach 7GB before loading the weights.

Can I ask the memory usage of M-PACT? Thanks,
Chang

Download weights

I can't download the weights neither from:

• sh scripts/shell/download_weights.sh
• http://academictorrents.com/details/dcea7fa53925b31215bd8437d2f0805253d6b00f
• https://app.nihaocloud.com/d/fb0c387c9a644f86b257/

Batch norm parameters are not updating

Hello,

I tried to replicate your results for Resnet50+LSTM on UCF101 data. The reported performance in the default setting is 80.20 and I got 80.30. However, I think there's a bug in training. The batch norm parameters (mean and variance) are not updating during the training, the model is using moving mean and variance from the loaded model (imagenet weights for resnet). Since, the params are not learnable params you need to put explicitly collect those update params and put control dependency on that. I think this will improve performance though I haven't run this change.

If this is true then this might improve performance for all the models and combinations.

Stale import in `utils/generate_tfrecords_dataset.py` causing error

Need to remove line no. 4: from utils import make_dir as the utils directory does not have a file named make_dir. Also, make_dir doesn't look being used in the entire script.

Test realtime

How do I test realtime? can you please share the code for online inference?

modality

Your code doesn't have the multi modality such as optical flow?

can not download the *.npy file as listed in "download_weights.sh"

As mentioned above, I am not sure whether it is the network problem or these files have been removed from their sources, I have tried many times. These weights just can not be downloaded. Can anyone share them with me?