Comments (19)
During preprocessing, frames are extracted from video, the glob function here is adopted to load frames. When it fails (the path is wrong or the frames are not extracted), the img_list is empty and leads to IndexError.
from vac_cslr.
Hello, author, I found that there seems to be a problem with data preparation, and the original image has not been successfully adjusted to 256x256.
Without this file "/dataset/phoenix2014/phoenix-2014-multisigner/features/fullFrame-256x256px"
Only this file"/dataset/phoenix2014/phoenix-2014-multisigner/features/fullFrame-210x260px"
"/dataset/phoenix2014/phoenix-2014-multisigner/features/trackedRightHand-92x132px"
I running
cd ./preprocess
python data_preprocess.py --process-image --multiprocessing
Terminal display
Generate information dict from ../dataset/phoenix2014/phoenix-2014-multisigner/annotations/manual/dev.corpus.csv
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 540/540 [00:00<00:00, 5993.77it/s]
Resize image to 256x256px
0%| | 0/540 [00:00<?, ?it/s]
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/home/com/anaconda3/envs/ywypytorch/lib/python3.9/multiprocessing/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "/home/user5/Downloads/VAC_CSLR/preprocess/dataset_preprocess.py", line 73, in resize_dataset
os.makedirs(rs_img_dir)
File "/home/com/anaconda3/envs/ywypytorch/lib/python3.9/os.py", line 215, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/home/com/anaconda3/envs/ywypytorch/lib/python3.9/os.py", line 215, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/home/com/anaconda3/envs/ywypytorch/lib/python3.9/os.py", line 215, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/home/com/anaconda3/envs/ywypytorch/lib/python3.9/os.py", line 225, in makedirs
mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '../dataset/phoenix2014/phoenix-2014-multisigner/features/fullFrame-256x256px'
I running
sudo python dataset_preprocess.py --process-image --multiprocessing
Terminal display
[sudo] user5 password:
File "dataset_preprocess.py", line 22
print(f"Generate information dict from {anno_path}")
^
SyntaxError: invalid syntax
I have modified the file permissions through chmod 777
drwxrwxrwx 4 2522 2000 4096 11月 3 2017 phoenix2014- release
from vac_cslr.
Thank you for your interest in our work. Currently, I do not have any plan to provide a step-by-step tutorial, but I will consider doing so in the future (maybe after June). You can reproduce the experiment following the README, as all necessary steps are clearly outlined there. If you have any questions, you can post them in an issue, which would be convenient for others to refer to as well.
from vac_cslr.
Thank you very much for your comments and I will try to use it. Thank you sincerely for your answer.
from vac_cslr.
Hello, author. I sincerely hope you can give me some help when you have time.
When I am running the following Data Preparation step 3
cd ./preprocess
python data_preprocess.py --process-image --multiprocessing
The terminal display is as follows
(base)user5@com:~/Downloads/VAC_CSL-main/preprocess$
python: can't open file '/home/user5/Downloads/VAC_CSLR-main/preprocess/data_preprocess.py': [Errno 2] No such file or directory
Then, I run dataset_preprocess.py in the pycharm
FileNotFoundError: [Errno 2] No such file or directory: '../dataset/phoenix2014/phoenix-2014-multisigner/annotations/manual/dev.corpus.csv'
I have already done the second step of Data Preparation, I can also see the phienix2014 after the soft link in the dataset.
home/public/ywy1/phoenix-2014-release
home/user5/Downloads/VAC_CSL-main/dataset/phoenix-2014
from vac_cslr.
You can check whether this file "dev.corpus.csv" exists, which is the annotation file of phienix2014 dataset.
from vac_cslr.
Hello, author.Thank you for your answer. I checked the folder and found this file "dev.corpus.csv". I'm sorry to mislead you, my file name is wrong, Ha ha ha.
I see
home/user5/Downloads/VAC_CSL-main/dataset/phoenix-2014
actually
home/user5/Downloads/VAC_CSL-main/dataset/phienix-2014
Author, I have a little comment that you can edit again
README
Data Preparation
step 3
before
ln -s PATH_TO_DATASET/phoenix2014-release ./dataset/phienix2014
after
ln -s PATH_TO_DATASET/phoenix2014-release ./dataset/phoenix2014
from vac_cslr.
I encountered a problem while running the "python main.py --load-weights resnet18_slr_pretrained.pt --phase test"
File "/home/user5/Downloads/VAC_CSLR-main/utils/device.py",line 57,in occupy.gpu
torch.zeros(1).cuda(g)
RuntimeError: CUDA error : invalid device ordinal
CUDA kernel errors might be asynchronously reponted at some other API call, so the stacktrace below might be inconect.For debugging consider passing cUDA_LAUNCH_BLOCKING=1.
I running "print(torch.cuda.device_count())"
show
1
I running
if torch.cuda.is_available():
print('CUDA device is available.')
else:
print("CUDA device is not available.")
show
CUDA device is available.
from vac_cslr.
Thank you for pointing out this typo and it has been corrected. The default configuration file use two GPUs. You can modify it according to your actual situation. For example, if you only want to use one card (e.g., cuda:0), you can make the necessary changes accordingly:
python main.py --load-weights resnet18_slr_pretrained.pt --phase test --device 0
from vac_cslr.
Hello, author. I ran into some problems while running the suggestions you gave me
python main.py --load-weights resnet18_slr_pretrained.pt --phase test --device 0
Terminal display
FileNotFoundError: [Errno 2] No such file or directory: 'resnet18_ sLr_pretrained.pt'
I downloaded the file on Baidu pan. The name of the file is "resnet18_slr_pretrained_distill25.pt" . I put it in this directory.
home/user5/Downloads/VAC_CSL-main/resnet18_slr_pretrained_distill25.pt (Didn't decompress it)
I running
python main.py --load-weights resnet18_slr_pretrained_distill25.pt --phase test --device 0
Terminal display
Loading model
Loading model finished.
Loading data
train 5671
Apply training transform.
train 5671
Apply testing transform.
dev 540
Apply testing transform.
test 629
Apply testing transform.
Loading data finished .
.git does not exist in current dir
[ Thu Mar 23 17:09:44 2023 ] Nodel: slr_network.SLRModel.
[ Thu Mar 23 17:09:44 2023 ] Weights: resnet18_slr_pretrained_distill25.pt.
0%| l 0/68 [00:00<?,?it/s]
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thnead-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc: 98] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
lW pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thead-pool after fork.(function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thnead-pool after fork. (function pthreadpool)
[W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork.(function pthreadpool)
lW pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork.(function pthreadpool)
0%| l 0/68 [00:00<?,?it/s]
Traceback (most recent call last):
File "/home/user5/Downloads/VAC_CSLR-main/main.py",line 211,in
processor.start( )
File "/home/user5/Downloads/VAC_CSLR-main/main.py",line 62,in start
dev_wer = seq_eval(self.arg,self.data_loader[ " dev"],self.model,self.device,
File "/home/user5/Downloads/VAC_CSLR-main/seq_scripts.py",line 49, in seq_evalfor
batch_idx, data in enumerate(tqdm(loader)):
File "/hnome/com/anaconda3/envs/ywypytorch/lib/python3.9/site-packages/tqdm/std.py",line 1178,in .iter
for obj in iterable:
File "/nome/com/anaconda/envs/ywypytonch/lib/python3.9/site-packages/torch/utils/data/dataloader.py",line 521,in .. e.t..
data = self._next_data()
File " /hnome/con/enacondaB/envs/ywypytonch/lit/python3.9/site-packages/toreh/utils/data/dataloader.py",,line 1283,in _next_data
return self.__process_data(data)
File "/hore/con /anaconds/envs/ ymypytonet/1tib/pythons.9/site-packages/tonch/utils/data/dataloader.py",line 1229,in process
datadata.reraise(
File "/home/com/anaconda3/envs/ywypytorch/lib/python3.9/site-packages/torch/_utils.py",line 425,in reraise
raise self.exc_type(msg)
IndexError: Caught IndexError in DataLoader worker process 0.
Original Traceback (most recent call last):
File " /hone/conl/anacondal/envs/ymypytorcth/ltitl/python5.9/site-packages/torch/utils/datal_utils/wonker .py"line 287,in _worker_loop
data = fetcher.fetch(index)
File "/hnone/com/anaconda/envs/ywypytorch/Lio/python3.9/site-packages/torch/utits/datal_utils/fetch.py" ,line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/hore/con/anaconda/envs/ywypytorch/titb/python3.9/site-packages/torch/utils/data/ uti1s/fetch.py",line 44, in <listcomp;
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/nome/user5/Downloads/VAC_CSLR-main/dataset/dataloader_video.py" , line 47,in -.getitem.-
input_data,label = self.normalize(input_data,label)
File "/nome/user5/Downloads/VAC_CSLR-main/dataset/dataloader_video.py",line 78,in normalize
video,label = self.data_aug(video,label, file_id)
File "/home/user5/Downloads/VAC_CSLR-main/utils/video_augmentation.py",line 24,in ..call-.
image = t(image)
File "/home/user5/Downloads/VAC_CSLR-main/utils/video_augmentation.py", line 157,in ._call..
im_h, im_w, im_c = clip[0].shape
IndexErrOr: list index out of range
from vac_cslr.
Hi, @kido1412y2y,similar issue here, you can check whether processed images are found.
from vac_cslr.
I see it, but I really don't understand.
Is meaning there is an error on line 62 of the "dataloader_video.py"?
How do I check whether processed images are found, Could you please describe it in more detail? What are the areas I need to compare and where do I need to correct them?
from vac_cslr.
Perhaps you need to check the permission of features
and features/fullFrame-256x256px
directories and there is no need to run the script with sudo.
from vac_cslr.
Hello , when I am running : python main.py --work-dir /home/user/VAC_CSLR-main/work_dir/baseline_res18/ --config /home/VAC_CSLR-main/configs/baseline.yaml --device 0 , the error will be reported as follows:
Loading model
Loading model finished.
Loading data
train 5671
Apply training transform.
train 5671
Apply testing transform.
dev 540
Apply testing transform.
test 629
Apply testing transform.
Loading data finished.
.git does not exist in current dir
[ Mon Mar 27 08:21:39 2023 ] Model: slr_network.SLRModel.
[ Mon Mar 27 08:21:39 2023 ] Weights: [].
0%| | 0/540 [00:01<?, ?it/s]
Traceback (most recent call last):
File "/home/user/VAC_CSLR-main/main.py", line 211, in
processor.start()
File "/home/user/VAC_CSLR-main/main.py", line 63, in start
"dev", 6667, self.arg.work_dir, self.recoder, self.arg.evaluate_tool)
File "/home/user/VAC_CSLR-main/seq_scripts.py", line 56, in seq_eval
ret_dict = model(vid, vid_lgt, label=label, label_lgt=label_lgt)
File "/home/user/anaconda3/envs/ctc_torch1.8/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/user/VAC_CSLR-main/slr_network.py", line 97, in forward
else self.decoder.decode(outputs, lgt, batch_first=False, probs=False)
File "/home/user/VAC_CSLR-main/utils/decode.py", line 28, in decode
return self.BeamSearch(nn_output, vid_lgt, probs)
File "/home/user/VAC_CSLR-main/utils/decode.py", line 42, in BeamSearch
beam_result, beam_scores, timesteps, out_seq_len = self.ctc_decoder.decode(nn_output, vid_lgt)
File "/home/user/anaconda3/envs/ctc_torch1.8/lib/python3.6/site-packages/ctcdecode/init.py", line 40, in decode
output, timesteps, scores, out_seq_len)
RuntimeError: Not enough space
After I set batch_size and test_batch_size to 1, this problem will still occur. Is there not enough graphics card memory or something else?(gpu can run normally)
Looking forward to your reply.
from vac_cslr.
Hi, @Dikay1. I have not met this problem before, it seems like a problem of ctcdecoder, you can find some information from this issue.
from vac_cslr.
Hello , when I am running : python main.py --work-dir /home/user/VAC_CSLR-main/work_dir/baseline_res18/ --config /home/VAC_CSLR-main/configs/baseline.yaml --device 0 , the error will be reported as follows:
Loading model Loading model finished. Loading data train 5671 Apply training transform.
train 5671 Apply testing transform.
dev 540 Apply testing transform.
test 629 Apply testing transform.
Loading data finished. .git does not exist in current dir [ Mon Mar 27 08:21:39 2023 ] Model: slr_network.SLRModel. [ Mon Mar 27 08:21:39 2023 ] Weights: []. 0%| | 0/540 [00:01<?, ?it/s] Traceback (most recent call last): File "/home/user/VAC_CSLR-main/main.py", line 211, in processor.start() File "/home/user/VAC_CSLR-main/main.py", line 63, in start "dev", 6667, self.arg.work_dir, self.recoder, self.arg.evaluate_tool) File "/home/user/VAC_CSLR-main/seq_scripts.py", line 56, in seq_eval ret_dict = model(vid, vid_lgt, label=label, label_lgt=label_lgt) File "/home/user/anaconda3/envs/ctc_torch1.8/lib/python3.6/site-packages/torch/nn/modules/module.py", line 889, in _call_impl result = self.forward(*input, **kwargs) File "/home/user/VAC_CSLR-main/slr_network.py", line 97, in forward else self.decoder.decode(outputs, lgt, batch_first=False, probs=False) File "/home/user/VAC_CSLR-main/utils/decode.py", line 28, in decode return self.BeamSearch(nn_output, vid_lgt, probs) File "/home/user/VAC_CSLR-main/utils/decode.py", line 42, in BeamSearch beam_result, beam_scores, timesteps, out_seq_len = self.ctc_decoder.decode(nn_output, vid_lgt) File "/home/user/anaconda3/envs/ctc_torch1.8/lib/python3.6/site-packages/ctcdecode/init.py", line 40, in decode output, timesteps, scores, out_seq_len) RuntimeError: Not enough space
After I set batch_size and test_batch_size to 1, this problem will still occur. Is there not enough graphics card memory or something else?(gpu can run normally) Looking forward to your reply.
Hello. I had the same problem. Have you solved it yet?
from vac_cslr.
Hi, @xxxiaosong .It's been a bit long and I can't remember some of it, I may have reinstalled ctcdecoder.
from vac_cslr.
Hi, @xxxiaosong .It's been a bit long and I can't remember some of it, I may have reinstalled ctcdecoder.
Thanks for your reply.
from vac_cslr.
Hello author ! When I run : python main.py --work-dir ./work_dir/vac/ --config ./configs/baseline.yaml --device 0 ,the error occurred:
/(vac) user2@com:~/data/VAC_CSLR-main$ python main.py --work-dir ./work_dir/vac/ --config ./configs/baseline.yaml --device 0
Loading model
/opt/anaconda3/envs/vac/lib/python3.7/site-packages/torchvision/models/_utils.py:209: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
f"The parameter '{pretrained_param}' is deprecated since 0.13 and may be removed in the future, "
/opt/anaconda3/envs/vac/lib/python3.7/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or None
for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing weights=ResNet18_Weights.IMAGENET1K_V1
. You can also use weights=ResNet18_Weights.DEFAULT
to get the most up-to-date weights.
warnings.warn(msg)
Loading model finished.
Loading data
train 5671
Apply training transform.
train 5671
Apply testing transform.
dev 540
Apply testing transform.
test 629
Apply testing transform.
Loading data finished.
.git does not exist in current dir
[ Thu Jul 13 21:34:29 2023 ] Parameters:
{'work_dir': './work_dir/vac/', 'config': './configs/baseline.yaml', 'random_fix': True, 'device': '0', 'phase': 'train', 'save_interval': 5, 'random_seed': 0, 'eval_interval': 1, 'print_log': True, 'log_interval': 50, 'evaluate_tool': 'sclite', 'feeder': 'dataset.dataloader_video.BaseFeeder', 'dataset': 'phoenix14', 'dataset_info': {'dataset_root': './dataset/phoenix2014/phoenix-2014-multisigner', 'dict_path': './preprocess/phoenix2014/gloss_dict.npy', 'evaluation_dir': './evaluation/slr_eval', 'evaluation_prefix': 'phoenix2014-groundtruth'}, 'num_worker': 10, 'feeder_args': {'mode': 'test', 'datatype': 'video', 'num_gloss': -1, 'drop_ratio': 1.0, 'prefix': './dataset/phoenix2014/phoenix-2014-multisigner', 'transform_mode': False}, 'model': 'slr_network.SLRModel', 'model_args': {'num_classes': 1296, 'c2d_type': 'resnet18', 'conv_type': 2, 'use_bn': 1, 'share_classifier': False, 'weight_norm': False}, 'load_weights': None, 'load_checkpoints': None, 'decode_mode': 'beam', 'ignore_weights': [], 'batch_size': 2, 'test_batch_size': 4, 'loss_weights': {'SeqCTC': 1.0}, 'optimizer_args': {'optimizer': 'Adam', 'base_lr': 0.0001, 'step': [20, 35], 'learning_ratio': 1, 'weight_decay': 0.0001, 'start_epoch': 0, 'nesterov': False}, 'num_epoch': 40}
0%| | 0/2835 [00:00<?, ?it/s]
Traceback (most recent call last):
File "main.py", line 211, in
processor.start()
File "main.py", line 45, in start
self.device, epoch, self.recoder)
File "/home/user2/data/VAC_CSLR-main/seq_scripts.py", line 18, in seq_train
for batch_idx, data in enumerate(tqdm(loader)):
File "/opt/anaconda3/envs/vac/lib/python3.7/site-packages/tqdm/std.py", line 1178, in iter
for obj in iterable:
File "/opt/anaconda3/envs/vac/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 628, in next
data = self._next_data()
File "/opt/anaconda3/envs/vac/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1333, in _next_data
return self._process_data(data)
File "/opt/anaconda3/envs/vac/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1359, in _process_data
data.reraise()
File "/opt/anaconda3/envs/vac/lib/python3.7/site-packages/torch/_utils.py", line 543, in reraise
raise exception
IndexError: Caught IndexError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/opt/anaconda3/envs/vac/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop
data = fetcher.fetch(index)
File "/opt/anaconda3/envs/vac/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 58, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/opt/anaconda3/envs/vac/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 58, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/user2/data/VAC_CSLR-main/dataset/dataloader_video.py", line 47, in getitem
input_data, label = self.normalize(input_data, label)
File "/home/user2/data/VAC_CSLR-main/dataset/dataloader_video.py", line 78, in normalize
video, label = self.data_aug(video, label, file_id)
File "/home/user2/data/VAC_CSLR-main/utils/video_augmentation.py", line 24, in call
image = t(image)
File "/home/user2/data/VAC_CSLR-main/utils/video_augmentation.py", line 119, in call
if isinstance(clip[0], np.ndarray):
IndexError: list index out of range
Can you help me please, Thank you very much!
I have successfully used you provide the pretrained models to inference, and have also obtained the same results.
python main.py --load-weights resnet18_slr_pretrained_distill25.pt --phase test --device 0
from vac_cslr.
Related Issues (20)
- Torch not compiled with CUDA enabled HOT 2
- Error when i run the training part main.py HOT 1
- unable to run successfully when I run main.py HOT 1
- unable to successfully install CTC HOT 5
- Visualizing the predicted alignments HOT 1
- unable to run these repository on google colab HOT 1
- Data augmention error HOT 4
- about the feature extractor architecture HOT 6
- Successfully inference, but unable to train HOT 3
- Visualize Epoch vs Loss HOT 1
- Hardware and Software Specifications for this research. HOT 3
- Gloss Segment Boundary Assignment Algorithm (GSBA) HOT 1
- Does your PHOENIX-2014-T dataset have /1/ folder? HOT 1
- model or model.parameters()? HOT 2
- Does your model produce different results each time it is run? HOT 4
- Training the baseline (without VAC, SMKD) HOT 2
- How can I use a trained model to test a video?
- unable to reconize any word but the loss is decreasing??? HOT 1
- Does preprocess.sh have an effect if using datasets other than phoneix?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from vac_cslr.