wenwenyu/master-pytorch

Code for the paper "MASTER: Multi-Aspect Non-local Network for Scene Text Recognition" (Pattern Recognition 2021)

master-pytorch's Issues

master 有人复现到论文的结果吗

有偿

验证时识别高的很，测试时不行，（图片大小已经调整的和训练一样了),有大佬可以解决下我的问题吗？有偿，我的微信 qq888lunwen

这个标签实在是有点迷，大佬可以帮忙解释下？

测试

测试代码有问题

Are the download links of the trained checkpoint weights provided?

Update Version of Table Structure MASTER

您好，请问可以开源一下 在ICDAR 2021 Competition on Scientific Literature Parsing Task B: Table recognition中使用的Table Structure MASTER版本吗？

请问有预训练model？

运行错误

在服务器上运行出现RuntimeError: DataLoader worker (pid 2119564) is killed by signal: Segmentation fault.
出现这个问题怎么解决？

pretrained_model

Hello author, can you provide the pre-training weights of MASTER?

感谢开源，请问有谁可以解释下这行代码的意思？谢谢！

train error

when I use python train.py -c configs/config_t.json -d 1 -dist false to train , I made such a mistake.

what's wrong with me?

inference time

Hi thanks for your great model
why inference time in validation phase is so slow?
training phase is faster than validation....
thanks best regards

预测脚本有bug

您好，我在使用您的代码进行预测时报错：
(torch) zhangzr@AI12:~/MASTER-pytorch-main$ python test.py --checkpoint /home/zhangzr/model_output/master_saved/models/MASTER_Default/example_0712_205619/model_best.pth --img_folder /home/zhangzr/table2latex_data/Training_TSR/train_images --width 400 --height 400 --output_folder /home/zhangzr/model_output/test --gpu 1 --batch_size 4
Loading checkpoint: /home/zhangzr/model_output/master_saved/models/MASTER_Default/example_0712_205619/model_best.pth
with saved best metric 0.0000
test data size: 43138 steps: 10785
0it [00:00, ?it/s]
Traceback (most recent call last):
File "test.py", line 128, in
predict(args)
File "test.py", line 73, in predict
outputs, probs = decode_util.greedy_decode_with_probability(model, images, LabelTransformer.max_length,
TypeError: greedy_decode_with_probability() got an unexpected keyword argument 'padding_symbol'

和paddleocr相比

您好，我之前一直是用Paddleocr，请问您这个仓库的效果和它相比，效果有提高或者是哪里有优势么？

使用debug模式进行单gpu训练报错，显示堆栈溢出

您好，我在尝试使用debug模式进行单gpu训练时报错，暂时找不到原因？
(torch) zhangzr@AI12:~/MASTER-pytorch-main$ python train.py -c configs/config.json -d 1 -dist false
[2021-07-11 18:34:01,939 - train - INFO] - One GPU or CPU training mode start...
train.py:137: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
logger.warn('You have chosen to deterministic training. '
[2021-07-11 18:34:01,943 - train - WARNING] - You have chosen to deterministic training. This will fix random seed, turn on the CUDNN deterministic setting, turn off the CUDNN benchmark which can slow down your training considerably!
[2021-07-11 18:34:02,568 - train - INFO] - Dataloader instances have finished. Train datasets: 3067 Val datasets: 2 Train_batch_size/gpu: 8 Val_batch_size/gpu: 8.
[2021-07-11 18:34:04,382 - train - INFO] - Model created, trainable parameters: 54600257.
[2021-07-11 18:34:04,383 - train - INFO] - Optimizer and lr_scheduler created.
[2021-07-11 18:34:04,383 - train - INFO] - Max_epochs: 600 Log_step_interval: 1 Validation_step_interval: 2000.
[2021-07-11 18:34:04,383 - train - INFO] - Training start...
[2021-07-11 18:34:04,412 - trainer - WARNING] - Training is using GPU 0!
Fatal Python error: Cannot recover from stack overflow.
Python runtime state: initialized

Current thread 0x00007f13c82e3340 (most recent call first):
File "/home/zhangzr/anaconda3/envs/torch/lib/python3.8/site-packages/PIL/_util.py", line 6 in isPath
File "/home/zhangzr/anaconda3/envs/torch/lib/python3.8/site-packages/PIL/Image.py", line 2964 in open
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 78 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
File "/home/zhangzr/MASTER-pytorch-main/data_utils/datasets.py", line 101 in getitem
...
Aborted (core dumped)

FileNotFoundError: LMDB file is not found. \data_lmdb_release\training\MJ\MJ_train

Hello, I appreciate your research, when I run the training, I get the following error. Please help, thanks.

### ### #

长文本识别效果如何改进？

默认训练图片是3*48*160进入网络，我尝试使用1*32*320训练中文数据集，STRING_MAX_LEN设置为150（足够满足大部分文本行长度）但是我在测试的时候发现，如果强制resize到32*320，有些长文本效果并不是很好，想请教一下有哪些改进方法？

对比SATRN

从实验精度的结果来看似乎确实没有satrn的精度高，看解码部分也挺像的，使用了传统的CNN网络作为编码部分。中文困难样本依然存在乱出字现象，出的字还是有语义信息的词组。

这一行代码要删除掉，不然回报错

MASTER-pytorch/data_utils/crop_synthtext.py

Line 281 in a8fdc2a

writer_process.close()

报错
··· AttributeError: 'Process' object has no attribute 'close'···

debug模式进行单gpu训练报错

python train.py -c configs/config.json -d 1 -dist false
可以进行训练，在测试验证集时，val_num_workers=0时，显示错误：Segmentation fault (core dumped)；
val_num_workers=2（>0）时，显示错误：
raise RuntimeError: DataLoader worker (pid(s) 22437) exited unexpectedly
您这边这个问题是怎么解决的？

MASTER

你好，关于master模型应该怎么进行部署可以简单指导下吗？你代码后面有关于torch.jit.trace在master里面这个操作是什么意思呢？

预测问题

我训练尺寸是48*240，短文本识别效果蛮好，但是长文本就几乎全错了。
有时候还会出现这类现象：比如图像上内容是“年龄：15”，但是预测结果却是“年龄：1555555555555555555555555555555555555555555555555......”好多个。
请问对该问题有什么看法吗？

label编码的代码在哪

dataset里的代码是
if self.training:
if not self.case_sensitive:
label = label.lower()
return (img, label)
DistCollateFn里没有看到相关代码

输入给模型的label不能是str吧

多行数据怎么标注呢？识别结果解码出来怎么分开呢

    for index, (pred, prob, img_name) in enumerate(zip(outputs[:, 1:], probs, file_names)):
        predict_text = ""
        # pred_list = []
        pred_score_list = []
        for i in range(len(pred)):  # decode one sample
            if pred[i] == LabelTransformer.EOS:
                pred_score_list.append(prob[i])
                break
            if pred[i] == LabelTransformer.UNK:
                continue
            decoded_char = LabelTransformer.decode(pred[i])
            predict_text += decoded_char
            # pred_list.append(decoded_char)
            pred_score_list.append(prob[i])
        pred_score = sum(pred_score_list) / len(pred_score_list)
        pred_item = {"filename": Path(img_name).name,
                     "result": predict_text,
                     "pred_score": pred_score.cpu().item()}
        pred_results.append(pred_item)

这段代码多行数据直接合成一条后怎么分开呢，标注的时候多行用特殊符号隔开，解码的时候怎么把两行数据分开呢，会预测出特殊符号？

Validation 精度很高，但是 test 精度很低

我使用了你们提供的 lmdb 数据集来复现论文结果，训练过程中的验证集精度很高：

[2021-10-30 06:14:34,656 - trainer - INFO] - [Step Validation] Epoch:[3/16] Step:[24000/24898] Word_acc: 0.961988 Word_acc_case_ins 0.976946Edit_distance_acc: 0.980846
[2021-10-30 06:14:36,834 - trainer - INFO] - Saving checkpoint: /root/paddlejob/workspace/output/models/MASTER_Default/example_1027_102421/checkpoint-epoch3-step24000.pdparams ...
[2021-10-30 06:14:42,358 - trainer - INFO] - Saving current best (at 3 epoch): model_best.pdparams Best word_acc: 0.961988

但是使用 test.py 读取 lmdb evaluation 数据集进行预测时得到的 Sequence Accuracy 却相去甚远。而我把 lmdb 格式的数据读取成 Image 格式并保存成 jpg 图片后再使用 test.py 预测，能得到不错的 Case_ins 精度的结果，但是 Sequence Accuracy 还是很低：

calculating metrics of IC03_867_pred
current sample idx: 0
2021-11-08 20:50:58,762 root  INFO     Sequence Accuracy: 0.429066 Case_ins: 0.950404
2021-11-08 20:50:58,762 root  INFO     Edit Distance Accuracy: 0.494192

我看到读取训练集时 transform 参数是这个类：CustomImagePreprocess，但是 test.py 的 transform 却是这个类：ResizeWeight，它们的作用都是 resize。test.py 默认读取的图片数据不是 lmdb，那么我换成直接读取 lmdb 格式的测试集：test.py，并使用跟读取训练集时 transform 参数同一个 CustomImagePreprocess 得到的测试精度就会很低。请问是什么原因呢？

还有一个问题，config_lmdb.json 中指定的 n_class 应该是论文中说的 66 类吧？

请问论文中这个表格的精度是 Case_ins 精度吗？

有尝试中文效果吗

想知道中文效果如何

Train with lmdb dataset

I want to train Master with LMDB dataset but I don't see any args related to image size. How can I train with fixed image size ?

ConcatDataset不存在

使用config_lmdb.json， 下面的Hierarchy_dataset调用了没实现的方法ConcatDataset(dataset_list)并报错

test.py运行报错

Loading checkpoint: /1/dl_pzx/MASTER-pytorch/model_output/master_saved/models/MASTER_Default/example_0314_212753/model_best.pth
with saved best metric 0.9623
test data size: 2 steps: 2
0%| | 0/2 [00:00<?, ?it/s]Fatal Python error: Cannot recover from stack overflow.

Current thread 0x00007fe5b3b2b3c0 (most recent call first):
File "/home/dell10/anaconda3/envs/pzx/lib/python3.7/site-packages/PIL/_util.py", line 6 in isPath
File "/home/dell10/anaconda3/envs/pzx/lib/python3.7/site-packages/PIL/Image.py", line 2971 in open
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 82 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
File "/1/dl_pzx/MASTER-pytorch/data_utils/datasets.py", line 104 in getitem
...
0%| | 0/2 [00:00<?, ?it/s]
Traceback (most recent call last):
File "/home/dell10/anaconda3/envs/pzx/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 990, in _try_get_data
data = self._data_queue.get(timeout=timeout)
File "/home/dell10/anaconda3/envs/pzx/lib/python3.7/multiprocessing/queues.py", line 104, in get
if not self._poll(timeout):
File "/home/dell10/anaconda3/envs/pzx/lib/python3.7/multiprocessing/connection.py", line 257, in poll
return self._poll(timeout)
File "/home/dell10/anaconda3/envs/pzx/lib/python3.7/multiprocessing/connection.py", line 414, in _poll
r = wait([self], timeout)
File "/home/dell10/anaconda3/envs/pzx/lib/python3.7/multiprocessing/connection.py", line 921, in wait
ready = selector.select(timeout)
File "/home/dell10/anaconda3/envs/pzx/lib/python3.7/selectors.py", line 415, in select
fd_event_list = self._selector.poll(timeout)
File "/home/dell10/anaconda3/envs/pzx/lib/python3.7/site-packages/torch/utils/data/_utils/signal_handling.py", line 66, in handler
_error_if_any_worker_fails()
RuntimeError: DataLoader worker (pid 31156) is killed by signal: Aborted.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/1/dl_pzx/MASTER-pytorch/test.py", line 130, in
predict(args)
File "/1/dl_pzx/MASTER-pytorch/test.py", line 61, in predict
for step_idx, input_data_item in tqdm(enumerate(test_data_loader),total=len(test_data_loader)):
File "/home/dell10/anaconda3/envs/pzx/lib/python3.7/site-packages/tqdm/std.py", line 1185, in iter
for obj in iterable:
File "/home/dell10/anaconda3/envs/pzx/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 521, in next
data = self._next_data()
File "/home/dell10/anaconda3/envs/pzx/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1186, in _next_data
idx, data = self._get_data()
File "/home/dell10/anaconda3/envs/pzx/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1152, in _get_data
success, data = self._try_get_data()
File "/home/dell10/anaconda3/envs/pzx/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1003, in _try_get_data
raise RuntimeError('DataLoader worker (pid(s) {}) exited unexpectedly error is {}'.format(pids_str,e)) from e
RuntimeError: DataLoader worker (pid(s) 31156) exited unexpectedly error is DataLoader worker (pid 31156) is killed by signal: Aborted.

bs设置为了1，num_worker也为1，DataLoader报错的pid也一直在变化，请问是什么原因？

Availability of your PyTorch checkpoints

Hey @wenwenyu ,
I was wondering whether you plan to make PyTorch checkpoints available or not.

I already see that you shared TF checkpoint, however having the PyTorch version could also be really useful for the whole community.

Anyway, thanks for open sourcing your code.