Giter Site home page Giter Site logo

Comments (7)

LiheYoung avatar LiheYoung commented on September 24, 2024 1

Our total batch size is 8 (2 imgs / card x 4 cards) for both labeled and unlabeled images. The corresponding learning rate is 0.001. So if you only use a single card and the batch size is 8 / K, the learning rate should also be 0.001 / K. For example, if your batch size is 4 only with a single card, the learning rate then will be 0.0005.

from unimatch.

nysp78 avatar nysp78 commented on September 24, 2024

I ran the unimatch with batch size 4 and initial lr 0.0005 on a single GPU for the 1/16 partition of pascal and I only took 75.50 mIOU. Τhere is any other trick I can do or perhaps the results can not be reproduced on a single GPU?

from unimatch.

LiheYoung avatar LiheYoung commented on September 24, 2024

Please detail which backbone and cropping size are adopted. And what the reported result is.

from unimatch.

nysp78 avatar nysp78 commented on September 24, 2024

[2022-11-25 16:02:33,141][ INFO] {'backbone': 'resnet101',
'batch_size': 4,
'conf_thresh': 0.95,
'criterion': {'kwargs': {'ignore_index': 255}, 'name': 'CELoss'},
'crop_size': 321,
'data_root': /path/to/pascal,
'dataset': 'pascal',
'dilations': [6, 12, 18],
'epochs': 80,
'lr': 0.0005,
'lr_multi': 10.0,
'multi_grid': False,
'nclass': 21,
'replace_stride_with_dilation': [False, False, True]}

[2022-11-25 16:02:34,849][ INFO] Total params: 59.5M

[2022-11-25 16:02:37,349][ INFO] ===========> Epoch: 0, LR: 0.0005, Previous best: 0.00
[2022-11-25 16:02:45,618][ INFO] Iters: 0, Total loss: 1.575, Loss x: 3.150, Loss s: 0.000, Loss w_fp: 0.000, Mask: 0.000
[2022-11-25 16:05:12,455][ INFO] Iters: 310, Total loss: 0.556, Loss x: 1.075, Loss s: 0.052, Loss w_fp: 0.021, Mask: 0.222
[2022-11-25 16:07:39,331][ INFO] Iters: 620, Total loss: 0.450, Loss x: 0.855, Loss s: 0.071, Loss w_fp: 0.019, Mask: 0.292
[2022-11-25 16:10:06,300][ INFO] Iters: 930, Total loss: 0.395, Loss x: 0.740, Loss s: 0.082, Loss w_fp: 0.018, Mask: 0.327
[2022-11-25 16:12:33,237][ INFO] Iters: 1240, Total loss: 0.361, Loss x: 0.668, Loss s: 0.090, Loss w_fp: 0.019, Mask: 0.366
[2022-11-25 16:14:58,762][ INFO] Iters: 1550, Total loss: 0.338, Loss x: 0.618, Loss s: 0.096, Loss w_fp: 0.019, Mask: 0.385
[2022-11-25 16:17:24,090][ INFO] Iters: 1860, Total loss: 0.318, Loss x: 0.576, Loss s: 0.102, Loss w_fp: 0.019, Mask: 0.404
[2022-11-25 16:19:49,541][ INFO] Iters: 2170, Total loss: 0.303, Loss x: 0.543, Loss s: 0.107, Loss w_fp: 0.020, Mask: 0.422
[2022-11-25 16:25:08,569][ INFO] ***** Evaluation original ***** >>>> meanIOU: 56.33

[2022-11-25 16:25:09,356][ INFO] ===========> Epoch: 1, LR: 0.0005, Previous best: 56.33
[2022-11-25 16:25:10,444][ INFO] Iters: 0, Total loss: 0.115, Loss x: 0.136, Loss s: 0.165, Loss w_fp: 0.022, Mask: 0.721
[2022-11-25 16:27:37,508][ INFO] Iters: 310, Total loss: 0.182, Loss x: 0.276, Loss s: 0.151, Loss w_fp: 0.024, Mask: 0.551
[2022-11-25 16:30:04,694][ INFO] Iters: 620, Total loss: 0.180, Loss x: 0.274, Loss s: 0.148, Loss w_fp: 0.022, Mask: 0.563
[2022-11-25 16:32:31,996][ INFO] Iters: 930, Total loss: 0.180, Loss x: 0.274, Loss s: 0.150, Loss w_fp: 0.022, Mask: 0.573
[2022-11-25 16:34:59,147][ INFO] Iters: 1240, Total loss: 0.176, Loss x: 0.265, Loss s: 0.151, Loss w_fp: 0.022, Mask: 0.584
[2022-11-25 16:37:26,400][ INFO] Iters: 1550, Total loss: 0.170, Loss x: 0.255, Loss s: 0.149, Loss w_fp: 0.022, Mask: 0.590
[2022-11-25 16:39:53,906][ INFO] Iters: 1860, Total loss: 0.169, Loss x: 0.252, Loss s: 0.150, Loss w_fp: 0.021, Mask: 0.595
[2022-11-25 16:42:21,170][ INFO] Iters: 2170, Total loss: 0.167, Loss x: 0.249, Loss s: 0.150, Loss w_fp: 0.021, Mask: 0.598
[2022-11-25 16:45:43,356][ INFO] ***** Evaluation original ***** >>>> meanIOU: 65.14

[2022-11-25 16:45:43,982][ INFO] ===========> Epoch: 2, LR: 0.0005, Previous best: 65.14
[2022-11-25 16:45:45,041][ INFO] Iters: 0, Total loss: 0.065, Loss x: 0.053, Loss s: 0.147, Loss w_fp: 0.008, Mask: 0.330
[2022-11-25 16:48:12,148][ INFO] Iters: 310, Total loss: 0.148, Loss x: 0.211, Loss s: 0.148, Loss w_fp: 0.020, Mask: 0.639
[2022-11-25 16:50:39,326][ INFO] Iters: 620, Total loss: 0.146, Loss x: 0.206, Loss s: 0.150, Loss w_fp: 0.021, Mask: 0.637
[2022-11-25 16:53:06,565][ INFO] Iters: 930, Total loss: 0.146, Loss x: 0.207, Loss s: 0.148, Loss w_fp: 0.020, Mask: 0.636
[2022-11-25 16:55:33,742][ INFO] Iters: 1240, Total loss: 0.142, Loss x: 0.201, Loss s: 0.148, Loss w_fp: 0.020, Mask: 0.640
[2022-11-25 16:58:00,975][ INFO] Iters: 1550, Total loss: 0.141, Loss x: 0.198, Loss s: 0.148, Loss w_fp: 0.020, Mask: 0.642
[2022-11-25 17:00:28,034][ INFO] Iters: 1860, Total loss: 0.139, Loss x: 0.194, Loss s: 0.146, Loss w_fp: 0.020, Mask: 0.646
[2022-11-25 17:02:55,229][ INFO] Iters: 2170, Total loss: 0.138, Loss x: 0.193, Loss s: 0.145, Loss w_fp: 0.019, Mask: 0.648
[2022-11-25 17:06:12,677][ INFO] ***** Evaluation original ***** >>>> meanIOU: 66.59

[2022-11-25 17:06:13,541][ INFO] ===========> Epoch: 3, LR: 0.0005, Previous best: 66.59
[2022-11-25 17:06:14,564][ INFO] Iters: 0, Total loss: 0.091, Loss x: 0.057, Loss s: 0.241, Loss w_fp: 0.009, Mask: 0.687
[2022-11-25 17:08:41,982][ INFO] Iters: 310, Total loss: 0.123, Loss x: 0.159, Loss s: 0.155, Loss w_fp: 0.019, Mask: 0.672
[2022-11-25 17:11:09,323][ INFO] Iters: 620, Total loss: 0.125, Loss x: 0.164, Loss s: 0.154, Loss w_fp: 0.018, Mask: 0.672
[2022-11-25 17:13:36,525][ INFO] Iters: 930, Total loss: 0.126, Loss x: 0.167, Loss s: 0.152, Loss w_fp: 0.018, Mask: 0.672
[2022-11-25 17:16:03,706][ INFO] Iters: 1240, Total loss: 0.126, Loss x: 0.169, Loss s: 0.149, Loss w_fp: 0.018, Mask: 0.669
[2022-11-25 17:18:30,888][ INFO] Iters: 1550, Total loss: 0.126, Loss x: 0.170, Loss s: 0.148, Loss w_fp: 0.018, Mask: 0.670
[2022-11-25 17:20:58,025][ INFO] Iters: 1860, Total loss: 0.125, Loss x: 0.168, Loss s: 0.147, Loss w_fp: 0.018, Mask: 0.670
[2022-11-25 17:23:25,015][ INFO] Iters: 2170, Total loss: 0.124, Loss x: 0.167, Loss s: 0.145, Loss w_fp: 0.018, Mask: 0.671
[2022-11-25 17:26:46,643][ INFO] ***** Evaluation original ***** >>>> meanIOU: 68.03

[2022-11-25 17:26:47,472][ INFO] ===========> Epoch: 4, LR: 0.0005, Previous best: 68.03
[2022-11-25 17:26:48,533][ INFO] Iters: 0, Total loss: 0.092, Loss x: 0.106, Loss s: 0.116, Loss w_fp: 0.039, Mask: 0.710
[2022-11-25 17:29:15,607][ INFO] Iters: 310, Total loss: 0.107, Loss x: 0.137, Loss s: 0.138, Loss w_fp: 0.015, Mask: 0.706
[2022-11-25 17:31:42,655][ INFO] Iters: 620, Total loss: 0.109, Loss x: 0.139, Loss s: 0.141, Loss w_fp: 0.016, Mask: 0.702
[2022-11-25 17:34:09,820][ INFO] Iters: 930, Total loss: 0.108, Loss x: 0.138, Loss s: 0.141, Loss w_fp: 0.016, Mask: 0.697
[2022-11-25 17:36:37,012][ INFO] Iters: 1240, Total loss: 0.107, Loss x: 0.137, Loss s: 0.138, Loss w_fp: 0.015, Mask: 0.699
[2022-11-25 17:39:04,242][ INFO] Iters: 1550, Total loss: 0.107, Loss x: 0.136, Loss s: 0.139, Loss w_fp: 0.015, Mask: 0.701
[2022-11-25 17:41:31,489][ INFO] Iters: 1860, Total loss: 0.107, Loss x: 0.137, Loss s: 0.138, Loss w_fp: 0.015, Mask: 0.703
[2022-11-25 17:43:58,650][ INFO] Iters: 2170, Total loss: 0.107, Loss x: 0.137, Loss s: 0.138, Loss w_fp: 0.015, Mask: 0.704
[2022-11-25 17:47:21,482][ INFO] ***** Evaluation original ***** >>>> meanIOU: 69.49

[2022-11-25 17:47:22,141][ INFO] ===========> Epoch: 5, LR: 0.0005, Previous best: 69.49
[2022-11-25 17:47:23,167][ INFO] Iters: 0, Total loss: 0.038, Loss x: 0.039, Loss s: 0.052, Loss w_fp: 0.022, Mask: 0.812
[2022-11-25 17:49:49,580][ INFO] Iters: 310, Total loss: 0.112, Loss x: 0.148, Loss s: 0.140, Loss w_fp: 0.015, Mask: 0.687
[2022-11-25 17:52:15,380][ INFO] Iters: 620, Total loss: 0.106, Loss x: 0.141, Loss s: 0.126, Loss w_fp: 0.014, Mask: 0.697
[2022-11-25 17:54:42,373][ INFO] Iters: 930, Total loss: 0.103, Loss x: 0.137, Loss s: 0.126, Loss w_fp: 0.014, Mask: 0.706
[2022-11-25 17:57:12,858][ INFO] Iters: 1240, Total loss: 0.102, Loss x: 0.134, Loss s: 0.124, Loss w_fp: 0.014, Mask: 0.708
[2022-11-25 17:59:44,193][ INFO] Iters: 1550, Total loss: 0.102, Loss x: 0.134, Loss s: 0.126, Loss w_fp: 0.014, Mask: 0.709
[2022-11-25 18:02:15,723][ INFO] Iters: 1860, Total loss: 0.102, Loss x: 0.134, Loss s: 0.128, Loss w_fp: 0.014, Mask: 0.708
[2022-11-25 18:04:47,196][ INFO] Iters: 2170, Total loss: 0.102, Loss x: 0.132, Loss s: 0.131, Loss w_fp: 0.014, Mask: 0.709
[2022-11-25 18:08:23,400][ INFO] ***** Evaluation original ***** >>>> meanIOU: 69.62

[2022-11-25 18:08:25,530][ INFO] ===========> Epoch: 6, LR: 0.0005, Previous best: 69.62
[2022-11-25 18:08:26,844][ INFO] Iters: 0, Total loss: 0.070, Loss x: 0.128, Loss s: 0.021, Loss w_fp: 0.001, Mask: 0.816
[2022-11-25 18:10:58,762][ INFO] Iters: 310, Total loss: 0.097, Loss x: 0.120, Loss s: 0.133, Loss w_fp: 0.013, Mask: 0.727
[2022-11-25 18:13:30,480][ INFO] Iters: 620, Total loss: 0.097, Loss x: 0.121, Loss s: 0.133, Loss w_fp: 0.013, Mask: 0.728
[2022-11-25 18:16:01,912][ INFO] Iters: 930, Total loss: 0.095, Loss x: 0.118, Loss s: 0.130, Loss w_fp: 0.014, Mask: 0.730
[2022-11-25 18:18:33,570][ INFO] Iters: 1240, Total loss: 0.094, Loss x: 0.117, Loss s: 0.131, Loss w_fp: 0.014, Mask: 0.730
[2022-11-25 18:21:05,225][ INFO] Iters: 1550, Total loss: 0.095, Loss x: 0.116, Loss s: 0.132, Loss w_fp: 0.014, Mask: 0.731
[2022-11-25 18:23:36,861][ INFO] Iters: 1860, Total loss: 0.095, Loss x: 0.116, Loss s: 0.134, Loss w_fp: 0.014, Mask: 0.733
[2022-11-25 18:26:08,254][ INFO] Iters: 2170, Total loss: 0.094, Loss x: 0.114, Loss s: 0.133, Loss w_fp: 0.014, Mask: 0.734
[2022-11-25 18:29:43,723][ INFO] ***** Evaluation original ***** >>>> meanIOU: 70.17

[2022-11-25 18:29:44,586][ INFO] ===========> Epoch: 7, LR: 0.0005, Previous best: 70.17
[2022-11-25 18:29:45,906][ INFO] Iters: 0, Total loss: 0.073, Loss x: 0.131, Loss s: 0.027, Loss w_fp: 0.005, Mask: 0.635
[2022-11-25 18:32:17,722][ INFO] Iters: 310, Total loss: 0.091, Loss x: 0.114, Loss s: 0.122, Loss w_fp: 0.013, Mask: 0.734
[2022-11-25 18:34:50,059][ INFO] Iters: 620, Total loss: 0.094, Loss x: 0.118, Loss s: 0.128, Loss w_fp: 0.013, Mask: 0.731
[2022-11-25 18:37:21,780][ INFO] Iters: 930, Total loss: 0.093, Loss x: 0.114, Loss s: 0.130, Loss w_fp: 0.013, Mask: 0.732
[2022-11-25 18:39:53,319][ INFO] Iters: 1240, Total loss: 0.092, Loss x: 0.114, Loss s: 0.128, Loss w_fp: 0.013, Mask: 0.734
[2022-11-25 18:42:25,440][ INFO] Iters: 1550, Total loss: 0.093, Loss x: 0.113, Loss s: 0.131, Loss w_fp: 0.013, Mask: 0.733
[2022-11-25 18:44:56,969][ INFO] Iters: 1860, Total loss: 0.092, Loss x: 0.112, Loss s: 0.130, Loss w_fp: 0.013, Mask: 0.733
[2022-11-25 18:47:28,275][ INFO] Iters: 2170, Total loss: 0.091, Loss x: 0.111, Loss s: 0.130, Loss w_fp: 0.013, Mask: 0.735
[2022-11-25 18:51:04,887][ INFO] ***** Evaluation original ***** >>>> meanIOU: 69.25

[2022-11-25 18:51:04,888][ INFO] ===========> Epoch: 8, LR: 0.0005, Previous best: 70.17
[2022-11-25 18:51:06,098][ INFO] Iters: 0, Total loss: 0.045, Loss x: 0.071, Loss s: 0.033, Loss w_fp: 0.006, Mask: 0.870
[2022-11-25 18:53:37,571][ INFO] Iters: 310, Total loss: 0.083, Loss x: 0.099, Loss s: 0.119, Loss w_fp: 0.014, Mask: 0.752
[2022-11-25 18:56:09,372][ INFO] Iters: 620, Total loss: 0.085, Loss x: 0.100, Loss s: 0.124, Loss w_fp: 0.014, Mask: 0.749
[2022-11-25 18:58:40,992][ INFO] Iters: 930, Total loss: 0.085, Loss x: 0.101, Loss s: 0.123, Loss w_fp: 0.014, Mask: 0.751
[2022-11-25 19:01:12,512][ INFO] Iters: 1240, Total loss: 0.085, Loss x: 0.102, Loss s: 0.123, Loss w_fp: 0.013, Mask: 0.749
[2022-11-25 19:03:43,991][ INFO] Iters: 1550, Total loss: 0.084, Loss x: 0.101, Loss s: 0.122, Loss w_fp: 0.013, Mask: 0.750
[2022-11-25 19:06:15,637][ INFO] Iters: 1860, Total loss: 0.083, Loss x: 0.100, Loss s: 0.120, Loss w_fp: 0.013, Mask: 0.751
[2022-11-25 19:08:46,753][ INFO] Iters: 2170, Total loss: 0.083, Loss x: 0.099, Loss s: 0.120, Loss w_fp: 0.013, Mask: 0.752
[2022-11-25 19:12:21,983][ INFO] ***** Evaluation original ***** >>>> meanIOU: 70.08

[2022-11-25 19:12:21,983][ INFO] ===========> Epoch: 9, LR: 0.0004, Previous best: 70.17
[2022-11-25 19:12:23,223][ INFO] Iters: 0, Total loss: 0.042, Loss x: 0.060, Loss s: 0.041, Loss w_fp: 0.006, Mask: 0.712
[2022-11-25 19:14:55,485][ INFO] Iters: 310, Total loss: 0.081, Loss x: 0.089, Loss s: 0.133, Loss w_fp: 0.014, Mask: 0.761
[2022-11-25 19:17:28,156][ INFO] Iters: 620, Total loss: 0.080, Loss x: 0.092, Loss s: 0.124, Loss w_fp: 0.013, Mask: 0.763
[2022-11-25 19:19:59,950][ INFO] Iters: 930, Total loss: 0.080, Loss x: 0.094, Loss s: 0.120, Loss w_fp: 0.013, Mask: 0.765
[2022-11-25 19:22:30,970][ INFO] Iters: 1240, Total loss: 0.080, Loss x: 0.094, Loss s: 0.119, Loss w_fp: 0.013, Mask: 0.764
[2022-11-25 19:25:03,080][ INFO] Iters: 1550, Total loss: 0.079, Loss x: 0.093, Loss s: 0.119, Loss w_fp: 0.012, Mask: 0.765
[2022-11-25 19:27:34,553][ INFO] Iters: 1860, Total loss: 0.079, Loss x: 0.093, Loss s: 0.118, Loss w_fp: 0.012, Mask: 0.763
[2022-11-25 19:30:05,906][ INFO] Iters: 2170, Total loss: 0.079, Loss x: 0.093, Loss s: 0.118, Loss w_fp: 0.012, Mask: 0.764
[2022-11-25 19:33:40,794][ INFO] ***** Evaluation original ***** >>>> meanIOU: 71.09

[2022-11-25 19:33:41,548][ INFO] ===========> Epoch: 10, LR: 0.0004, Previous best: 71.09
[2022-11-25 19:33:42,738][ INFO] Iters: 0, Total loss: 0.207, Loss x: 0.108, Loss s: 0.588, Loss w_fp: 0.026, Mask: 0.719
[2022-11-25 19:36:14,250][ INFO] Iters: 310, Total loss: 0.079, Loss x: 0.093, Loss s: 0.120, Loss w_fp: 0.011, Mask: 0.760
[2022-11-25 19:38:45,860][ INFO] Iters: 620, Total loss: 0.080, Loss x: 0.096, Loss s: 0.118, Loss w_fp: 0.011, Mask: 0.758
[2022-11-25 19:41:17,631][ INFO] Iters: 930, Total loss: 0.080, Loss x: 0.095, Loss s: 0.118, Loss w_fp: 0.011, Mask: 0.762
[2022-11-25 19:43:49,302][ INFO] Iters: 1240, Total loss: 0.080, Loss x: 0.096, Loss s: 0.116, Loss w_fp: 0.012, Mask: 0.761
[2022-11-25 19:46:21,250][ INFO] Iters: 1550, Total loss: 0.080, Loss x: 0.096, Loss s: 0.118, Loss w_fp: 0.012, Mask: 0.762
[2022-11-25 19:48:53,459][ INFO] Iters: 1860, Total loss: 0.080, Loss x: 0.095, Loss s: 0.118, Loss w_fp: 0.012, Mask: 0.760
[2022-11-25 19:51:25,058][ INFO] Iters: 2170, Total loss: 0.080, Loss x: 0.096, Loss s: 0.117, Loss w_fp: 0.012, Mask: 0.761
[2022-11-25 19:55:00,769][ INFO] ***** Evaluation original ***** >>>> meanIOU: 72.05

[2022-11-25 19:55:01,511][ INFO] ===========> Epoch: 11, LR: 0.0004, Previous best: 72.05
[2022-11-25 19:55:02,744][ INFO] Iters: 0, Total loss: 0.098, Loss x: 0.028, Loss s: 0.335, Loss w_fp: 0.003, Mask: 0.270
[2022-11-25 19:57:34,080][ INFO] Iters: 310, Total loss: 0.083, Loss x: 0.105, Loss s: 0.112, Loss w_fp: 0.011, Mask: 0.768
[2022-11-25 20:00:05,952][ INFO] Iters: 620, Total loss: 0.078, Loss x: 0.097, Loss s: 0.108, Loss w_fp: 0.011, Mask: 0.771
[2022-11-25 20:02:37,223][ INFO] Iters: 930, Total loss: 0.078, Loss x: 0.095, Loss s: 0.110, Loss w_fp: 0.011, Mask: 0.775
[2022-11-25 20:05:09,282][ INFO] Iters: 1240, Total loss: 0.077, Loss x: 0.093, Loss s: 0.112, Loss w_fp: 0.011, Mask: 0.773
[2022-11-25 20:07:41,667][ INFO] Iters: 1550, Total loss: 0.076, Loss x: 0.091, Loss s: 0.112, Loss w_fp: 0.011, Mask: 0.774
[2022-11-25 20:10:14,422][ INFO] Iters: 1860, Total loss: 0.077, Loss x: 0.090, Loss s: 0.116, Loss w_fp: 0.011, Mask: 0.774
[2022-11-25 20:12:46,237][ INFO] Iters: 2170, Total loss: 0.076, Loss x: 0.090, Loss s: 0.115, Loss w_fp: 0.011, Mask: 0.776
[2022-11-25 20:16:23,248][ INFO] ***** Evaluation original ***** >>>> meanIOU: 71.68

[2022-11-25 20:16:23,248][ INFO] ===========> Epoch: 12, LR: 0.0004, Previous best: 72.05
[2022-11-25 20:16:24,392][ INFO] Iters: 0, Total loss: 0.051, Loss x: 0.050, Loss s: 0.071, Loss w_fp: 0.034, Mask: 0.730
[2022-11-25 20:18:56,379][ INFO] Iters: 310, Total loss: 0.077, Loss x: 0.088, Loss s: 0.122, Loss w_fp: 0.011, Mask: 0.779
[2022-11-25 20:21:28,478][ INFO] Iters: 620, Total loss: 0.075, Loss x: 0.085, Loss s: 0.122, Loss w_fp: 0.011, Mask: 0.782
[2022-11-25 20:24:00,077][ INFO] Iters: 930, Total loss: 0.073, Loss x: 0.084, Loss s: 0.115, Loss w_fp: 0.011, Mask: 0.783
[2022-11-25 20:26:31,537][ INFO] Iters: 1240, Total loss: 0.074, Loss x: 0.084, Loss s: 0.117, Loss w_fp: 0.011, Mask: 0.782
[2022-11-25 20:29:03,227][ INFO] Iters: 1550, Total loss: 0.074, Loss x: 0.085, Loss s: 0.116, Loss w_fp: 0.011, Mask: 0.779
[2022-11-25 20:31:35,276][ INFO] Iters: 1860, Total loss: 0.074, Loss x: 0.085, Loss s: 0.115, Loss w_fp: 0.011, Mask: 0.779
[2022-11-25 20:34:06,577][ INFO] Iters: 2170, Total loss: 0.074, Loss x: 0.084, Loss s: 0.115, Loss w_fp: 0.011, Mask: 0.781
[2022-11-25 20:37:42,721][ INFO] ***** Evaluation original ***** >>>> meanIOU: 71.79

[2022-11-25 20:37:42,721][ INFO] ===========> Epoch: 13, LR: 0.0004, Previous best: 72.05
[2022-11-25 20:37:44,065][ INFO] Iters: 0, Total loss: 0.057, Loss x: 0.095, Loss s: 0.035, Loss w_fp: 0.004, Mask: 0.880
[2022-11-25 20:40:15,848][ INFO] Iters: 310, Total loss: 0.073, Loss x: 0.085, Loss s: 0.112, Loss w_fp: 0.009, Mask: 0.781
[2022-11-25 20:42:47,639][ INFO] Iters: 620, Total loss: 0.072, Loss x: 0.084, Loss s: 0.113, Loss w_fp: 0.010, Mask: 0.784
[2022-11-25 20:45:18,999][ INFO] Iters: 930, Total loss: 0.072, Loss x: 0.082, Loss s: 0.111, Loss w_fp: 0.010, Mask: 0.785
[2022-11-25 20:47:51,388][ INFO] Iters: 1240, Total loss: 0.072, Loss x: 0.083, Loss s: 0.112, Loss w_fp: 0.010, Mask: 0.785
[2022-11-25 20:50:23,184][ INFO] Iters: 1550, Total loss: 0.071, Loss x: 0.082, Loss s: 0.111, Loss w_fp: 0.011, Mask: 0.787
[2022-11-25 20:52:54,934][ INFO] Iters: 1860, Total loss: 0.071, Loss x: 0.083, Loss s: 0.109, Loss w_fp: 0.011, Mask: 0.789
[2022-11-25 20:55:26,701][ INFO] Iters: 2170, Total loss: 0.072, Loss x: 0.083, Loss s: 0.111, Loss w_fp: 0.010, Mask: 0.789
[2022-11-25 20:59:03,060][ INFO] ***** Evaluation original ***** >>>> meanIOU: 72.21

[2022-11-25 20:59:03,741][ INFO] ===========> Epoch: 14, LR: 0.0004, Previous best: 72.21
[2022-11-25 20:59:04,979][ INFO] Iters: 0, Total loss: 0.110, Loss x: 0.145, Loss s: 0.132, Loss w_fp: 0.016, Mask: 0.884
[2022-11-25 21:01:36,628][ INFO] Iters: 310, Total loss: 0.069, Loss x: 0.080, Loss s: 0.106, Loss w_fp: 0.011, Mask: 0.795
[2022-11-25 21:04:08,967][ INFO] Iters: 620, Total loss: 0.069, Loss x: 0.080, Loss s: 0.108, Loss w_fp: 0.011, Mask: 0.796
[2022-11-25 21:06:40,441][ INFO] Iters: 930, Total loss: 0.068, Loss x: 0.078, Loss s: 0.106, Loss w_fp: 0.010, Mask: 0.796
[2022-11-25 21:09:12,309][ INFO] Iters: 1240, Total loss: 0.069, Loss x: 0.080, Loss s: 0.106, Loss w_fp: 0.010, Mask: 0.794
[2022-11-25 21:11:44,643][ INFO] Iters: 1550, Total loss: 0.068, Loss x: 0.079, Loss s: 0.106, Loss w_fp: 0.010, Mask: 0.794
[2022-11-25 21:14:16,856][ INFO] Iters: 1860, Total loss: 0.068, Loss x: 0.078, Loss s: 0.107, Loss w_fp: 0.010, Mask: 0.793
[2022-11-25 21:16:48,650][ INFO] Iters: 2170, Total loss: 0.069, Loss x: 0.079, Loss s: 0.108, Loss w_fp: 0.010, Mask: 0.791
[2022-11-25 21:20:24,370][ INFO] ***** Evaluation original ***** >>>> meanIOU: 73.29

[2022-11-25 21:20:25,107][ INFO] ===========> Epoch: 15, LR: 0.0004, Previous best: 73.29
[2022-11-25 21:20:26,304][ INFO] Iters: 0, Total loss: 0.046, Loss x: 0.058, Loss s: 0.051, Loss w_fp: 0.018, Mask: 0.899
[2022-11-25 21:22:57,513][ INFO] Iters: 310, Total loss: 0.067, Loss x: 0.077, Loss s: 0.103, Loss w_fp: 0.009, Mask: 0.804
[2022-11-25 21:25:29,872][ INFO] Iters: 620, Total loss: 0.067, Loss x: 0.076, Loss s: 0.107, Loss w_fp: 0.010, Mask: 0.797
[2022-11-25 21:28:01,570][ INFO] Iters: 930, Total loss: 0.069, Loss x: 0.077, Loss s: 0.112, Loss w_fp: 0.010, Mask: 0.796
[2022-11-25 21:30:33,722][ INFO] Iters: 1240, Total loss: 0.069, Loss x: 0.078, Loss s: 0.112, Loss w_fp: 0.010, Mask: 0.793
[2022-11-25 21:33:05,826][ INFO] Iters: 1550, Total loss: 0.068, Loss x: 0.077, Loss s: 0.110, Loss w_fp: 0.010, Mask: 0.795
[2022-11-25 21:35:37,127][ INFO] Iters: 1860, Total loss: 0.068, Loss x: 0.077, Loss s: 0.111, Loss w_fp: 0.010, Mask: 0.795
[2022-11-25 21:38:09,178][ INFO] Iters: 2170, Total loss: 0.069, Loss x: 0.077, Loss s: 0.110, Loss w_fp: 0.009, Mask: 0.793
[2022-11-25 21:41:44,043][ INFO] ***** Evaluation original ***** >>>> meanIOU: 73.38

[2022-11-25 21:41:44,735][ INFO] ===========> Epoch: 16, LR: 0.0004, Previous best: 73.38
[2022-11-25 21:41:45,856][ INFO] Iters: 0, Total loss: 0.032, Loss x: 0.036, Loss s: 0.054, Loss w_fp: 0.004, Mask: 0.877
[2022-11-25 21:44:17,663][ INFO] Iters: 310, Total loss: 0.070, Loss x: 0.080, Loss s: 0.112, Loss w_fp: 0.009, Mask: 0.796
[2022-11-25 21:46:49,893][ INFO] Iters: 620, Total loss: 0.067, Loss x: 0.077, Loss s: 0.105, Loss w_fp: 0.011, Mask: 0.798
[2022-11-25 21:49:21,569][ INFO] Iters: 930, Total loss: 0.067, Loss x: 0.076, Loss s: 0.106, Loss w_fp: 0.010, Mask: 0.799
[2022-11-25 21:51:53,145][ INFO] Iters: 1240, Total loss: 0.067, Loss x: 0.076, Loss s: 0.105, Loss w_fp: 0.010, Mask: 0.798
[2022-11-25 21:54:24,692][ INFO] Iters: 1550, Total loss: 0.067, Loss x: 0.076, Loss s: 0.106, Loss w_fp: 0.010, Mask: 0.801
[2022-11-25 21:56:56,705][ INFO] Iters: 1860, Total loss: 0.066, Loss x: 0.075, Loss s: 0.106, Loss w_fp: 0.010, Mask: 0.801
[2022-11-25 21:59:28,836][ INFO] Iters: 2170, Total loss: 0.066, Loss x: 0.075, Loss s: 0.106, Loss w_fp: 0.010, Mask: 0.799
[2022-11-25 22:03:05,959][ INFO] ***** Evaluation original ***** >>>> meanIOU: 73.11

[2022-11-25 22:03:05,959][ INFO] ===========> Epoch: 17, LR: 0.0004, Previous best: 73.38
[2022-11-25 22:03:07,126][ INFO] Iters: 0, Total loss: 0.060, Loss x: 0.083, Loss s: 0.064, Loss w_fp: 0.012, Mask: 0.715
[2022-11-25 22:05:38,886][ INFO] Iters: 310, Total loss: 0.064, Loss x: 0.075, Loss s: 0.097, Loss w_fp: 0.010, Mask: 0.810
[2022-11-25 22:08:10,586][ INFO] Iters: 620, Total loss: 0.065, Loss x: 0.074, Loss s: 0.103, Loss w_fp: 0.010, Mask: 0.807
[2022-11-25 22:10:42,156][ INFO] Iters: 930, Total loss: 0.067, Loss x: 0.075, Loss s: 0.106, Loss w_fp: 0.010, Mask: 0.805
[2022-11-25 22:13:13,982][ INFO] Iters: 1240, Total loss: 0.066, Loss x: 0.074, Loss s: 0.106, Loss w_fp: 0.010, Mask: 0.805
[2022-11-25 22:15:46,311][ INFO] Iters: 1550, Total loss: 0.066, Loss x: 0.073, Loss s: 0.107, Loss w_fp: 0.010, Mask: 0.807
[2022-11-25 22:18:18,647][ INFO] Iters: 1860, Total loss: 0.066, Loss x: 0.073, Loss s: 0.107, Loss w_fp: 0.010, Mask: 0.807
[2022-11-25 22:20:50,422][ INFO] Iters: 2170, Total loss: 0.066, Loss x: 0.073, Loss s: 0.107, Loss w_fp: 0.010, Mask: 0.805
[2022-11-25 22:24:26,194][ INFO] ***** Evaluation original ***** >>>> meanIOU: 72.51

[2022-11-25 22:24:26,195][ INFO] ===========> Epoch: 18, LR: 0.0004, Previous best: 73.38
[2022-11-25 22:24:27,416][ INFO] Iters: 0, Total loss: 0.075, Loss x: 0.096, Loss s: 0.101, Loss w_fp: 0.009, Mask: 0.850
[2022-11-25 22:26:59,062][ INFO] Iters: 310, Total loss: 0.063, Loss x: 0.069, Loss s: 0.104, Loss w_fp: 0.010, Mask: 0.802
[2022-11-25 22:29:31,213][ INFO] Iters: 620, Total loss: 0.063, Loss x: 0.070, Loss s: 0.101, Loss w_fp: 0.010, Mask: 0.805
[2022-11-25 22:32:03,120][ INFO] Iters: 930, Total loss: 0.065, Loss x: 0.072, Loss s: 0.106, Loss w_fp: 0.010, Mask: 0.800
[2022-11-25 22:34:34,807][ INFO] Iters: 1240, Total loss: 0.065, Loss x: 0.073, Loss s: 0.106, Loss w_fp: 0.010, Mask: 0.802
[2022-11-25 22:37:06,460][ INFO] Iters: 1550, Total loss: 0.066, Loss x: 0.073, Loss s: 0.107, Loss w_fp: 0.010, Mask: 0.803
[2022-11-25 22:39:38,286][ INFO] Iters: 1860, Total loss: 0.066, Loss x: 0.075, Loss s: 0.107, Loss w_fp: 0.010, Mask: 0.802
[2022-11-25 22:42:10,421][ INFO] Iters: 2170, Total loss: 0.067, Loss x: 0.074, Loss s: 0.108, Loss w_fp: 0.010, Mask: 0.802
[2022-11-25 22:45:47,389][ INFO] ***** Evaluation original ***** >>>> meanIOU: 72.03

[2022-11-25 22:45:47,390][ INFO] ===========> Epoch: 19, LR: 0.0004, Previous best: 73.38
[2022-11-25 22:45:48,542][ INFO] Iters: 0, Total loss: 0.044, Loss x: 0.023, Loss s: 0.126, Loss w_fp: 0.006, Mask: 0.831
[2022-11-25 22:48:20,031][ INFO] Iters: 310, Total loss: 0.061, Loss x: 0.067, Loss s: 0.103, Loss w_fp: 0.009, Mask: 0.807
[2022-11-25 22:50:51,460][ INFO] Iters: 620, Total loss: 0.062, Loss x: 0.067, Loss s: 0.105, Loss w_fp: 0.009, Mask: 0.813
[2022-11-25 22:53:23,441][ INFO] Iters: 930, Total loss: 0.063, Loss x: 0.068, Loss s: 0.104, Loss w_fp: 0.009, Mask: 0.810
[2022-11-25 22:55:55,332][ INFO] Iters: 1240, Total loss: 0.063, Loss x: 0.068, Loss s: 0.105, Loss w_fp: 0.009, Mask: 0.810
[2022-11-25 22:58:27,054][ INFO] Iters: 1550, Total loss: 0.063, Loss x: 0.068, Loss s: 0.104, Loss w_fp: 0.009, Mask: 0.810
[2022-11-25 23:00:59,185][ INFO] Iters: 1860, Total loss: 0.063, Loss x: 0.069, Loss s: 0.104, Loss w_fp: 0.009, Mask: 0.810
[2022-11-25 23:03:30,841][ INFO] Iters: 2170, Total loss: 0.063, Loss x: 0.069, Loss s: 0.104, Loss w_fp: 0.009, Mask: 0.811
[2022-11-25 23:07:07,834][ INFO] ***** Evaluation original ***** >>>> meanIOU: 73.40

[2022-11-25 23:07:08,760][ INFO] ===========> Epoch: 20, LR: 0.0004, Previous best: 73.40
[2022-11-25 23:07:10,052][ INFO] Iters: 0, Total loss: 0.109, Loss x: 0.097, Loss s: 0.241, Loss w_fp: 0.002, Mask: 0.909
[2022-11-25 23:09:41,545][ INFO] Iters: 310, Total loss: 0.064, Loss x: 0.072, Loss s: 0.102, Loss w_fp: 0.010, Mask: 0.804
[2022-11-25 23:12:13,575][ INFO] Iters: 620, Total loss: 0.063, Loss x: 0.071, Loss s: 0.101, Loss w_fp: 0.009, Mask: 0.803
[2022-11-25 23:14:45,947][ INFO] Iters: 930, Total loss: 0.062, Loss x: 0.070, Loss s: 0.101, Loss w_fp: 0.009, Mask: 0.807
[2022-11-25 23:17:18,124][ INFO] Iters: 1240, Total loss: 0.062, Loss x: 0.070, Loss s: 0.099, Loss w_fp: 0.009, Mask: 0.808
[2022-11-25 23:19:50,623][ INFO] Iters: 1550, Total loss: 0.062, Loss x: 0.071, Loss s: 0.099, Loss w_fp: 0.009, Mask: 0.811
[2022-11-25 23:22:23,298][ INFO] Iters: 1860, Total loss: 0.063, Loss x: 0.071, Loss s: 0.099, Loss w_fp: 0.009, Mask: 0.812
[2022-11-25 23:24:55,918][ INFO] Iters: 2170, Total loss: 0.063, Loss x: 0.071, Loss s: 0.101, Loss w_fp: 0.009, Mask: 0.811
[2022-11-25 23:28:33,052][ INFO] ***** Evaluation original ***** >>>> meanIOU: 72.97

[2022-11-25 23:28:33,053][ INFO] ===========> Epoch: 21, LR: 0.0004, Previous best: 73.40
[2022-11-25 23:28:34,409][ INFO] Iters: 0, Total loss: 0.164, Loss x: 0.097, Loss s: 0.416, Loss w_fp: 0.048, Mask: 0.562
[2022-11-25 23:31:06,463][ INFO] Iters: 310, Total loss: 0.065, Loss x: 0.072, Loss s: 0.106, Loss w_fp: 0.008, Mask: 0.819
[2022-11-25 23:33:38,695][ INFO] Iters: 620, Total loss: 0.065, Loss x: 0.071, Loss s: 0.108, Loss w_fp: 0.009, Mask: 0.820
[2022-11-25 23:36:10,812][ INFO] Iters: 930, Total loss: 0.063, Loss x: 0.070, Loss s: 0.101, Loss w_fp: 0.009, Mask: 0.819
[2022-11-25 23:38:43,080][ INFO] Iters: 1240, Total loss: 0.063, Loss x: 0.070, Loss s: 0.103, Loss w_fp: 0.009, Mask: 0.817
[2022-11-25 23:41:14,832][ INFO] Iters: 1550, Total loss: 0.063, Loss x: 0.069, Loss s: 0.104, Loss w_fp: 0.009, Mask: 0.816
[2022-11-25 23:43:46,867][ INFO] Iters: 1860, Total loss: 0.062, Loss x: 0.069, Loss s: 0.104, Loss w_fp: 0.009, Mask: 0.817
[2022-11-25 23:46:18,692][ INFO] Iters: 2170, Total loss: 0.062, Loss x: 0.068, Loss s: 0.102, Loss w_fp: 0.009, Mask: 0.817
[2022-11-25 23:49:56,159][ INFO] ***** Evaluation original ***** >>>> meanIOU: 73.91

[2022-11-25 23:49:56,920][ INFO] ===========> Epoch: 22, LR: 0.0004, Previous best: 73.91
[2022-11-25 23:49:58,238][ INFO] Iters: 0, Total loss: 0.044, Loss x: 0.050, Loss s: 0.072, Loss w_fp: 0.005, Mask: 0.837
[2022-11-25 23:52:30,334][ INFO] Iters: 310, Total loss: 0.062, Loss x: 0.065, Loss s: 0.107, Loss w_fp: 0.010, Mask: 0.814
[2022-11-25 23:55:02,457][ INFO] Iters: 620, Total loss: 0.060, Loss x: 0.064, Loss s: 0.103, Loss w_fp: 0.009, Mask: 0.817
[2022-11-25 23:57:34,409][ INFO] Iters: 930, Total loss: 0.061, Loss x: 0.065, Loss s: 0.105, Loss w_fp: 0.009, Mask: 0.819
[2022-11-26 00:00:06,086][ INFO] Iters: 1240, Total loss: 0.061, Loss x: 0.065, Loss s: 0.105, Loss w_fp: 0.009, Mask: 0.817
[2022-11-26 00:02:37,744][ INFO] Iters: 1550, Total loss: 0.062, Loss x: 0.066, Loss s: 0.105, Loss w_fp: 0.009, Mask: 0.818
[2022-11-26 00:05:09,704][ INFO] Iters: 1860, Total loss: 0.061, Loss x: 0.066, Loss s: 0.104, Loss w_fp: 0.009, Mask: 0.817
[2022-11-26 00:07:42,003][ INFO] Iters: 2170, Total loss: 0.061, Loss x: 0.066, Loss s: 0.103, Loss w_fp: 0.009, Mask: 0.816
[2022-11-26 00:11:18,139][ INFO] ***** Evaluation original ***** >>>> meanIOU: 73.62

[2022-11-26 00:11:18,139][ INFO] ===========> Epoch: 23, LR: 0.0004, Previous best: 73.91
[2022-11-26 00:11:19,442][ INFO] Iters: 0, Total loss: 0.109, Loss x: 0.082, Loss s: 0.261, Loss w_fp: 0.013, Mask: 0.752
[2022-11-26 00:13:51,170][ INFO] Iters: 310, Total loss: 0.058, Loss x: 0.066, Loss s: 0.091, Loss w_fp: 0.008, Mask: 0.813
[2022-11-26 00:16:23,112][ INFO] Iters: 620, Total loss: 0.057, Loss x: 0.064, Loss s: 0.093, Loss w_fp: 0.008, Mask: 0.820
[2022-11-26 00:18:55,130][ INFO] Iters: 930, Total loss: 0.058, Loss x: 0.063, Loss s: 0.095, Loss w_fp: 0.009, Mask: 0.818
[2022-11-26 00:21:26,712][ INFO] Iters: 1240, Total loss: 0.058, Loss x: 0.064, Loss s: 0.095, Loss w_fp: 0.008, Mask: 0.820
[2022-11-26 00:23:58,878][ INFO] Iters: 1550, Total loss: 0.058, Loss x: 0.065, Loss s: 0.096, Loss w_fp: 0.009, Mask: 0.820
[2022-11-26 00:26:26,518][ INFO] Iters: 1860, Total loss: 0.059, Loss x: 0.065, Loss s: 0.097, Loss w_fp: 0.009, Mask: 0.821
[2022-11-26 00:28:53,696][ INFO] Iters: 2170, Total loss: 0.059, Loss x: 0.065, Loss s: 0.097, Loss w_fp: 0.009, Mask: 0.822
[2022-11-26 00:32:15,014][ INFO] ***** Evaluation original ***** >>>> meanIOU: 73.14

[2022-11-26 00:32:15,015][ INFO] ===========> Epoch: 24, LR: 0.0004, Previous best: 73.91
[2022-11-26 00:32:16,136][ INFO] Iters: 0, Total loss: 0.022, Loss x: 0.030, Loss s: 0.024, Loss w_fp: 0.003, Mask: 0.900
[2022-11-26 00:34:43,617][ INFO] Iters: 310, Total loss: 0.056, Loss x: 0.064, Loss s: 0.087, Loss w_fp: 0.008, Mask: 0.829
[2022-11-26 00:37:15,260][ INFO] Iters: 620, Total loss: 0.056, Loss x: 0.063, Loss s: 0.089, Loss w_fp: 0.008, Mask: 0.828
[2022-11-26 00:39:46,748][ INFO] Iters: 930, Total loss: 0.057, Loss x: 0.063, Loss s: 0.094, Loss w_fp: 0.008, Mask: 0.826
[2022-11-26 00:42:18,556][ INFO] Iters: 1240, Total loss: 0.057, Loss x: 0.063, Loss s: 0.095, Loss w_fp: 0.008, Mask: 0.825
[2022-11-26 00:44:50,817][ INFO] Iters: 1550, Total loss: 0.058, Loss x: 0.063, Loss s: 0.098, Loss w_fp: 0.008, Mask: 0.825
[2022-11-26 00:47:22,182][ INFO] Iters: 1860, Total loss: 0.058, Loss x: 0.063, Loss s: 0.097, Loss w_fp: 0.008, Mask: 0.825
[2022-11-26 00:49:53,805][ INFO] Iters: 2170, Total loss: 0.058, Loss x: 0.063, Loss s: 0.097, Loss w_fp: 0.008, Mask: 0.824
[2022-11-26 00:53:30,153][ INFO] ***** Evaluation original ***** >>>> meanIOU: 72.66

[2022-11-26 00:53:30,153][ INFO] ===========> Epoch: 25, LR: 0.0004, Previous best: 73.91
[2022-11-26 00:53:31,353][ INFO] Iters: 0, Total loss: 0.062, Loss x: 0.113, Loss s: 0.018, Loss w_fp: 0.006, Mask: 0.870
[2022-11-26 00:56:02,955][ INFO] Iters: 310, Total loss: 0.059, Loss x: 0.063, Loss s: 0.102, Loss w_fp: 0.009, Mask: 0.825
[2022-11-26 00:58:34,438][ INFO] Iters: 620, Total loss: 0.060, Loss x: 0.063, Loss s: 0.104, Loss w_fp: 0.009, Mask: 0.820
[2022-11-26 01:01:06,570][ INFO] Iters: 930, Total loss: 0.059, Loss x: 0.064, Loss s: 0.101, Loss w_fp: 0.009, Mask: 0.822
[2022-11-26 01:03:37,871][ INFO] Iters: 1240, Total loss: 0.059, Loss x: 0.064, Loss s: 0.099, Loss w_fp: 0.009, Mask: 0.823
[2022-11-26 01:06:10,063][ INFO] Iters: 1550, Total loss: 0.058, Loss x: 0.063, Loss s: 0.098, Loss w_fp: 0.008, Mask: 0.824
[2022-11-26 01:08:41,765][ INFO] Iters: 1860, Total loss: 0.058, Loss x: 0.064, Loss s: 0.098, Loss w_fp: 0.009, Mask: 0.824
[2022-11-26 01:11:13,881][ INFO] Iters: 2170, Total loss: 0.059, Loss x: 0.064, Loss s: 0.098, Loss w_fp: 0.009, Mask: 0.824
[2022-11-26 01:14:49,221][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.33

[2022-11-26 01:14:49,940][ INFO] ===========> Epoch: 26, LR: 0.0004, Previous best: 74.33
[2022-11-26 01:14:51,106][ INFO] Iters: 0, Total loss: 0.052, Loss x: 0.058, Loss s: 0.075, Loss w_fp: 0.018, Mask: 0.844
[2022-11-26 01:17:22,892][ INFO] Iters: 310, Total loss: 0.056, Loss x: 0.062, Loss s: 0.091, Loss w_fp: 0.008, Mask: 0.832
[2022-11-26 01:19:55,890][ INFO] Iters: 620, Total loss: 0.055, Loss x: 0.061, Loss s: 0.091, Loss w_fp: 0.009, Mask: 0.832
[2022-11-26 01:22:27,307][ INFO] Iters: 930, Total loss: 0.057, Loss x: 0.061, Loss s: 0.097, Loss w_fp: 0.009, Mask: 0.833
[2022-11-26 01:24:59,476][ INFO] Iters: 1240, Total loss: 0.057, Loss x: 0.061, Loss s: 0.097, Loss w_fp: 0.009, Mask: 0.832
[2022-11-26 01:27:31,186][ INFO] Iters: 1550, Total loss: 0.057, Loss x: 0.061, Loss s: 0.097, Loss w_fp: 0.008, Mask: 0.830
[2022-11-26 01:30:03,458][ INFO] Iters: 1860, Total loss: 0.057, Loss x: 0.061, Loss s: 0.097, Loss w_fp: 0.008, Mask: 0.830
[2022-11-26 01:32:35,568][ INFO] Iters: 2170, Total loss: 0.057, Loss x: 0.061, Loss s: 0.097, Loss w_fp: 0.009, Mask: 0.831
[2022-11-26 01:36:12,444][ INFO] ***** Evaluation original ***** >>>> meanIOU: 73.00

[2022-11-26 01:36:12,445][ INFO] ===========> Epoch: 27, LR: 0.0003, Previous best: 74.33
[2022-11-26 01:36:13,783][ INFO] Iters: 0, Total loss: 0.070, Loss x: 0.092, Loss s: 0.091, Loss w_fp: 0.006, Mask: 0.827
[2022-11-26 01:38:45,911][ INFO] Iters: 310, Total loss: 0.064, Loss x: 0.065, Loss s: 0.115, Loss w_fp: 0.010, Mask: 0.821
[2022-11-26 01:41:17,162][ INFO] Iters: 620, Total loss: 0.062, Loss x: 0.065, Loss s: 0.108, Loss w_fp: 0.009, Mask: 0.823
[2022-11-26 01:43:49,374][ INFO] Iters: 930, Total loss: 0.061, Loss x: 0.064, Loss s: 0.106, Loss w_fp: 0.009, Mask: 0.823
[2022-11-26 01:46:21,581][ INFO] Iters: 1240, Total loss: 0.060, Loss x: 0.063, Loss s: 0.103, Loss w_fp: 0.009, Mask: 0.824
[2022-11-26 01:48:52,891][ INFO] Iters: 1550, Total loss: 0.059, Loss x: 0.063, Loss s: 0.102, Loss w_fp: 0.009, Mask: 0.825
[2022-11-26 01:51:24,832][ INFO] Iters: 1860, Total loss: 0.059, Loss x: 0.063, Loss s: 0.101, Loss w_fp: 0.009, Mask: 0.826
[2022-11-26 01:53:56,355][ INFO] Iters: 2170, Total loss: 0.059, Loss x: 0.063, Loss s: 0.101, Loss w_fp: 0.009, Mask: 0.827
[2022-11-26 01:57:24,997][ INFO] ***** Evaluation original ***** >>>> meanIOU: 75.16

[2022-11-26 01:57:25,717][ INFO] ===========> Epoch: 28, LR: 0.0003, Previous best: 75.16
[2022-11-26 01:57:26,886][ INFO] Iters: 0, Total loss: 0.075, Loss x: 0.039, Loss s: 0.217, Loss w_fp: 0.003, Mask: 0.723
[2022-11-26 01:59:54,536][ INFO] Iters: 310, Total loss: 0.056, Loss x: 0.060, Loss s: 0.095, Loss w_fp: 0.008, Mask: 0.833
[2022-11-26 02:02:22,276][ INFO] Iters: 620, Total loss: 0.058, Loss x: 0.061, Loss s: 0.100, Loss w_fp: 0.008, Mask: 0.834
[2022-11-26 02:04:53,495][ INFO] Iters: 930, Total loss: 0.058, Loss x: 0.061, Loss s: 0.102, Loss w_fp: 0.008, Mask: 0.832
[2022-11-26 02:07:25,745][ INFO] Iters: 1240, Total loss: 0.057, Loss x: 0.061, Loss s: 0.100, Loss w_fp: 0.008, Mask: 0.831
[2022-11-26 02:09:58,255][ INFO] Iters: 1550, Total loss: 0.057, Loss x: 0.061, Loss s: 0.099, Loss w_fp: 0.008, Mask: 0.831
[2022-11-26 02:12:29,832][ INFO] Iters: 1860, Total loss: 0.057, Loss x: 0.061, Loss s: 0.097, Loss w_fp: 0.008, Mask: 0.833
[2022-11-26 02:15:01,580][ INFO] Iters: 2170, Total loss: 0.057, Loss x: 0.061, Loss s: 0.097, Loss w_fp: 0.008, Mask: 0.832
[2022-11-26 02:18:38,219][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.21

[2022-11-26 02:18:38,219][ INFO] ===========> Epoch: 29, LR: 0.0003, Previous best: 75.16
[2022-11-26 02:18:39,527][ INFO] Iters: 0, Total loss: 0.039, Loss x: 0.040, Loss s: 0.070, Loss w_fp: 0.008, Mask: 0.750
[2022-11-26 02:21:11,341][ INFO] Iters: 310, Total loss: 0.055, Loss x: 0.061, Loss s: 0.091, Loss w_fp: 0.009, Mask: 0.833
[2022-11-26 02:23:42,798][ INFO] Iters: 620, Total loss: 0.056, Loss x: 0.060, Loss s: 0.094, Loss w_fp: 0.008, Mask: 0.832
[2022-11-26 02:26:14,431][ INFO] Iters: 930, Total loss: 0.055, Loss x: 0.059, Loss s: 0.093, Loss w_fp: 0.008, Mask: 0.833
[2022-11-26 02:28:46,564][ INFO] Iters: 1240, Total loss: 0.055, Loss x: 0.059, Loss s: 0.092, Loss w_fp: 0.008, Mask: 0.836
[2022-11-26 02:31:18,480][ INFO] Iters: 1550, Total loss: 0.055, Loss x: 0.059, Loss s: 0.094, Loss w_fp: 0.008, Mask: 0.833
[2022-11-26 02:33:50,321][ INFO] Iters: 1860, Total loss: 0.054, Loss x: 0.058, Loss s: 0.093, Loss w_fp: 0.008, Mask: 0.835
[2022-11-26 02:36:22,296][ INFO] Iters: 2170, Total loss: 0.055, Loss x: 0.059, Loss s: 0.093, Loss w_fp: 0.008, Mask: 0.834
[2022-11-26 02:39:58,409][ INFO] ***** Evaluation original ***** >>>> meanIOU: 75.50

[2022-11-26 02:39:59,175][ INFO] ===========> Epoch: 30, LR: 0.0003, Previous best: 75.50
[2022-11-26 02:40:00,395][ INFO] Iters: 0, Total loss: 0.050, Loss x: 0.060, Loss s: 0.073, Loss w_fp: 0.007, Mask: 0.738
[2022-11-26 02:42:32,158][ INFO] Iters: 310, Total loss: 0.056, Loss x: 0.059, Loss s: 0.097, Loss w_fp: 0.007, Mask: 0.836
[2022-11-26 02:45:04,883][ INFO] Iters: 620, Total loss: 0.055, Loss x: 0.058, Loss s: 0.095, Loss w_fp: 0.008, Mask: 0.838
[2022-11-26 02:47:37,097][ INFO] Iters: 930, Total loss: 0.054, Loss x: 0.058, Loss s: 0.094, Loss w_fp: 0.007, Mask: 0.838
[2022-11-26 02:50:08,985][ INFO] Iters: 1240, Total loss: 0.056, Loss x: 0.061, Loss s: 0.094, Loss w_fp: 0.007, Mask: 0.834
[2022-11-26 02:52:40,988][ INFO] Iters: 1550, Total loss: 0.057, Loss x: 0.063, Loss s: 0.096, Loss w_fp: 0.008, Mask: 0.831
[2022-11-26 02:55:12,729][ INFO] Iters: 1860, Total loss: 0.058, Loss x: 0.064, Loss s: 0.095, Loss w_fp: 0.008, Mask: 0.828
[2022-11-26 02:57:44,575][ INFO] Iters: 2170, Total loss: 0.057, Loss x: 0.063, Loss s: 0.095, Loss w_fp: 0.008, Mask: 0.829
[2022-11-26 03:01:19,901][ INFO] ***** Evaluation original ***** >>>> meanIOU: 73.61

[2022-11-26 03:01:19,901][ INFO] ===========> Epoch: 31, LR: 0.0003, Previous best: 75.50
[2022-11-26 03:01:21,159][ INFO] Iters: 0, Total loss: 0.050, Loss x: 0.049, Loss s: 0.099, Loss w_fp: 0.005, Mask: 0.923
[2022-11-26 03:03:52,176][ INFO] Iters: 310, Total loss: 0.056, Loss x: 0.059, Loss s: 0.097, Loss w_fp: 0.008, Mask: 0.829
[2022-11-26 03:06:23,637][ INFO] Iters: 620, Total loss: 0.056, Loss x: 0.061, Loss s: 0.095, Loss w_fp: 0.008, Mask: 0.830
[2022-11-26 03:08:55,391][ INFO] Iters: 930, Total loss: 0.056, Loss x: 0.061, Loss s: 0.094, Loss w_fp: 0.008, Mask: 0.831
[2022-11-26 03:11:26,776][ INFO] Iters: 1240, Total loss: 0.056, Loss x: 0.061, Loss s: 0.094, Loss w_fp: 0.008, Mask: 0.833
[2022-11-26 03:13:58,162][ INFO] Iters: 1550, Total loss: 0.057, Loss x: 0.061, Loss s: 0.096, Loss w_fp: 0.008, Mask: 0.832
[2022-11-26 03:16:29,520][ INFO] Iters: 1860, Total loss: 0.057, Loss x: 0.061, Loss s: 0.096, Loss w_fp: 0.008, Mask: 0.833
[2022-11-26 03:19:01,022][ INFO] Iters: 2170, Total loss: 0.056, Loss x: 0.061, Loss s: 0.094, Loss w_fp: 0.008, Mask: 0.834
[2022-11-26 03:22:35,723][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.38

[2022-11-26 03:22:35,723][ INFO] ===========> Epoch: 32, LR: 0.0003, Previous best: 75.50
[2022-11-26 03:22:36,886][ INFO] Iters: 0, Total loss: 0.066, Loss x: 0.079, Loss s: 0.097, Loss w_fp: 0.009, Mask: 0.688
[2022-11-26 03:25:04,079][ INFO] Iters: 310, Total loss: 0.053, Loss x: 0.058, Loss s: 0.088, Loss w_fp: 0.008, Mask: 0.832
[2022-11-26 03:27:31,214][ INFO] Iters: 620, Total loss: 0.053, Loss x: 0.057, Loss s: 0.090, Loss w_fp: 0.008, Mask: 0.833
[2022-11-26 03:29:58,622][ INFO] Iters: 930, Total loss: 0.053, Loss x: 0.058, Loss s: 0.090, Loss w_fp: 0.008, Mask: 0.834
[2022-11-26 03:32:30,543][ INFO] Iters: 1240, Total loss: 0.054, Loss x: 0.057, Loss s: 0.093, Loss w_fp: 0.008, Mask: 0.836
[2022-11-26 03:35:02,917][ INFO] Iters: 1550, Total loss: 0.054, Loss x: 0.057, Loss s: 0.093, Loss w_fp: 0.008, Mask: 0.838
[2022-11-26 03:37:35,553][ INFO] Iters: 1860, Total loss: 0.054, Loss x: 0.057, Loss s: 0.092, Loss w_fp: 0.008, Mask: 0.839
[2022-11-26 03:40:07,923][ INFO] Iters: 2170, Total loss: 0.053, Loss x: 0.057, Loss s: 0.092, Loss w_fp: 0.008, Mask: 0.839
[2022-11-26 03:43:45,001][ INFO] ***** Evaluation original ***** >>>> meanIOU: 73.81

[2022-11-26 03:43:45,001][ INFO] ===========> Epoch: 33, LR: 0.0003, Previous best: 75.50
[2022-11-26 03:43:46,280][ INFO] Iters: 0, Total loss: 0.046, Loss x: 0.062, Loss s: 0.056, Loss w_fp: 0.004, Mask: 0.851
[2022-11-26 03:46:18,370][ INFO] Iters: 310, Total loss: 0.049, Loss x: 0.053, Loss s: 0.084, Loss w_fp: 0.008, Mask: 0.847
[2022-11-26 03:48:50,543][ INFO] Iters: 620, Total loss: 0.050, Loss x: 0.055, Loss s: 0.083, Loss w_fp: 0.007, Mask: 0.848
[2022-11-26 03:51:22,944][ INFO] Iters: 930, Total loss: 0.051, Loss x: 0.055, Loss s: 0.086, Loss w_fp: 0.007, Mask: 0.845
[2022-11-26 03:53:55,383][ INFO] Iters: 1240, Total loss: 0.051, Loss x: 0.055, Loss s: 0.086, Loss w_fp: 0.007, Mask: 0.844
[2022-11-26 03:56:27,481][ INFO] Iters: 1550, Total loss: 0.052, Loss x: 0.055, Loss s: 0.089, Loss w_fp: 0.008, Mask: 0.843
[2022-11-26 03:58:59,236][ INFO] Iters: 1860, Total loss: 0.052, Loss x: 0.055, Loss s: 0.089, Loss w_fp: 0.007, Mask: 0.844
[2022-11-26 04:01:31,274][ INFO] Iters: 2170, Total loss: 0.052, Loss x: 0.056, Loss s: 0.089, Loss w_fp: 0.007, Mask: 0.844
[2022-11-26 04:05:08,973][ INFO] ***** Evaluation original ***** >>>> meanIOU: 73.42

[2022-11-26 04:05:08,974][ INFO] ===========> Epoch: 34, LR: 0.0003, Previous best: 75.50
[2022-11-26 04:05:10,274][ INFO] Iters: 0, Total loss: 0.017, Loss x: 0.023, Loss s: 0.019, Loss w_fp: 0.005, Mask: 0.923
[2022-11-26 04:07:42,168][ INFO] Iters: 310, Total loss: 0.055, Loss x: 0.059, Loss s: 0.092, Loss w_fp: 0.009, Mask: 0.834
[2022-11-26 04:10:14,693][ INFO] Iters: 620, Total loss: 0.055, Loss x: 0.060, Loss s: 0.092, Loss w_fp: 0.008, Mask: 0.833
[2022-11-26 04:12:46,611][ INFO] Iters: 930, Total loss: 0.054, Loss x: 0.059, Loss s: 0.089, Loss w_fp: 0.008, Mask: 0.837
[2022-11-26 04:15:18,449][ INFO] Iters: 1240, Total loss: 0.054, Loss x: 0.058, Loss s: 0.091, Loss w_fp: 0.007, Mask: 0.837
[2022-11-26 04:17:50,212][ INFO] Iters: 1550, Total loss: 0.053, Loss x: 0.058, Loss s: 0.091, Loss w_fp: 0.007, Mask: 0.838
[2022-11-26 04:20:22,630][ INFO] Iters: 1860, Total loss: 0.054, Loss x: 0.057, Loss s: 0.092, Loss w_fp: 0.007, Mask: 0.838
[2022-11-26 04:22:54,465][ INFO] Iters: 2170, Total loss: 0.054, Loss x: 0.057, Loss s: 0.093, Loss w_fp: 0.008, Mask: 0.837
[2022-11-26 04:26:31,838][ INFO] ***** Evaluation original ***** >>>> meanIOU: 73.96

[2022-11-26 04:26:31,838][ INFO] ===========> Epoch: 35, LR: 0.0003, Previous best: 75.50
[2022-11-26 04:26:33,074][ INFO] Iters: 0, Total loss: 0.068, Loss x: 0.064, Loss s: 0.129, Loss w_fp: 0.016, Mask: 0.818
[2022-11-26 04:29:05,422][ INFO] Iters: 310, Total loss: 0.052, Loss x: 0.056, Loss s: 0.088, Loss w_fp: 0.007, Mask: 0.841
[2022-11-26 04:31:37,662][ INFO] Iters: 620, Total loss: 0.053, Loss x: 0.058, Loss s: 0.087, Loss w_fp: 0.007, Mask: 0.839
[2022-11-26 04:34:10,203][ INFO] Iters: 930, Total loss: 0.053, Loss x: 0.057, Loss s: 0.089, Loss w_fp: 0.007, Mask: 0.840
[2022-11-26 04:36:42,221][ INFO] Iters: 1240, Total loss: 0.052, Loss x: 0.056, Loss s: 0.088, Loss w_fp: 0.007, Mask: 0.841
[2022-11-26 04:39:14,643][ INFO] Iters: 1550, Total loss: 0.052, Loss x: 0.056, Loss s: 0.089, Loss w_fp: 0.007, Mask: 0.842
[2022-11-26 04:41:46,537][ INFO] Iters: 1860, Total loss: 0.052, Loss x: 0.056, Loss s: 0.088, Loss w_fp: 0.007, Mask: 0.843
[2022-11-26 04:44:19,157][ INFO] Iters: 2170, Total loss: 0.051, Loss x: 0.055, Loss s: 0.088, Loss w_fp: 0.007, Mask: 0.843
[2022-11-26 04:47:55,833][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.02

[2022-11-26 04:47:55,834][ INFO] ===========> Epoch: 36, LR: 0.0003, Previous best: 75.50
[2022-11-26 04:47:56,947][ INFO] Iters: 0, Total loss: 0.043, Loss x: 0.053, Loss s: 0.064, Loss w_fp: 0.004, Mask: 0.848
[2022-11-26 04:50:29,061][ INFO] Iters: 310, Total loss: 0.052, Loss x: 0.056, Loss s: 0.087, Loss w_fp: 0.008, Mask: 0.841
[2022-11-26 04:52:56,854][ INFO] Iters: 620, Total loss: 0.052, Loss x: 0.054, Loss s: 0.090, Loss w_fp: 0.008, Mask: 0.841
[2022-11-26 04:55:24,589][ INFO] Iters: 930, Total loss: 0.052, Loss x: 0.055, Loss s: 0.089, Loss w_fp: 0.008, Mask: 0.841
[2022-11-26 04:57:52,093][ INFO] Iters: 1240, Total loss: 0.052, Loss x: 0.054, Loss s: 0.090, Loss w_fp: 0.007, Mask: 0.842
[2022-11-26 05:00:19,320][ INFO] Iters: 1550, Total loss: 0.051, Loss x: 0.054, Loss s: 0.089, Loss w_fp: 0.007, Mask: 0.843
[2022-11-26 05:02:46,526][ INFO] Iters: 1860, Total loss: 0.051, Loss x: 0.054, Loss s: 0.088, Loss w_fp: 0.007, Mask: 0.843
[2022-11-26 05:05:13,866][ INFO] Iters: 2170, Total loss: 0.051, Loss x: 0.055, Loss s: 0.088, Loss w_fp: 0.007, Mask: 0.844
[2022-11-26 05:08:39,956][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.34

[2022-11-26 05:08:39,956][ INFO] ===========> Epoch: 37, LR: 0.0003, Previous best: 75.50
[2022-11-26 05:08:41,038][ INFO] Iters: 0, Total loss: 0.055, Loss x: 0.042, Loss s: 0.135, Loss w_fp: 0.002, Mask: 0.713
[2022-11-26 05:11:08,250][ INFO] Iters: 310, Total loss: 0.050, Loss x: 0.054, Loss s: 0.081, Loss w_fp: 0.008, Mask: 0.837
[2022-11-26 05:13:35,415][ INFO] Iters: 620, Total loss: 0.051, Loss x: 0.053, Loss s: 0.088, Loss w_fp: 0.008, Mask: 0.841
[2022-11-26 05:16:02,777][ INFO] Iters: 930, Total loss: 0.051, Loss x: 0.053, Loss s: 0.091, Loss w_fp: 0.008, Mask: 0.845
[2022-11-26 05:18:29,924][ INFO] Iters: 1240, Total loss: 0.051, Loss x: 0.053, Loss s: 0.091, Loss w_fp: 0.008, Mask: 0.847
[2022-11-26 05:20:57,195][ INFO] Iters: 1550, Total loss: 0.051, Loss x: 0.053, Loss s: 0.091, Loss w_fp: 0.008, Mask: 0.846
[2022-11-26 05:23:24,363][ INFO] Iters: 1860, Total loss: 0.052, Loss x: 0.054, Loss s: 0.091, Loss w_fp: 0.008, Mask: 0.844
[2022-11-26 05:25:51,538][ INFO] Iters: 2170, Total loss: 0.052, Loss x: 0.055, Loss s: 0.090, Loss w_fp: 0.008, Mask: 0.844
[2022-11-26 05:29:15,612][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.90

[2022-11-26 05:29:15,613][ INFO] ===========> Epoch: 38, LR: 0.0003, Previous best: 75.50
[2022-11-26 05:29:16,730][ INFO] Iters: 0, Total loss: 0.072, Loss x: 0.073, Loss s: 0.138, Loss w_fp: 0.003, Mask: 0.870
[2022-11-26 05:31:43,928][ INFO] Iters: 310, Total loss: 0.050, Loss x: 0.052, Loss s: 0.090, Loss w_fp: 0.007, Mask: 0.841
[2022-11-26 05:34:11,015][ INFO] Iters: 620, Total loss: 0.051, Loss x: 0.054, Loss s: 0.087, Loss w_fp: 0.007, Mask: 0.841
[2022-11-26 05:36:38,046][ INFO] Iters: 930, Total loss: 0.051, Loss x: 0.054, Loss s: 0.089, Loss w_fp: 0.007, Mask: 0.842
[2022-11-26 05:39:05,319][ INFO] Iters: 1240, Total loss: 0.051, Loss x: 0.053, Loss s: 0.091, Loss w_fp: 0.008, Mask: 0.843
[2022-11-26 05:41:32,492][ INFO] Iters: 1550, Total loss: 0.051, Loss x: 0.053, Loss s: 0.089, Loss w_fp: 0.007, Mask: 0.844
[2022-11-26 05:43:59,655][ INFO] Iters: 1860, Total loss: 0.051, Loss x: 0.054, Loss s: 0.089, Loss w_fp: 0.007, Mask: 0.846
[2022-11-26 05:46:26,853][ INFO] Iters: 2170, Total loss: 0.051, Loss x: 0.053, Loss s: 0.088, Loss w_fp: 0.007, Mask: 0.848
[2022-11-26 05:49:50,750][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.73

[2022-11-26 05:49:50,750][ INFO] ===========> Epoch: 39, LR: 0.0003, Previous best: 75.50
[2022-11-26 05:49:51,794][ INFO] Iters: 0, Total loss: 0.037, Loss x: 0.023, Loss s: 0.093, Loss w_fp: 0.010, Mask: 0.905
[2022-11-26 05:52:19,005][ INFO] Iters: 310, Total loss: 0.048, Loss x: 0.052, Loss s: 0.083, Loss w_fp: 0.007, Mask: 0.846
[2022-11-26 05:54:46,391][ INFO] Iters: 620, Total loss: 0.051, Loss x: 0.053, Loss s: 0.089, Loss w_fp: 0.008, Mask: 0.844
[2022-11-26 05:57:13,646][ INFO] Iters: 930, Total loss: 0.051, Loss x: 0.053, Loss s: 0.091, Loss w_fp: 0.008, Mask: 0.844
[2022-11-26 05:59:41,046][ INFO] Iters: 1240, Total loss: 0.051, Loss x: 0.054, Loss s: 0.088, Loss w_fp: 0.007, Mask: 0.849
[2022-11-26 06:02:08,181][ INFO] Iters: 1550, Total loss: 0.051, Loss x: 0.053, Loss s: 0.088, Loss w_fp: 0.007, Mask: 0.850
[2022-11-26 06:04:35,452][ INFO] Iters: 1860, Total loss: 0.051, Loss x: 0.053, Loss s: 0.089, Loss w_fp: 0.007, Mask: 0.849
[2022-11-26 06:07:02,763][ INFO] Iters: 2170, Total loss: 0.051, Loss x: 0.054, Loss s: 0.089, Loss w_fp: 0.007, Mask: 0.849
[2022-11-26 06:10:26,666][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.32

[2022-11-26 06:10:26,666][ INFO] ===========> Epoch: 40, LR: 0.0003, Previous best: 75.50
[2022-11-26 06:10:27,756][ INFO] Iters: 0, Total loss: 0.117, Loss x: 0.091, Loss s: 0.276, Loss w_fp: 0.011, Mask: 0.765
[2022-11-26 06:12:54,928][ INFO] Iters: 310, Total loss: 0.050, Loss x: 0.052, Loss s: 0.087, Loss w_fp: 0.009, Mask: 0.846
[2022-11-26 06:15:22,432][ INFO] Iters: 620, Total loss: 0.050, Loss x: 0.052, Loss s: 0.086, Loss w_fp: 0.009, Mask: 0.846
[2022-11-26 06:17:49,693][ INFO] Iters: 930, Total loss: 0.049, Loss x: 0.051, Loss s: 0.087, Loss w_fp: 0.008, Mask: 0.847
[2022-11-26 06:20:17,024][ INFO] Iters: 1240, Total loss: 0.050, Loss x: 0.052, Loss s: 0.090, Loss w_fp: 0.008, Mask: 0.847
[2022-11-26 06:22:44,320][ INFO] Iters: 1550, Total loss: 0.050, Loss x: 0.052, Loss s: 0.088, Loss w_fp: 0.008, Mask: 0.849
[2022-11-26 06:25:11,534][ INFO] Iters: 1860, Total loss: 0.050, Loss x: 0.052, Loss s: 0.089, Loss w_fp: 0.008, Mask: 0.848
[2022-11-26 06:27:38,850][ INFO] Iters: 2170, Total loss: 0.050, Loss x: 0.052, Loss s: 0.089, Loss w_fp: 0.008, Mask: 0.849
[2022-11-26 06:31:02,729][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.43

[2022-11-26 06:31:02,729][ INFO] ===========> Epoch: 41, LR: 0.0003, Previous best: 75.50
[2022-11-26 06:31:03,893][ INFO] Iters: 0, Total loss: 0.033, Loss x: 0.041, Loss s: 0.050, Loss w_fp: 0.001, Mask: 0.920
[2022-11-26 06:33:31,166][ INFO] Iters: 310, Total loss: 0.053, Loss x: 0.055, Loss s: 0.093, Loss w_fp: 0.008, Mask: 0.849
[2022-11-26 06:35:58,148][ INFO] Iters: 620, Total loss: 0.051, Loss x: 0.053, Loss s: 0.088, Loss w_fp: 0.007, Mask: 0.846
[2022-11-26 06:38:24,089][ INFO] Iters: 930, Total loss: 0.050, Loss x: 0.053, Loss s: 0.086, Loss w_fp: 0.007, Mask: 0.848
[2022-11-26 06:40:49,982][ INFO] Iters: 1240, Total loss: 0.049, Loss x: 0.053, Loss s: 0.086, Loss w_fp: 0.007, Mask: 0.850
[2022-11-26 06:43:15,828][ INFO] Iters: 1550, Total loss: 0.049, Loss x: 0.052, Loss s: 0.085, Loss w_fp: 0.007, Mask: 0.851
[2022-11-26 06:45:41,294][ INFO] Iters: 1860, Total loss: 0.050, Loss x: 0.053, Loss s: 0.086, Loss w_fp: 0.007, Mask: 0.850
[2022-11-26 06:48:06,862][ INFO] Iters: 2170, Total loss: 0.049, Loss x: 0.052, Loss s: 0.085, Loss w_fp: 0.007, Mask: 0.852
[2022-11-26 06:51:17,615][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.23

[2022-11-26 06:51:17,615][ INFO] ===========> Epoch: 42, LR: 0.0003, Previous best: 75.50
[2022-11-26 06:51:18,636][ INFO] Iters: 0, Total loss: 0.082, Loss x: 0.122, Loss s: 0.080, Loss w_fp: 0.003, Mask: 0.934
[2022-11-26 06:53:44,099][ INFO] Iters: 310, Total loss: 0.046, Loss x: 0.051, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.860
[2022-11-26 06:56:09,596][ INFO] Iters: 620, Total loss: 0.047, Loss x: 0.050, Loss s: 0.080, Loss w_fp: 0.006, Mask: 0.861
[2022-11-26 06:58:35,110][ INFO] Iters: 930, Total loss: 0.047, Loss x: 0.050, Loss s: 0.083, Loss w_fp: 0.007, Mask: 0.857
[2022-11-26 07:01:00,495][ INFO] Iters: 1240, Total loss: 0.048, Loss x: 0.050, Loss s: 0.083, Loss w_fp: 0.007, Mask: 0.855
[2022-11-26 07:03:25,899][ INFO] Iters: 1550, Total loss: 0.048, Loss x: 0.051, Loss s: 0.084, Loss w_fp: 0.007, Mask: 0.857
[2022-11-26 07:05:51,458][ INFO] Iters: 1860, Total loss: 0.048, Loss x: 0.051, Loss s: 0.084, Loss w_fp: 0.007, Mask: 0.857
[2022-11-26 07:08:16,978][ INFO] Iters: 2170, Total loss: 0.048, Loss x: 0.051, Loss s: 0.083, Loss w_fp: 0.007, Mask: 0.857
[2022-11-26 07:11:28,024][ INFO] ***** Evaluation original ***** >>>> meanIOU: 73.96

[2022-11-26 07:11:28,024][ INFO] ===========> Epoch: 43, LR: 0.0002, Previous best: 75.50
[2022-11-26 07:11:28,965][ INFO] Iters: 0, Total loss: 0.066, Loss x: 0.053, Loss s: 0.147, Loss w_fp: 0.008, Mask: 0.662
[2022-11-26 07:13:54,466][ INFO] Iters: 310, Total loss: 0.050, Loss x: 0.052, Loss s: 0.087, Loss w_fp: 0.008, Mask: 0.853
[2022-11-26 07:16:19,959][ INFO] Iters: 620, Total loss: 0.049, Loss x: 0.052, Loss s: 0.083, Loss w_fp: 0.007, Mask: 0.855
[2022-11-26 07:18:45,415][ INFO] Iters: 930, Total loss: 0.049, Loss x: 0.052, Loss s: 0.083, Loss w_fp: 0.007, Mask: 0.856
[2022-11-26 07:21:10,911][ INFO] Iters: 1240, Total loss: 0.049, Loss x: 0.051, Loss s: 0.085, Loss w_fp: 0.007, Mask: 0.856
[2022-11-26 07:23:36,492][ INFO] Iters: 1550, Total loss: 0.049, Loss x: 0.052, Loss s: 0.085, Loss w_fp: 0.007, Mask: 0.856
[2022-11-26 07:26:02,107][ INFO] Iters: 1860, Total loss: 0.049, Loss x: 0.052, Loss s: 0.085, Loss w_fp: 0.007, Mask: 0.854
[2022-11-26 07:28:27,663][ INFO] Iters: 2170, Total loss: 0.049, Loss x: 0.052, Loss s: 0.085, Loss w_fp: 0.007, Mask: 0.853
[2022-11-26 07:31:38,971][ INFO] ***** Evaluation original ***** >>>> meanIOU: 73.28

[2022-11-26 07:31:38,971][ INFO] ===========> Epoch: 44, LR: 0.0002, Previous best: 75.50
[2022-11-26 07:31:39,927][ INFO] Iters: 0, Total loss: 0.028, Loss x: 0.043, Loss s: 0.023, Loss w_fp: 0.003, Mask: 0.564
[2022-11-26 07:34:05,276][ INFO] Iters: 310, Total loss: 0.051, Loss x: 0.052, Loss s: 0.092, Loss w_fp: 0.007, Mask: 0.847
[2022-11-26 07:36:30,847][ INFO] Iters: 620, Total loss: 0.050, Loss x: 0.051, Loss s: 0.092, Loss w_fp: 0.007, Mask: 0.848
[2022-11-26 07:38:56,425][ INFO] Iters: 930, Total loss: 0.049, Loss x: 0.051, Loss s: 0.087, Loss w_fp: 0.007, Mask: 0.852
[2022-11-26 07:41:21,879][ INFO] Iters: 1240, Total loss: 0.048, Loss x: 0.051, Loss s: 0.085, Loss w_fp: 0.007, Mask: 0.852
[2022-11-26 07:43:47,336][ INFO] Iters: 1550, Total loss: 0.049, Loss x: 0.050, Loss s: 0.087, Loss w_fp: 0.007, Mask: 0.852
[2022-11-26 07:46:12,760][ INFO] Iters: 1860, Total loss: 0.049, Loss x: 0.051, Loss s: 0.087, Loss w_fp: 0.007, Mask: 0.853
[2022-11-26 07:48:38,283][ INFO] Iters: 2170, Total loss: 0.049, Loss x: 0.051, Loss s: 0.086, Loss w_fp: 0.007, Mask: 0.853
[2022-11-26 07:51:50,323][ INFO] ***** Evaluation original ***** >>>> meanIOU: 73.96

[2022-11-26 07:51:50,323][ INFO] ===========> Epoch: 45, LR: 0.0002, Previous best: 75.50
[2022-11-26 07:51:51,311][ INFO] Iters: 0, Total loss: 0.048, Loss x: 0.041, Loss s: 0.105, Loss w_fp: 0.005, Mask: 0.853
[2022-11-26 07:54:16,848][ INFO] Iters: 310, Total loss: 0.049, Loss x: 0.051, Loss s: 0.088, Loss w_fp: 0.007, Mask: 0.856
[2022-11-26 07:56:42,407][ INFO] Iters: 620, Total loss: 0.048, Loss x: 0.050, Loss s: 0.087, Loss w_fp: 0.007, Mask: 0.860
[2022-11-26 07:59:07,930][ INFO] Iters: 930, Total loss: 0.049, Loss x: 0.050, Loss s: 0.088, Loss w_fp: 0.007, Mask: 0.860
[2022-11-26 08:01:33,449][ INFO] Iters: 1240, Total loss: 0.049, Loss x: 0.050, Loss s: 0.087, Loss w_fp: 0.007, Mask: 0.860
[2022-11-26 08:03:59,016][ INFO] Iters: 1550, Total loss: 0.049, Loss x: 0.050, Loss s: 0.087, Loss w_fp: 0.007, Mask: 0.859
[2022-11-26 08:06:24,576][ INFO] Iters: 1860, Total loss: 0.049, Loss x: 0.050, Loss s: 0.087, Loss w_fp: 0.008, Mask: 0.858
[2022-11-26 08:08:50,084][ INFO] Iters: 2170, Total loss: 0.048, Loss x: 0.050, Loss s: 0.086, Loss w_fp: 0.007, Mask: 0.858
[2022-11-26 08:12:01,618][ INFO] ***** Evaluation original ***** >>>> meanIOU: 73.92

[2022-11-26 08:12:01,619][ INFO] ===========> Epoch: 46, LR: 0.0002, Previous best: 75.50
[2022-11-26 08:12:02,567][ INFO] Iters: 0, Total loss: 0.041, Loss x: 0.043, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.894
[2022-11-26 08:14:28,025][ INFO] Iters: 310, Total loss: 0.048, Loss x: 0.049, Loss s: 0.087, Loss w_fp: 0.007, Mask: 0.861
[2022-11-26 08:16:53,576][ INFO] Iters: 620, Total loss: 0.048, Loss x: 0.049, Loss s: 0.086, Loss w_fp: 0.007, Mask: 0.859
[2022-11-26 08:19:19,101][ INFO] Iters: 930, Total loss: 0.048, Loss x: 0.049, Loss s: 0.085, Loss w_fp: 0.007, Mask: 0.859
[2022-11-26 08:21:44,638][ INFO] Iters: 1240, Total loss: 0.047, Loss x: 0.049, Loss s: 0.083, Loss w_fp: 0.007, Mask: 0.859
[2022-11-26 08:24:10,203][ INFO] Iters: 1550, Total loss: 0.047, Loss x: 0.049, Loss s: 0.081, Loss w_fp: 0.007, Mask: 0.859
[2022-11-26 08:26:35,756][ INFO] Iters: 1860, Total loss: 0.047, Loss x: 0.049, Loss s: 0.081, Loss w_fp: 0.007, Mask: 0.860
[2022-11-26 08:29:01,371][ INFO] Iters: 2170, Total loss: 0.047, Loss x: 0.050, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.858
[2022-11-26 08:32:11,983][ INFO] ***** Evaluation original ***** >>>> meanIOU: 75.14

[2022-11-26 08:32:11,983][ INFO] ===========> Epoch: 47, LR: 0.0002, Previous best: 75.50
[2022-11-26 08:32:12,975][ INFO] Iters: 0, Total loss: 0.058, Loss x: 0.101, Loss s: 0.024, Loss w_fp: 0.004, Mask: 0.937
[2022-11-26 08:34:38,364][ INFO] Iters: 310, Total loss: 0.047, Loss x: 0.050, Loss s: 0.080, Loss w_fp: 0.006, Mask: 0.860
[2022-11-26 08:37:03,967][ INFO] Iters: 620, Total loss: 0.048, Loss x: 0.050, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.859
[2022-11-26 08:39:29,659][ INFO] Iters: 930, Total loss: 0.047, Loss x: 0.050, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.859
[2022-11-26 08:41:55,223][ INFO] Iters: 1240, Total loss: 0.047, Loss x: 0.050, Loss s: 0.081, Loss w_fp: 0.007, Mask: 0.860
[2022-11-26 08:44:20,985][ INFO] Iters: 1550, Total loss: 0.047, Loss x: 0.049, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.860
[2022-11-26 08:46:46,526][ INFO] Iters: 1860, Total loss: 0.048, Loss x: 0.050, Loss s: 0.083, Loss w_fp: 0.007, Mask: 0.860
[2022-11-26 08:49:12,020][ INFO] Iters: 2170, Total loss: 0.047, Loss x: 0.050, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.858
[2022-11-26 08:52:22,672][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.65

[2022-11-26 08:52:22,672][ INFO] ===========> Epoch: 48, LR: 0.0002, Previous best: 75.50
[2022-11-26 08:52:23,697][ INFO] Iters: 0, Total loss: 0.048, Loss x: 0.050, Loss s: 0.085, Loss w_fp: 0.007, Mask: 0.909
[2022-11-26 08:54:49,490][ INFO] Iters: 310, Total loss: 0.046, Loss x: 0.050, Loss s: 0.078, Loss w_fp: 0.006, Mask: 0.864
[2022-11-26 08:57:15,084][ INFO] Iters: 620, Total loss: 0.046, Loss x: 0.050, Loss s: 0.076, Loss w_fp: 0.007, Mask: 0.863
[2022-11-26 08:59:40,743][ INFO] Iters: 930, Total loss: 0.048, Loss x: 0.052, Loss s: 0.081, Loss w_fp: 0.007, Mask: 0.857
[2022-11-26 09:02:06,297][ INFO] Iters: 1240, Total loss: 0.049, Loss x: 0.051, Loss s: 0.085, Loss w_fp: 0.007, Mask: 0.855
[2022-11-26 09:04:32,002][ INFO] Iters: 1550, Total loss: 0.048, Loss x: 0.051, Loss s: 0.085, Loss w_fp: 0.007, Mask: 0.854
[2022-11-26 09:06:57,657][ INFO] Iters: 1860, Total loss: 0.048, Loss x: 0.050, Loss s: 0.086, Loss w_fp: 0.007, Mask: 0.854
[2022-11-26 09:09:23,299][ INFO] Iters: 2170, Total loss: 0.049, Loss x: 0.050, Loss s: 0.086, Loss w_fp: 0.007, Mask: 0.855
[2022-11-26 09:12:35,224][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.26

[2022-11-26 09:12:35,225][ INFO] ===========> Epoch: 49, LR: 0.0002, Previous best: 75.50
[2022-11-26 09:12:36,191][ INFO] Iters: 0, Total loss: 0.043, Loss x: 0.026, Loss s: 0.118, Loss w_fp: 0.003, Mask: 0.927
[2022-11-26 09:15:01,760][ INFO] Iters: 310, Total loss: 0.046, Loss x: 0.049, Loss s: 0.081, Loss w_fp: 0.006, Mask: 0.862
[2022-11-26 09:17:27,337][ INFO] Iters: 620, Total loss: 0.047, Loss x: 0.049, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.861
[2022-11-26 09:19:52,931][ INFO] Iters: 930, Total loss: 0.047, Loss x: 0.049, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.861
[2022-11-26 09:22:18,609][ INFO] Iters: 1240, Total loss: 0.047, Loss x: 0.049, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.861
[2022-11-26 09:24:44,144][ INFO] Iters: 1550, Total loss: 0.047, Loss x: 0.049, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.861
[2022-11-26 09:27:09,714][ INFO] Iters: 1860, Total loss: 0.047, Loss x: 0.049, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.861
[2022-11-26 09:29:35,470][ INFO] Iters: 2170, Total loss: 0.047, Loss x: 0.049, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.863
[2022-11-26 09:32:47,801][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.28

[2022-11-26 09:32:47,801][ INFO] ===========> Epoch: 50, LR: 0.0002, Previous best: 75.50
[2022-11-26 09:32:48,807][ INFO] Iters: 0, Total loss: 0.040, Loss x: 0.063, Loss s: 0.032, Loss w_fp: 0.001, Mask: 0.884
[2022-11-26 09:35:14,531][ INFO] Iters: 310, Total loss: 0.046, Loss x: 0.049, Loss s: 0.079, Loss w_fp: 0.007, Mask: 0.868
[2022-11-26 09:37:40,257][ INFO] Iters: 620, Total loss: 0.046, Loss x: 0.048, Loss s: 0.083, Loss w_fp: 0.007, Mask: 0.862
[2022-11-26 09:40:06,319][ INFO] Iters: 930, Total loss: 0.046, Loss x: 0.047, Loss s: 0.083, Loss w_fp: 0.007, Mask: 0.861
[2022-11-26 09:42:32,331][ INFO] Iters: 1240, Total loss: 0.047, Loss x: 0.048, Loss s: 0.085, Loss w_fp: 0.007, Mask: 0.860
[2022-11-26 09:44:58,249][ INFO] Iters: 1550, Total loss: 0.047, Loss x: 0.048, Loss s: 0.085, Loss w_fp: 0.007, Mask: 0.860
[2022-11-26 09:47:24,310][ INFO] Iters: 1860, Total loss: 0.046, Loss x: 0.047, Loss s: 0.084, Loss w_fp: 0.007, Mask: 0.860
[2022-11-26 09:49:50,410][ INFO] Iters: 2170, Total loss: 0.047, Loss x: 0.047, Loss s: 0.086, Loss w_fp: 0.007, Mask: 0.860
[2022-11-26 09:53:03,597][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.61

[2022-11-26 09:53:03,597][ INFO] ===========> Epoch: 51, LR: 0.0002, Previous best: 75.50
[2022-11-26 09:53:04,569][ INFO] Iters: 0, Total loss: 0.028, Loss x: 0.046, Loss s: 0.016, Loss w_fp: 0.003, Mask: 0.708
[2022-11-26 09:55:30,218][ INFO] Iters: 310, Total loss: 0.048, Loss x: 0.049, Loss s: 0.087, Loss w_fp: 0.007, Mask: 0.858
[2022-11-26 09:57:56,160][ INFO] Iters: 620, Total loss: 0.049, Loss x: 0.050, Loss s: 0.088, Loss w_fp: 0.007, Mask: 0.855
[2022-11-26 10:00:21,889][ INFO] Iters: 930, Total loss: 0.048, Loss x: 0.049, Loss s: 0.087, Loss w_fp: 0.007, Mask: 0.857
[2022-11-26 10:02:47,769][ INFO] Iters: 1240, Total loss: 0.047, Loss x: 0.048, Loss s: 0.084, Loss w_fp: 0.007, Mask: 0.857
[2022-11-26 10:05:13,434][ INFO] Iters: 1550, Total loss: 0.046, Loss x: 0.048, Loss s: 0.081, Loss w_fp: 0.007, Mask: 0.859
[2022-11-26 10:07:39,109][ INFO] Iters: 1860, Total loss: 0.046, Loss x: 0.047, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.861
[2022-11-26 10:10:04,923][ INFO] Iters: 2170, Total loss: 0.046, Loss x: 0.048, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.861
[2022-11-26 10:13:17,711][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.48

[2022-11-26 10:13:17,711][ INFO] ===========> Epoch: 52, LR: 0.0002, Previous best: 75.50
[2022-11-26 10:13:18,711][ INFO] Iters: 0, Total loss: 0.050, Loss x: 0.068, Loss s: 0.056, Loss w_fp: 0.008, Mask: 0.751
[2022-11-26 10:15:44,407][ INFO] Iters: 310, Total loss: 0.045, Loss x: 0.048, Loss s: 0.077, Loss w_fp: 0.006, Mask: 0.869
[2022-11-26 10:18:10,372][ INFO] Iters: 620, Total loss: 0.046, Loss x: 0.048, Loss s: 0.081, Loss w_fp: 0.006, Mask: 0.862
[2022-11-26 10:20:36,209][ INFO] Iters: 930, Total loss: 0.046, Loss x: 0.048, Loss s: 0.082, Loss w_fp: 0.006, Mask: 0.862
[2022-11-26 10:23:01,994][ INFO] Iters: 1240, Total loss: 0.046, Loss x: 0.048, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.861
[2022-11-26 10:25:27,779][ INFO] Iters: 1550, Total loss: 0.046, Loss x: 0.048, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.861
[2022-11-26 10:27:53,873][ INFO] Iters: 1860, Total loss: 0.047, Loss x: 0.049, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.862
[2022-11-26 10:30:19,787][ INFO] Iters: 2170, Total loss: 0.046, Loss x: 0.049, Loss s: 0.081, Loss w_fp: 0.007, Mask: 0.863
[2022-11-26 10:33:32,393][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.49

[2022-11-26 10:33:32,393][ INFO] ===========> Epoch: 53, LR: 0.0002, Previous best: 75.50
[2022-11-26 10:33:33,332][ INFO] Iters: 0, Total loss: 0.026, Loss x: 0.028, Loss s: 0.044, Loss w_fp: 0.003, Mask: 0.903
[2022-11-26 10:35:58,714][ INFO] Iters: 310, Total loss: 0.045, Loss x: 0.048, Loss s: 0.078, Loss w_fp: 0.006, Mask: 0.869
[2022-11-26 10:38:24,337][ INFO] Iters: 620, Total loss: 0.045, Loss x: 0.047, Loss s: 0.080, Loss w_fp: 0.006, Mask: 0.868
[2022-11-26 10:40:50,150][ INFO] Iters: 930, Total loss: 0.045, Loss x: 0.046, Loss s: 0.081, Loss w_fp: 0.006, Mask: 0.865
[2022-11-26 10:43:16,044][ INFO] Iters: 1240, Total loss: 0.045, Loss x: 0.047, Loss s: 0.081, Loss w_fp: 0.007, Mask: 0.865
[2022-11-26 10:45:41,914][ INFO] Iters: 1550, Total loss: 0.045, Loss x: 0.047, Loss s: 0.080, Loss w_fp: 0.007, Mask: 0.865
[2022-11-26 10:48:07,904][ INFO] Iters: 1860, Total loss: 0.045, Loss x: 0.047, Loss s: 0.079, Loss w_fp: 0.007, Mask: 0.864
[2022-11-26 10:50:33,652][ INFO] Iters: 2170, Total loss: 0.045, Loss x: 0.047, Loss s: 0.079, Loss w_fp: 0.007, Mask: 0.864
[2022-11-26 10:53:45,375][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.35

[2022-11-26 10:53:45,375][ INFO] ===========> Epoch: 54, LR: 0.0002, Previous best: 75.50
[2022-11-26 10:53:46,323][ INFO] Iters: 0, Total loss: 0.032, Loss x: 0.040, Loss s: 0.045, Loss w_fp: 0.002, Mask: 0.881
[2022-11-26 10:56:11,853][ INFO] Iters: 310, Total loss: 0.045, Loss x: 0.048, Loss s: 0.077, Loss w_fp: 0.007, Mask: 0.862
[2022-11-26 10:58:37,966][ INFO] Iters: 620, Total loss: 0.045, Loss x: 0.047, Loss s: 0.079, Loss w_fp: 0.007, Mask: 0.862
[2022-11-26 11:01:04,017][ INFO] Iters: 930, Total loss: 0.045, Loss x: 0.047, Loss s: 0.080, Loss w_fp: 0.007, Mask: 0.865
[2022-11-26 11:03:30,321][ INFO] Iters: 1240, Total loss: 0.044, Loss x: 0.046, Loss s: 0.078, Loss w_fp: 0.007, Mask: 0.866
[2022-11-26 11:05:56,510][ INFO] Iters: 1550, Total loss: 0.044, Loss x: 0.046, Loss s: 0.078, Loss w_fp: 0.007, Mask: 0.867
[2022-11-26 11:08:22,646][ INFO] Iters: 1860, Total loss: 0.045, Loss x: 0.047, Loss s: 0.080, Loss w_fp: 0.007, Mask: 0.865
[2022-11-26 11:10:48,504][ INFO] Iters: 2170, Total loss: 0.045, Loss x: 0.047, Loss s: 0.081, Loss w_fp: 0.007, Mask: 0.864
[2022-11-26 11:14:01,102][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.29

[2022-11-26 11:14:01,103][ INFO] ===========> Epoch: 55, LR: 0.0002, Previous best: 75.50
[2022-11-26 11:14:02,100][ INFO] Iters: 0, Total loss: 0.025, Loss x: 0.034, Loss s: 0.030, Loss w_fp: 0.001, Mask: 0.925
[2022-11-26 11:16:27,599][ INFO] Iters: 310, Total loss: 0.045, Loss x: 0.048, Loss s: 0.077, Loss w_fp: 0.006, Mask: 0.861
[2022-11-26 11:18:53,285][ INFO] Iters: 620, Total loss: 0.045, Loss x: 0.048, Loss s: 0.076, Loss w_fp: 0.006, Mask: 0.861
[2022-11-26 11:21:18,824][ INFO] Iters: 930, Total loss: 0.045, Loss x: 0.047, Loss s: 0.078, Loss w_fp: 0.007, Mask: 0.862
[2022-11-26 11:23:44,518][ INFO] Iters: 1240, Total loss: 0.044, Loss x: 0.047, Loss s: 0.076, Loss w_fp: 0.007, Mask: 0.863
[2022-11-26 11:26:10,227][ INFO] Iters: 1550, Total loss: 0.045, Loss x: 0.047, Loss s: 0.079, Loss w_fp: 0.007, Mask: 0.862
[2022-11-26 11:28:35,855][ INFO] Iters: 1860, Total loss: 0.045, Loss x: 0.047, Loss s: 0.081, Loss w_fp: 0.007, Mask: 0.863
[2022-11-26 11:31:01,421][ INFO] Iters: 2170, Total loss: 0.045, Loss x: 0.047, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.864
[2022-11-26 11:34:13,650][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.33

[2022-11-26 11:34:13,650][ INFO] ===========> Epoch: 56, LR: 0.0002, Previous best: 75.50
[2022-11-26 11:34:14,687][ INFO] Iters: 0, Total loss: 0.035, Loss x: 0.037, Loss s: 0.061, Loss w_fp: 0.005, Mask: 0.893
[2022-11-26 11:36:40,506][ INFO] Iters: 310, Total loss: 0.046, Loss x: 0.048, Loss s: 0.083, Loss w_fp: 0.006, Mask: 0.865
[2022-11-26 11:39:06,332][ INFO] Iters: 620, Total loss: 0.045, Loss x: 0.047, Loss s: 0.081, Loss w_fp: 0.007, Mask: 0.865
[2022-11-26 11:41:32,073][ INFO] Iters: 930, Total loss: 0.045, Loss x: 0.046, Loss s: 0.083, Loss w_fp: 0.006, Mask: 0.866
[2022-11-26 11:43:57,916][ INFO] Iters: 1240, Total loss: 0.045, Loss x: 0.046, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.867
[2022-11-26 11:46:23,908][ INFO] Iters: 1550, Total loss: 0.045, Loss x: 0.046, Loss s: 0.081, Loss w_fp: 0.007, Mask: 0.867
[2022-11-26 11:48:49,777][ INFO] Iters: 1860, Total loss: 0.045, Loss x: 0.046, Loss s: 0.080, Loss w_fp: 0.007, Mask: 0.867
[2022-11-26 11:51:15,520][ INFO] Iters: 2170, Total loss: 0.045, Loss x: 0.046, Loss s: 0.080, Loss w_fp: 0.007, Mask: 0.868
[2022-11-26 11:54:27,968][ INFO] ***** Evaluation original ***** >>>> meanIOU: 73.50

[2022-11-26 11:54:27,968][ INFO] ===========> Epoch: 57, LR: 0.0002, Previous best: 75.50
[2022-11-26 11:54:28,900][ INFO] Iters: 0, Total loss: 0.038, Loss x: 0.045, Loss s: 0.058, Loss w_fp: 0.002, Mask: 0.883
[2022-11-26 11:56:54,730][ INFO] Iters: 310, Total loss: 0.044, Loss x: 0.046, Loss s: 0.078, Loss w_fp: 0.005, Mask: 0.863
[2022-11-26 11:59:20,621][ INFO] Iters: 620, Total loss: 0.045, Loss x: 0.046, Loss s: 0.081, Loss w_fp: 0.006, Mask: 0.859
[2022-11-26 12:01:46,475][ INFO] Iters: 930, Total loss: 0.045, Loss x: 0.046, Loss s: 0.081, Loss w_fp: 0.007, Mask: 0.860
[2022-11-26 12:04:12,280][ INFO] Iters: 1240, Total loss: 0.044, Loss x: 0.046, Loss s: 0.080, Loss w_fp: 0.006, Mask: 0.863
[2022-11-26 12:06:38,034][ INFO] Iters: 1550, Total loss: 0.045, Loss x: 0.046, Loss s: 0.081, Loss w_fp: 0.007, Mask: 0.865
[2022-11-26 12:09:03,909][ INFO] Iters: 1860, Total loss: 0.044, Loss x: 0.046, Loss s: 0.080, Loss w_fp: 0.007, Mask: 0.865
[2022-11-26 12:11:30,059][ INFO] Iters: 2170, Total loss: 0.044, Loss x: 0.046, Loss s: 0.079, Loss w_fp: 0.007, Mask: 0.865
[2022-11-26 12:14:42,647][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.24

[2022-11-26 12:14:42,647][ INFO] ===========> Epoch: 58, LR: 0.0002, Previous best: 75.50
[2022-11-26 12:14:43,629][ INFO] Iters: 0, Total loss: 0.038, Loss x: 0.054, Loss s: 0.042, Loss w_fp: 0.003, Mask: 0.819
[2022-11-26 12:17:09,267][ INFO] Iters: 310, Total loss: 0.043, Loss x: 0.045, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.869
[2022-11-26 12:19:35,127][ INFO] Iters: 620, Total loss: 0.043, Loss x: 0.046, Loss s: 0.074, Loss w_fp: 0.006, Mask: 0.869
[2022-11-26 12:22:01,166][ INFO] Iters: 930, Total loss: 0.044, Loss x: 0.046, Loss s: 0.077, Loss w_fp: 0.006, Mask: 0.870
[2022-11-26 12:24:27,059][ INFO] Iters: 1240, Total loss: 0.044, Loss x: 0.046, Loss s: 0.077, Loss w_fp: 0.006, Mask: 0.871
[2022-11-26 12:26:53,013][ INFO] Iters: 1550, Total loss: 0.044, Loss x: 0.046, Loss s: 0.077, Loss w_fp: 0.006, Mask: 0.871
[2022-11-26 12:29:19,101][ INFO] Iters: 1860, Total loss: 0.044, Loss x: 0.046, Loss s: 0.077, Loss w_fp: 0.006, Mask: 0.871
[2022-11-26 12:31:45,082][ INFO] Iters: 2170, Total loss: 0.044, Loss x: 0.046, Loss s: 0.078, Loss w_fp: 0.007, Mask: 0.870
[2022-11-26 12:34:56,832][ INFO] ***** Evaluation original ***** >>>> meanIOU: 75.20

[2022-11-26 12:34:56,832][ INFO] ===========> Epoch: 59, LR: 0.0002, Previous best: 75.50
[2022-11-26 12:34:57,782][ INFO] Iters: 0, Total loss: 0.026, Loss x: 0.026, Loss s: 0.041, Loss w_fp: 0.012, Mask: 0.897
[2022-11-26 12:37:23,437][ INFO] Iters: 310, Total loss: 0.042, Loss x: 0.043, Loss s: 0.077, Loss w_fp: 0.007, Mask: 0.874
[2022-11-26 12:39:49,310][ INFO] Iters: 620, Total loss: 0.043, Loss x: 0.044, Loss s: 0.077, Loss w_fp: 0.007, Mask: 0.872
[2022-11-26 12:42:15,185][ INFO] Iters: 930, Total loss: 0.043, Loss x: 0.045, Loss s: 0.077, Loss w_fp: 0.006, Mask: 0.872
[2022-11-26 12:44:41,006][ INFO] Iters: 1240, Total loss: 0.043, Loss x: 0.044, Loss s: 0.078, Loss w_fp: 0.006, Mask: 0.871
[2022-11-26 12:47:06,909][ INFO] Iters: 1550, Total loss: 0.043, Loss x: 0.045, Loss s: 0.078, Loss w_fp: 0.006, Mask: 0.870
[2022-11-26 12:49:32,478][ INFO] Iters: 1860, Total loss: 0.044, Loss x: 0.045, Loss s: 0.079, Loss w_fp: 0.007, Mask: 0.870
[2022-11-26 12:51:57,936][ INFO] Iters: 2170, Total loss: 0.044, Loss x: 0.045, Loss s: 0.079, Loss w_fp: 0.007, Mask: 0.869
[2022-11-26 12:55:09,624][ INFO] ***** Evaluation original ***** >>>> meanIOU: 75.18

[2022-11-26 12:55:09,624][ INFO] ===========> Epoch: 60, LR: 0.0001, Previous best: 75.50
[2022-11-26 12:55:10,612][ INFO] Iters: 0, Total loss: 0.045, Loss x: 0.033, Loss s: 0.108, Loss w_fp: 0.006, Mask: 0.899
[2022-11-26 12:57:36,220][ INFO] Iters: 310, Total loss: 0.042, Loss x: 0.044, Loss s: 0.072, Loss w_fp: 0.006, Mask: 0.862
[2022-11-26 13:00:02,062][ INFO] Iters: 620, Total loss: 0.042, Loss x: 0.045, Loss s: 0.072, Loss w_fp: 0.006, Mask: 0.866
[2022-11-26 13:02:27,916][ INFO] Iters: 930, Total loss: 0.042, Loss x: 0.045, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.866
[2022-11-26 13:04:53,793][ INFO] Iters: 1240, Total loss: 0.042, Loss x: 0.044, Loss s: 0.074, Loss w_fp: 0.006, Mask: 0.868
[2022-11-26 13:07:19,528][ INFO] Iters: 1550, Total loss: 0.043, Loss x: 0.045, Loss s: 0.074, Loss w_fp: 0.007, Mask: 0.868
[2022-11-26 13:09:45,282][ INFO] Iters: 1860, Total loss: 0.043, Loss x: 0.045, Loss s: 0.074, Loss w_fp: 0.007, Mask: 0.869
[2022-11-26 13:12:11,231][ INFO] Iters: 2170, Total loss: 0.042, Loss x: 0.045, Loss s: 0.074, Loss w_fp: 0.007, Mask: 0.870
[2022-11-26 13:15:22,930][ INFO] ***** Evaluation original ***** >>>> meanIOU: 73.15

[2022-11-26 13:15:22,931][ INFO] ===========> Epoch: 61, LR: 0.0001, Previous best: 75.50
[2022-11-26 13:15:23,886][ INFO] Iters: 0, Total loss: 0.044, Loss x: 0.063, Loss s: 0.046, Loss w_fp: 0.004, Mask: 0.843
[2022-11-26 13:17:49,647][ INFO] Iters: 310, Total loss: 0.046, Loss x: 0.046, Loss s: 0.087, Loss w_fp: 0.007, Mask: 0.869
[2022-11-26 13:20:15,348][ INFO] Iters: 620, Total loss: 0.045, Loss x: 0.046, Loss s: 0.081, Loss w_fp: 0.007, Mask: 0.868
[2022-11-26 13:22:41,402][ INFO] Iters: 930, Total loss: 0.045, Loss x: 0.046, Loss s: 0.080, Loss w_fp: 0.007, Mask: 0.867
[2022-11-26 13:25:06,971][ INFO] Iters: 1240, Total loss: 0.044, Loss x: 0.046, Loss s: 0.078, Loss w_fp: 0.006, Mask: 0.868
[2022-11-26 13:27:33,111][ INFO] Iters: 1550, Total loss: 0.044, Loss x: 0.045, Loss s: 0.079, Loss w_fp: 0.006, Mask: 0.869
[2022-11-26 13:29:58,903][ INFO] Iters: 1860, Total loss: 0.043, Loss x: 0.045, Loss s: 0.077, Loss w_fp: 0.007, Mask: 0.870
[2022-11-26 13:32:24,713][ INFO] Iters: 2170, Total loss: 0.043, Loss x: 0.045, Loss s: 0.077, Loss w_fp: 0.007, Mask: 0.871
[2022-11-26 13:35:36,523][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.47

[2022-11-26 13:35:36,523][ INFO] ===========> Epoch: 62, LR: 0.0001, Previous best: 75.50
[2022-11-26 13:35:37,471][ INFO] Iters: 0, Total loss: 0.014, Loss x: 0.004, Loss s: 0.034, Loss w_fp: 0.013, Mask: 0.875
[2022-11-26 13:38:03,005][ INFO] Iters: 310, Total loss: 0.043, Loss x: 0.046, Loss s: 0.074, Loss w_fp: 0.007, Mask: 0.869
[2022-11-26 13:40:28,712][ INFO] Iters: 620, Total loss: 0.044, Loss x: 0.045, Loss s: 0.080, Loss w_fp: 0.007, Mask: 0.865
[2022-11-26 13:42:54,664][ INFO] Iters: 930, Total loss: 0.044, Loss x: 0.045, Loss s: 0.079, Loss w_fp: 0.007, Mask: 0.870
[2022-11-26 13:45:20,332][ INFO] Iters: 1240, Total loss: 0.043, Loss x: 0.044, Loss s: 0.077, Loss w_fp: 0.007, Mask: 0.871
[2022-11-26 13:47:46,133][ INFO] Iters: 1550, Total loss: 0.043, Loss x: 0.044, Loss s: 0.078, Loss w_fp: 0.007, Mask: 0.870
[2022-11-26 13:50:11,919][ INFO] Iters: 1860, Total loss: 0.043, Loss x: 0.044, Loss s: 0.078, Loss w_fp: 0.007, Mask: 0.871
[2022-11-26 13:52:37,632][ INFO] Iters: 2170, Total loss: 0.043, Loss x: 0.044, Loss s: 0.078, Loss w_fp: 0.007, Mask: 0.870
[2022-11-26 13:55:49,728][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.41

[2022-11-26 13:55:49,728][ INFO] ===========> Epoch: 63, LR: 0.0001, Previous best: 75.50
[2022-11-26 13:55:50,670][ INFO] Iters: 0, Total loss: 0.029, Loss x: 0.051, Loss s: 0.015, Loss w_fp: 0.001, Mask: 0.958
[2022-11-26 13:58:16,224][ INFO] Iters: 310, Total loss: 0.042, Loss x: 0.044, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.876
[2022-11-26 14:00:42,107][ INFO] Iters: 620, Total loss: 0.043, Loss x: 0.045, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.873
[2022-11-26 14:03:07,846][ INFO] Iters: 930, Total loss: 0.043, Loss x: 0.045, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.871
[2022-11-26 14:05:33,621][ INFO] Iters: 1240, Total loss: 0.043, Loss x: 0.045, Loss s: 0.076, Loss w_fp: 0.006, Mask: 0.869
[2022-11-26 14:07:59,267][ INFO] Iters: 1550, Total loss: 0.043, Loss x: 0.045, Loss s: 0.076, Loss w_fp: 0.007, Mask: 0.870
[2022-11-26 14:10:24,949][ INFO] Iters: 1860, Total loss: 0.043, Loss x: 0.044, Loss s: 0.076, Loss w_fp: 0.007, Mask: 0.871
[2022-11-26 14:12:50,610][ INFO] Iters: 2170, Total loss: 0.043, Loss x: 0.045, Loss s: 0.075, Loss w_fp: 0.007, Mask: 0.870
[2022-11-26 14:16:01,629][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.38

[2022-11-26 14:16:01,630][ INFO] ===========> Epoch: 64, LR: 0.0001, Previous best: 75.50
[2022-11-26 14:16:02,649][ INFO] Iters: 0, Total loss: 0.043, Loss x: 0.064, Loss s: 0.031, Loss w_fp: 0.011, Mask: 0.900
[2022-11-26 14:18:28,307][ INFO] Iters: 310, Total loss: 0.042, Loss x: 0.043, Loss s: 0.074, Loss w_fp: 0.007, Mask: 0.869
[2022-11-26 14:20:54,204][ INFO] Iters: 620, Total loss: 0.041, Loss x: 0.043, Loss s: 0.071, Loss w_fp: 0.007, Mask: 0.874
[2022-11-26 14:23:20,072][ INFO] Iters: 930, Total loss: 0.041, Loss x: 0.044, Loss s: 0.072, Loss w_fp: 0.006, Mask: 0.873
[2022-11-26 14:25:45,909][ INFO] Iters: 1240, Total loss: 0.042, Loss x: 0.044, Loss s: 0.072, Loss w_fp: 0.006, Mask: 0.872
[2022-11-26 14:28:11,749][ INFO] Iters: 1550, Total loss: 0.042, Loss x: 0.043, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.873
[2022-11-26 14:30:37,476][ INFO] Iters: 1860, Total loss: 0.042, Loss x: 0.044, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.872
[2022-11-26 14:33:03,438][ INFO] Iters: 2170, Total loss: 0.042, Loss x: 0.043, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.873
[2022-11-26 14:36:15,417][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.00

[2022-11-26 14:36:15,418][ INFO] ===========> Epoch: 65, LR: 0.0001, Previous best: 75.50
[2022-11-26 14:36:16,383][ INFO] Iters: 0, Total loss: 0.120, Loss x: 0.051, Loss s: 0.358, Loss w_fp: 0.017, Mask: 0.739
[2022-11-26 14:38:41,883][ INFO] Iters: 310, Total loss: 0.045, Loss x: 0.045, Loss s: 0.083, Loss w_fp: 0.007, Mask: 0.867
[2022-11-26 14:41:07,715][ INFO] Iters: 620, Total loss: 0.044, Loss x: 0.044, Loss s: 0.081, Loss w_fp: 0.007, Mask: 0.869
[2022-11-26 14:43:33,642][ INFO] Iters: 930, Total loss: 0.043, Loss x: 0.044, Loss s: 0.078, Loss w_fp: 0.007, Mask: 0.872
[2022-11-26 14:45:59,682][ INFO] Iters: 1240, Total loss: 0.042, Loss x: 0.044, Loss s: 0.076, Loss w_fp: 0.007, Mask: 0.872
[2022-11-26 14:48:25,603][ INFO] Iters: 1550, Total loss: 0.042, Loss x: 0.044, Loss s: 0.074, Loss w_fp: 0.006, Mask: 0.873
[2022-11-26 14:50:51,235][ INFO] Iters: 1860, Total loss: 0.042, Loss x: 0.044, Loss s: 0.074, Loss w_fp: 0.006, Mask: 0.874
[2022-11-26 14:53:16,984][ INFO] Iters: 2170, Total loss: 0.042, Loss x: 0.044, Loss s: 0.073, Loss w_fp: 0.007, Mask: 0.874
[2022-11-26 14:56:29,377][ INFO] ***** Evaluation original ***** >>>> meanIOU: 73.41

[2022-11-26 14:56:29,377][ INFO] ===========> Epoch: 66, LR: 0.0001, Previous best: 75.50
[2022-11-26 14:56:30,308][ INFO] Iters: 0, Total loss: 0.037, Loss x: 0.030, Loss s: 0.085, Loss w_fp: 0.004, Mask: 0.910
[2022-11-26 14:58:56,270][ INFO] Iters: 310, Total loss: 0.039, Loss x: 0.042, Loss s: 0.068, Loss w_fp: 0.006, Mask: 0.874
[2022-11-26 15:01:22,247][ INFO] Iters: 620, Total loss: 0.040, Loss x: 0.042, Loss s: 0.071, Loss w_fp: 0.006, Mask: 0.875
[2022-11-26 15:03:48,455][ INFO] Iters: 930, Total loss: 0.042, Loss x: 0.042, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.873
[2022-11-26 15:06:14,616][ INFO] Iters: 1240, Total loss: 0.042, Loss x: 0.042, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.874
[2022-11-26 15:08:40,501][ INFO] Iters: 1550, Total loss: 0.042, Loss x: 0.043, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.875
[2022-11-26 15:11:06,328][ INFO] Iters: 1860, Total loss: 0.042, Loss x: 0.042, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.875
[2022-11-26 15:13:32,193][ INFO] Iters: 2170, Total loss: 0.042, Loss x: 0.043, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.875
[2022-11-26 15:16:45,312][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.90

[2022-11-26 15:16:45,312][ INFO] ===========> Epoch: 67, LR: 0.0001, Previous best: 75.50
[2022-11-26 15:16:46,296][ INFO] Iters: 0, Total loss: 0.028, Loss x: 0.037, Loss s: 0.030, Loss w_fp: 0.009, Mask: 0.900
[2022-11-26 15:19:12,017][ INFO] Iters: 310, Total loss: 0.043, Loss x: 0.045, Loss s: 0.077, Loss w_fp: 0.006, Mask: 0.883
[2022-11-26 15:21:37,881][ INFO] Iters: 620, Total loss: 0.042, Loss x: 0.044, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.874
[2022-11-26 15:24:03,621][ INFO] Iters: 930, Total loss: 0.042, Loss x: 0.043, Loss s: 0.074, Loss w_fp: 0.006, Mask: 0.874
[2022-11-26 15:26:29,487][ INFO] Iters: 1240, Total loss: 0.041, Loss x: 0.043, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.875
[2022-11-26 15:28:55,299][ INFO] Iters: 1550, Total loss: 0.041, Loss x: 0.043, Loss s: 0.074, Loss w_fp: 0.007, Mask: 0.875
[2022-11-26 15:31:20,999][ INFO] Iters: 1860, Total loss: 0.042, Loss x: 0.043, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.874
[2022-11-26 15:33:46,605][ INFO] Iters: 2170, Total loss: 0.042, Loss x: 0.043, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.874
[2022-11-26 15:36:58,628][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.40

[2022-11-26 15:36:58,628][ INFO] ===========> Epoch: 68, LR: 0.0001, Previous best: 75.50
[2022-11-26 15:36:59,587][ INFO] Iters: 0, Total loss: 0.036, Loss x: 0.031, Loss s: 0.078, Loss w_fp: 0.003, Mask: 0.893
[2022-11-26 15:39:25,359][ INFO] Iters: 310, Total loss: 0.043, Loss x: 0.043, Loss s: 0.078, Loss w_fp: 0.006, Mask: 0.873
[2022-11-26 15:41:51,302][ INFO] Iters: 620, Total loss: 0.044, Loss x: 0.043, Loss s: 0.082, Loss w_fp: 0.006, Mask: 0.871
[2022-11-26 15:44:17,272][ INFO] Iters: 930, Total loss: 0.043, Loss x: 0.044, Loss s: 0.077, Loss w_fp: 0.006, Mask: 0.874
[2022-11-26 15:46:43,027][ INFO] Iters: 1240, Total loss: 0.043, Loss x: 0.043, Loss s: 0.077, Loss w_fp: 0.006, Mask: 0.875
[2022-11-26 15:49:08,844][ INFO] Iters: 1550, Total loss: 0.043, Loss x: 0.043, Loss s: 0.079, Loss w_fp: 0.006, Mask: 0.874
[2022-11-26 15:51:34,646][ INFO] Iters: 1860, Total loss: 0.043, Loss x: 0.043, Loss s: 0.077, Loss w_fp: 0.007, Mask: 0.873
[2022-11-26 15:54:00,462][ INFO] Iters: 2170, Total loss: 0.043, Loss x: 0.043, Loss s: 0.078, Loss w_fp: 0.007, Mask: 0.874
[2022-11-26 15:57:12,666][ INFO] ***** Evaluation original ***** >>>> meanIOU: 73.46

[2022-11-26 15:57:12,667][ INFO] ===========> Epoch: 69, LR: 0.0001, Previous best: 75.50
[2022-11-26 15:57:13,584][ INFO] Iters: 0, Total loss: 0.020, Loss x: 0.019, Loss s: 0.040, Loss w_fp: 0.002, Mask: 0.936
[2022-11-26 15:59:39,474][ INFO] Iters: 310, Total loss: 0.042, Loss x: 0.042, Loss s: 0.078, Loss w_fp: 0.007, Mask: 0.883
[2022-11-26 16:02:05,623][ INFO] Iters: 620, Total loss: 0.042, Loss x: 0.043, Loss s: 0.075, Loss w_fp: 0.007, Mask: 0.880
[2022-11-26 16:04:31,799][ INFO] Iters: 930, Total loss: 0.042, Loss x: 0.043, Loss s: 0.077, Loss w_fp: 0.007, Mask: 0.879
[2022-11-26 16:06:57,971][ INFO] Iters: 1240, Total loss: 0.042, Loss x: 0.043, Loss s: 0.077, Loss w_fp: 0.006, Mask: 0.878
[2022-11-26 16:09:23,926][ INFO] Iters: 1550, Total loss: 0.042, Loss x: 0.043, Loss s: 0.077, Loss w_fp: 0.007, Mask: 0.877
[2022-11-26 16:11:49,954][ INFO] Iters: 1860, Total loss: 0.042, Loss x: 0.043, Loss s: 0.076, Loss w_fp: 0.007, Mask: 0.876
[2022-11-26 16:14:16,064][ INFO] Iters: 2170, Total loss: 0.042, Loss x: 0.043, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.875
[2022-11-26 16:17:27,339][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.04

[2022-11-26 16:17:27,339][ INFO] ===========> Epoch: 70, LR: 0.0001, Previous best: 75.50
[2022-11-26 16:17:28,286][ INFO] Iters: 0, Total loss: 0.067, Loss x: 0.055, Loss s: 0.154, Loss w_fp: 0.004, Mask: 0.897
[2022-11-26 16:19:53,816][ INFO] Iters: 310, Total loss: 0.042, Loss x: 0.044, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.877
[2022-11-26 16:22:19,542][ INFO] Iters: 620, Total loss: 0.041, Loss x: 0.043, Loss s: 0.072, Loss w_fp: 0.006, Mask: 0.875
[2022-11-26 16:24:45,129][ INFO] Iters: 930, Total loss: 0.041, Loss x: 0.042, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.874
[2022-11-26 16:27:10,740][ INFO] Iters: 1240, Total loss: 0.041, Loss x: 0.042, Loss s: 0.074, Loss w_fp: 0.007, Mask: 0.875
[2022-11-26 16:29:36,433][ INFO] Iters: 1550, Total loss: 0.042, Loss x: 0.042, Loss s: 0.076, Loss w_fp: 0.007, Mask: 0.873
[2022-11-26 16:32:02,092][ INFO] Iters: 1860, Total loss: 0.041, Loss x: 0.042, Loss s: 0.075, Loss w_fp: 0.007, Mask: 0.872
[2022-11-26 16:34:27,946][ INFO] Iters: 2170, Total loss: 0.042, Loss x: 0.042, Loss s: 0.075, Loss w_fp: 0.007, Mask: 0.872
[2022-11-26 16:37:40,700][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.45

[2022-11-26 16:37:40,700][ INFO] ===========> Epoch: 71, LR: 0.0001, Previous best: 75.50
[2022-11-26 16:37:41,687][ INFO] Iters: 0, Total loss: 0.034, Loss x: 0.042, Loss s: 0.050, Loss w_fp: 0.003, Mask: 0.958
[2022-11-26 16:40:07,409][ INFO] Iters: 310, Total loss: 0.042, Loss x: 0.040, Loss s: 0.079, Loss w_fp: 0.007, Mask: 0.872
[2022-11-26 16:42:33,304][ INFO] Iters: 620, Total loss: 0.041, Loss x: 0.041, Loss s: 0.075, Loss w_fp: 0.007, Mask: 0.875
[2022-11-26 16:44:59,122][ INFO] Iters: 930, Total loss: 0.041, Loss x: 0.042, Loss s: 0.075, Loss w_fp: 0.007, Mask: 0.874
[2022-11-26 16:47:25,134][ INFO] Iters: 1240, Total loss: 0.041, Loss x: 0.042, Loss s: 0.074, Loss w_fp: 0.007, Mask: 0.874
[2022-11-26 16:49:51,268][ INFO] Iters: 1550, Total loss: 0.041, Loss x: 0.042, Loss s: 0.074, Loss w_fp: 0.006, Mask: 0.876
[2022-11-26 16:52:17,082][ INFO] Iters: 1860, Total loss: 0.041, Loss x: 0.042, Loss s: 0.073, Loss w_fp: 0.007, Mask: 0.876
[2022-11-26 16:54:42,751][ INFO] Iters: 2170, Total loss: 0.041, Loss x: 0.042, Loss s: 0.074, Loss w_fp: 0.007, Mask: 0.876
[2022-11-26 16:57:54,588][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.28

[2022-11-26 16:57:54,589][ INFO] ===========> Epoch: 72, LR: 0.0001, Previous best: 75.50
[2022-11-26 16:57:55,606][ INFO] Iters: 0, Total loss: 0.021, Loss x: 0.024, Loss s: 0.031, Loss w_fp: 0.003, Mask: 0.906
[2022-11-26 17:00:21,296][ INFO] Iters: 310, Total loss: 0.041, Loss x: 0.042, Loss s: 0.074, Loss w_fp: 0.007, Mask: 0.873
[2022-11-26 17:02:47,255][ INFO] Iters: 620, Total loss: 0.040, Loss x: 0.042, Loss s: 0.072, Loss w_fp: 0.006, Mask: 0.872
[2022-11-26 17:05:13,095][ INFO] Iters: 930, Total loss: 0.041, Loss x: 0.042, Loss s: 0.072, Loss w_fp: 0.006, Mask: 0.872
[2022-11-26 17:07:38,972][ INFO] Iters: 1240, Total loss: 0.040, Loss x: 0.042, Loss s: 0.071, Loss w_fp: 0.007, Mask: 0.872
[2022-11-26 17:10:04,906][ INFO] Iters: 1550, Total loss: 0.040, Loss x: 0.042, Loss s: 0.071, Loss w_fp: 0.007, Mask: 0.874
[2022-11-26 17:12:30,715][ INFO] Iters: 1860, Total loss: 0.041, Loss x: 0.042, Loss s: 0.071, Loss w_fp: 0.007, Mask: 0.875
[2022-11-26 17:14:56,443][ INFO] Iters: 2170, Total loss: 0.041, Loss x: 0.042, Loss s: 0.071, Loss w_fp: 0.007, Mask: 0.875
[2022-11-26 17:18:08,370][ INFO] ***** Evaluation original ***** >>>> meanIOU: 73.72

[2022-11-26 17:18:08,370][ INFO] ===========> Epoch: 73, LR: 0.0001, Previous best: 75.50
[2022-11-26 17:18:09,303][ INFO] Iters: 0, Total loss: 0.036, Loss x: 0.055, Loss s: 0.021, Loss w_fp: 0.012, Mask: 0.858
[2022-11-26 17:20:35,162][ INFO] Iters: 310, Total loss: 0.042, Loss x: 0.043, Loss s: 0.074, Loss w_fp: 0.007, Mask: 0.869
[2022-11-26 17:23:01,109][ INFO] Iters: 620, Total loss: 0.041, Loss x: 0.042, Loss s: 0.074, Loss w_fp: 0.006, Mask: 0.874
[2022-11-26 17:25:27,178][ INFO] Iters: 930, Total loss: 0.041, Loss x: 0.043, Loss s: 0.074, Loss w_fp: 0.006, Mask: 0.874
[2022-11-26 17:27:53,151][ INFO] Iters: 1240, Total loss: 0.042, Loss x: 0.043, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.875
[2022-11-26 17:30:19,070][ INFO] Iters: 1550, Total loss: 0.041, Loss x: 0.042, Loss s: 0.074, Loss w_fp: 0.006, Mask: 0.875
[2022-11-26 17:32:44,909][ INFO] Iters: 1860, Total loss: 0.041, Loss x: 0.042, Loss s: 0.074, Loss w_fp: 0.006, Mask: 0.876
[2022-11-26 17:35:11,114][ INFO] Iters: 2170, Total loss: 0.041, Loss x: 0.042, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.876
[2022-11-26 17:38:25,497][ INFO] ***** Evaluation original ***** >>>> meanIOU: 75.23

[2022-11-26 17:38:25,498][ INFO] ===========> Epoch: 74, LR: 0.0000, Previous best: 75.50
[2022-11-26 17:38:26,570][ INFO] Iters: 0, Total loss: 0.027, Loss x: 0.040, Loss s: 0.012, Loss w_fp: 0.017, Mask: 0.965
[2022-11-26 17:40:52,452][ INFO] Iters: 310, Total loss: 0.039, Loss x: 0.040, Loss s: 0.070, Loss w_fp: 0.006, Mask: 0.879
[2022-11-26 17:43:18,544][ INFO] Iters: 620, Total loss: 0.040, Loss x: 0.042, Loss s: 0.072, Loss w_fp: 0.006, Mask: 0.877
[2022-11-26 17:45:44,546][ INFO] Iters: 930, Total loss: 0.040, Loss x: 0.042, Loss s: 0.072, Loss w_fp: 0.006, Mask: 0.877
[2022-11-26 17:48:10,223][ INFO] Iters: 1240, Total loss: 0.041, Loss x: 0.042, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.877
[2022-11-26 17:50:36,061][ INFO] Iters: 1550, Total loss: 0.041, Loss x: 0.042, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.877
[2022-11-26 17:53:01,906][ INFO] Iters: 1860, Total loss: 0.041, Loss x: 0.042, Loss s: 0.074, Loss w_fp: 0.006, Mask: 0.878
[2022-11-26 17:55:27,752][ INFO] Iters: 2170, Total loss: 0.041, Loss x: 0.041, Loss s: 0.074, Loss w_fp: 0.007, Mask: 0.878
[2022-11-26 17:58:40,788][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.61

[2022-11-26 17:58:40,789][ INFO] ===========> Epoch: 75, LR: 0.0000, Previous best: 75.50
[2022-11-26 17:58:41,776][ INFO] Iters: 0, Total loss: 0.044, Loss x: 0.064, Loss s: 0.040, Loss w_fp: 0.007, Mask: 0.950
[2022-11-26 18:01:07,456][ INFO] Iters: 310, Total loss: 0.039, Loss x: 0.042, Loss s: 0.067, Loss w_fp: 0.006, Mask: 0.885
[2022-11-26 18:03:33,128][ INFO] Iters: 620, Total loss: 0.039, Loss x: 0.041, Loss s: 0.067, Loss w_fp: 0.006, Mask: 0.881
[2022-11-26 18:05:59,445][ INFO] Iters: 930, Total loss: 0.040, Loss x: 0.042, Loss s: 0.070, Loss w_fp: 0.006, Mask: 0.880
[2022-11-26 18:08:25,245][ INFO] Iters: 1240, Total loss: 0.041, Loss x: 0.042, Loss s: 0.072, Loss w_fp: 0.006, Mask: 0.878
[2022-11-26 18:10:51,085][ INFO] Iters: 1550, Total loss: 0.041, Loss x: 0.042, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.879
[2022-11-26 18:13:16,942][ INFO] Iters: 1860, Total loss: 0.041, Loss x: 0.041, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.878
[2022-11-26 18:15:42,945][ INFO] Iters: 2170, Total loss: 0.040, Loss x: 0.041, Loss s: 0.072, Loss w_fp: 0.006, Mask: 0.879
[2022-11-26 18:18:56,812][ INFO] ***** Evaluation original ***** >>>> meanIOU: 73.65

[2022-11-26 18:18:56,813][ INFO] ===========> Epoch: 76, LR: 0.0000, Previous best: 75.50
[2022-11-26 18:18:57,858][ INFO] Iters: 0, Total loss: 0.019, Loss x: 0.028, Loss s: 0.017, Loss w_fp: 0.002, Mask: 0.947
[2022-11-26 18:21:23,727][ INFO] Iters: 310, Total loss: 0.038, Loss x: 0.040, Loss s: 0.065, Loss w_fp: 0.006, Mask: 0.883
[2022-11-26 18:23:49,527][ INFO] Iters: 620, Total loss: 0.039, Loss x: 0.041, Loss s: 0.068, Loss w_fp: 0.006, Mask: 0.883
[2022-11-26 18:26:15,655][ INFO] Iters: 930, Total loss: 0.040, Loss x: 0.041, Loss s: 0.074, Loss w_fp: 0.006, Mask: 0.881
[2022-11-26 18:28:41,375][ INFO] Iters: 1240, Total loss: 0.040, Loss x: 0.041, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.881
[2022-11-26 18:31:07,247][ INFO] Iters: 1550, Total loss: 0.040, Loss x: 0.041, Loss s: 0.072, Loss w_fp: 0.006, Mask: 0.880
[2022-11-26 18:33:32,964][ INFO] Iters: 1860, Total loss: 0.040, Loss x: 0.041, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.880
[2022-11-26 18:35:58,774][ INFO] Iters: 2170, Total loss: 0.040, Loss x: 0.041, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.879
[2022-11-26 18:39:12,715][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.40

[2022-11-26 18:39:12,715][ INFO] ===========> Epoch: 77, LR: 0.0000, Previous best: 75.50
[2022-11-26 18:39:13,694][ INFO] Iters: 0, Total loss: 0.023, Loss x: 0.024, Loss s: 0.040, Loss w_fp: 0.005, Mask: 0.893
[2022-11-26 18:41:39,716][ INFO] Iters: 310, Total loss: 0.039, Loss x: 0.040, Loss s: 0.068, Loss w_fp: 0.006, Mask: 0.879
[2022-11-26 18:44:05,647][ INFO] Iters: 620, Total loss: 0.040, Loss x: 0.041, Loss s: 0.071, Loss w_fp: 0.006, Mask: 0.878
[2022-11-26 18:46:31,521][ INFO] Iters: 930, Total loss: 0.040, Loss x: 0.041, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.877
[2022-11-26 18:48:57,314][ INFO] Iters: 1240, Total loss: 0.041, Loss x: 0.042, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.877
[2022-11-26 18:51:23,071][ INFO] Iters: 1550, Total loss: 0.041, Loss x: 0.042, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.877
[2022-11-26 18:53:48,921][ INFO] Iters: 1860, Total loss: 0.041, Loss x: 0.042, Loss s: 0.074, Loss w_fp: 0.006, Mask: 0.878
[2022-11-26 18:56:14,890][ INFO] Iters: 2170, Total loss: 0.041, Loss x: 0.041, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.878
[2022-11-26 18:59:28,298][ INFO] ***** Evaluation original ***** >>>> meanIOU: 74.88

[2022-11-26 18:59:28,298][ INFO] ===========> Epoch: 78, LR: 0.0000, Previous best: 75.50
[2022-11-26 18:59:29,233][ INFO] Iters: 0, Total loss: 0.042, Loss x: 0.070, Loss s: 0.027, Loss w_fp: 0.002, Mask: 0.941
[2022-11-26 19:01:55,023][ INFO] Iters: 310, Total loss: 0.040, Loss x: 0.042, Loss s: 0.071, Loss w_fp: 0.007, Mask: 0.876
[2022-11-26 19:04:21,088][ INFO] Iters: 620, Total loss: 0.042, Loss x: 0.042, Loss s: 0.076, Loss w_fp: 0.007, Mask: 0.877
[2022-11-26 19:06:46,743][ INFO] Iters: 930, Total loss: 0.041, Loss x: 0.042, Loss s: 0.075, Loss w_fp: 0.007, Mask: 0.876
[2022-11-26 19:09:12,492][ INFO] Iters: 1240, Total loss: 0.042, Loss x: 0.042, Loss s: 0.077, Loss w_fp: 0.007, Mask: 0.876
[2022-11-26 19:11:38,280][ INFO] Iters: 1550, Total loss: 0.041, Loss x: 0.042, Loss s: 0.075, Loss w_fp: 0.007, Mask: 0.878
[2022-11-26 19:14:04,024][ INFO] Iters: 1860, Total loss: 0.041, Loss x: 0.041, Loss s: 0.075, Loss w_fp: 0.007, Mask: 0.879

from unimatch.

LiheYoung avatar LiheYoung commented on September 24, 2024

Since the batch size is reduced by 2 times, I think it can be expected that the performance drops by 1%. You can try to use two GPUs, or reduce the cropping size from 321 to 281 and maintain the batch size as 8 if GPU memory is allowed.

from unimatch.

jerrywyn avatar jerrywyn commented on September 24, 2024

I ran the unimatch with batch size 4 and initial lr 0.0005 on a single GPU for the 1/16 partition of pascal and I took 78.63 mIOU without other change except eliminated distributed code. The best result is trained in epoch 38. This is a bit of a huge difference from your result of 75.50?

from unimatch.

nysp78 avatar nysp78 commented on September 24, 2024

@jerrywyn Which part of the distributed code do you cross out? Also, what crop size do you use?

Thanks

from unimatch.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.