Comments (20)
LiheYoung
commented on September 25, 2024
Please wait for two days. I will re-run our UniMatch on the Cityscapes under the splits of 1/16, 1/8, 1/4, and 1/2 with ResNet-101. And then I will upload the trained models and logs for better reproduction.
from unimatch.
LiheYoung
commented on September 25, 2024
Here are the logs and trained models on the Cityscapes under different data partitions with ResNet-101 and DeepLabV3Plus.
| Partitions | 1/16 (186) | 1/8 (372) | 1/4 (744) | 1/2 (1488) |
|---|---|---|---|---|
| Reported | 75.7 | 77.3 | 78.7 | - |
| Re-run | 76.0 | 77.6 | 78.6 | 79.3 |
| Logs | log | log | log | log |
| Checkpoints | ckpt | ckpt | ckpt | ckpt |
Please let me know if you have any problem reproducing our results. Thank you!
from unimatch.
zhibotian
commented on September 25, 2024
Could you please provide the logs and trained models on the VOC under different data partitions with ResNet-101 and DeepLabV3Plus, Thanks!
from unimatch.
LiheYoung
commented on September 25, 2024
Hi @zhibotian, you may refer to #5.
from unimatch.
zhibotian
commented on September 25, 2024
Hi @zhibotian, you may refer to #5.
Ok, get it. Thanks a lot.
from unimatch.
Jack-bo1220
commented on September 25, 2024
Thanks a lot. It really help me a lot.
Also, I would like to ask how the backbone initialization seed is set? Random or fixed? I do not find a description of seed setting in your code. Can you provide this information to me?
from unimatch.
LiheYoung
commented on September 25, 2024
Actually, I did not set any seed in my experiments. The backbone is pre-trained from ImageNet, while the head parameters are randomly initialized by default.
from unimatch.
Jack-bo1220
commented on September 25, 2024
Here are the logs and trained models on the Cityscapes under different data partitions with ResNet-101 and DeepLabV3Plus.
Partitions 1/16 (186) 1/8 (372) 1/4 (744) 1/2 (1488)
Reported 75.7 77.3 78.7 -
Re-run 76.0 77.6 78.6 79.3
Logs log log log log
Checkpoints ckpt ckpt ckpt ckpt
Please let me know if you have any problem reproducing our results. Thank you!
I'm sorry to disturb you again.
I have evaluated the weights you provided offline using a single GPU, but there are always some small gaps in the results. For example, with the Cityscapes 1/4 setting, I ran a result of 78.66 (you provided: 78.61). With the cityscapes 1/2 setting, I ran a result of 79.39 (you provided: 79.29).
Even though the difference is lower for both, I would like to ask if it is the difference between multi-gpu and single-gpu references? Or is it some other difference (other settings I did not modify). Thanks.
from unimatch.
LiheYoung
commented on September 25, 2024
I also observe some minor performance differences when I shuffle the data sequence. But with the same sequence defined in our val.txt and the same number of GPUs, the results are always consistent. So you may try to evaluate these checkpoints with 8 GPUs.
from unimatch.
Jack-bo1220
commented on September 25, 2024
Unfortunately, I don't have 8GPU to perform the verification.
But I think a minor difference (only 0.05 or 0.1) is acceptable. Since I excluded the data sequence difference, then I guess the difference may come from the difference between single and multiple GPU?
Thanks for your help, what is the approximate performance difference from the data sequence you mentioned?
from unimatch.
LiheYoung
commented on September 25, 2024
I think so. The differences caused by data sequences are always less than 0.1%. And if you have any idea about other reasons for the minor fluctuation, please tell me. Thank you.
from unimatch.
Jack-bo1220
commented on September 25, 2024
I think so. The differences caused by data sequences are always less than 0.1%. And if you have any idea about other reasons for the minor fluctuation, please tell me. Thank you.
thanks a lot.
0.1% you mentioned means the gap like between 78.1% and 78.2%(mIOU)?
from unimatch.
LiheYoung
commented on September 25, 2024
Yes.
from unimatch.
JoyHuYY1412
commented on September 25, 2024
log
Hi Young,
Thanks for sharing the logs!
However, I tried reproducing the results in 1/16 data with 4GPUs*2 images per GPU, but the results were not so good. Currently, the highest performance in the already run 70 epochs is 67.59, 2% lower than your log at the same epoch (69.86%). I am not so sure whether it can still catch up in the later training and whether this gap is normal. Perhaps you could help me check with the logs in the beginning 10 epochs? I found that the mask value is much lower than yours. Maybe that is the case? Thank you so much!
from unimatch.
JoyHuYY1412
commented on September 25, 2024
log
Hi Young,
Thanks for sharing the logs! However, I tried reproducing the results in 1/16 data with 4GPUs*2 images per GPU, but the results were not so good. Currently, the highest performance in the already run 70 epochs is 67.59, 2% lower than your log at the same epoch (69.86%). I am not so sure whether it can still catch up in the later training. Perhaps you could help me check with the logs in the beginning 10 epochs? I found that the mask value is much lower than yours. Maybe that is the case? Thank you so much!
/home/anaconda3/envs/unimatch2/lib/python3.6/site-packages/torch/distributed/launch.py:186: FutureWarning: The module torch.distributed.launch is deprecated
and will be removed in future. Use torchrun.
Note that --use_env is set by default in torchrun.
If your script expects --local_rank argument to be set, please
change it to read from os.environ['LOCAL_RANK'] instead. See
https://pytorch.org/docs/stable/distributed.html#launch-utility for
further instructions
FutureWarning,
WARNING:torch.distributed.run:
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
[2023-02-12 20:17:40,374][ INFO] {'backbone': 'resnet101',
'batch_size': 2,
'conf_thresh': 0.95,
'criterion': {'kwargs': {'ignore_index': 255,
'min_kept': 100000,
'thresh': 0.7},
'name': 'OHEM'},
'crop_size': 769,
'data_root': '/BS/SETR/work/Projects/UniMatch_ICON/data/cityscapes',
'dataset': 'cityscapes',
'dilations': [12, 24, 36],
'epochs': 240,
'lr': 0.005,
'lr_multi': 1.0,
'multi_grid': True,
'nclass': 19,
'replace_stride_with_dilation': [False, True, True]}
[2023-02-12 20:17:53,387][ INFO] Total params: 59.5M
[2023-02-12 20:17:57,276][ INFO] Start resuming from exp/cityscapes/1_16/unimatch...
[2023-02-12 20:17:57,277][ INFO] Start resuming from exp/cityscapes/1_16/unimatch...
[2023-02-12 20:17:57,297][ INFO] Start resuming from exp/cityscapes/1_16/unimatch...
[2023-02-12 20:17:57,376][ INFO] Start resuming from exp/cityscapes/1_16/unimatch...
[2023-02-12 20:17:57,697][ INFO] ===========> Epoch: 0, LR: 0.0050, Previous best: 0.00
[2023-02-12 20:18:28,467][ INFO] Iters: 0, Total loss: 1.502, Loss x: 3.005, Loss s: 0.000, Loss w_fp: 0.000, Mask: 0.000
[2023-02-12 20:20:09,990][ INFO] Iters: 43, Total loss: 0.948, Loss x: 1.890, Loss s: 0.005, Loss w_fp: 0.009, Mask: 0.020
[2023-02-12 20:21:51,586][ INFO] Iters: 86, Total loss: 0.854, Loss x: 1.698, Loss s: 0.008, Loss w_fp: 0.012, Mask: 0.054
[2023-02-12 20:23:33,208][ INFO] Iters: 129, Total loss: 0.785, Loss x: 1.560, Loss s: 0.010, Loss w_fp: 0.011, Mask: 0.062
[2023-02-12 20:25:14,839][ INFO] Iters: 172, Total loss: 0.751, Loss x: 1.492, Loss s: 0.010, Loss w_fp: 0.010, Mask: 0.061
[2023-02-12 20:26:56,547][ INFO] Iters: 215, Total loss: 0.721, Loss x: 1.433, Loss s: 0.009, Loss w_fp: 0.009, Mask: 0.057
[2023-02-12 20:28:38,250][ INFO] Iters: 258, Total loss: 0.695, Loss x: 1.381, Loss s: 0.009, Loss w_fp: 0.009, Mask: 0.055
[2023-02-12 20:30:19,985][ INFO] Iters: 301, Total loss: 0.675, Loss x: 1.342, Loss s: 0.009, Loss w_fp: 0.009, Mask: 0.056
[2023-02-12 20:32:01,728][ INFO] Iters: 344, Total loss: 0.662, Loss x: 1.314, Loss s: 0.011, Loss w_fp: 0.009, Mask: 0.069
[2023-02-12 20:32:29,928][ INFO] ***** Evaluation center_crop ***** >>>> meanIOU: 47.56
[2023-02-12 20:32:33,077][ INFO] ===========> Epoch: 1, LR: 0.0050, Previous best: 47.56
[2023-02-12 20:32:36,496][ INFO] Iters: 0, Total loss: 0.545, Loss x: 1.082, Loss s: 0.011, Loss w_fp: 0.005, Mask: 0.099
[2023-02-12 20:34:18,108][ INFO] Iters: 43, Total loss: 0.602, Loss x: 1.188, Loss s: 0.025, Loss w_fp: 0.008, Mask: 0.184
[2023-02-12 20:35:59,770][ INFO] Iters: 86, Total loss: 0.588, Loss x: 1.161, Loss s: 0.022, Loss w_fp: 0.006, Mask: 0.171
[2023-02-12 20:37:41,362][ INFO] Iters: 129, Total loss: 0.583, Loss x: 1.152, Loss s: 0.022, Loss w_fp: 0.006, Mask: 0.177
[2023-02-12 20:39:22,972][ INFO] Iters: 172, Total loss: 0.577, Loss x: 1.139, Loss s: 0.022, Loss w_fp: 0.006, Mask: 0.191
[2023-02-12 20:41:04,621][ INFO] Iters: 215, Total loss: 0.576, Loss x: 1.137, Loss s: 0.023, Loss w_fp: 0.006, Mask: 0.195
[2023-02-12 20:42:46,271][ INFO] Iters: 258, Total loss: 0.572, Loss x: 1.129, Loss s: 0.023, Loss w_fp: 0.006, Mask: 0.193
[2023-02-12 20:44:27,947][ INFO] Iters: 301, Total loss: 0.570, Loss x: 1.125, Loss s: 0.024, Loss w_fp: 0.006, Mask: 0.197
[2023-02-12 20:46:09,630][ INFO] Iters: 344, Total loss: 0.563, Loss x: 1.111, Loss s: 0.024, Loss w_fp: 0.006, Mask: 0.201
[2023-02-12 20:46:35,203][ INFO] ***** Evaluation center_crop ***** >>>> meanIOU: 54.04
[2023-02-12 20:46:38,287][ INFO] ===========> Epoch: 2, LR: 0.0050, Previous best: 54.04
[2023-02-12 20:46:41,870][ INFO] Iters: 0, Total loss: 0.543, Loss x: 1.082, Loss s: 0.004, Loss w_fp: 0.003, Mask: 0.086
[2023-02-12 20:48:23,522][ INFO] Iters: 43, Total loss: 0.538, Loss x: 1.067, Loss s: 0.016, Loss w_fp: 0.004, Mask: 0.215
[2023-02-12 20:50:05,211][ INFO] Iters: 86, Total loss: 0.539, Loss x: 1.066, Loss s: 0.019, Loss w_fp: 0.004, Mask: 0.226
[2023-02-12 20:51:46,828][ INFO] Iters: 129, Total loss: 0.547, Loss x: 1.081, Loss s: 0.020, Loss w_fp: 0.004, Mask: 0.228
[2023-02-12 20:53:28,461][ INFO] Iters: 172, Total loss: 0.531, Loss x: 1.049, Loss s: 0.019, Loss w_fp: 0.004, Mask: 0.226
[2023-02-12 20:55:10,070][ INFO] Iters: 215, Total loss: 0.524, Loss x: 1.036, Loss s: 0.020, Loss w_fp: 0.004, Mask: 0.228
[2023-02-12 20:56:51,823][ INFO] Iters: 258, Total loss: 0.522, Loss x: 1.033, Loss s: 0.020, Loss w_fp: 0.004, Mask: 0.227
[2023-02-12 20:58:33,645][ INFO] Iters: 301, Total loss: 0.516, Loss x: 1.020, Loss s: 0.020, Loss w_fp: 0.005, Mask: 0.233
[2023-02-12 21:00:15,524][ INFO] Iters: 344, Total loss: 0.514, Loss x: 1.016, Loss s: 0.019, Loss w_fp: 0.005, Mask: 0.240
[2023-02-12 21:00:41,295][ INFO] ***** Evaluation center_crop ***** >>>> meanIOU: 58.39
[2023-02-12 21:00:44,620][ INFO] ===========> Epoch: 3, LR: 0.0049, Previous best: 58.39
[2023-02-12 21:00:48,105][ INFO] Iters: 0, Total loss: 0.463, Loss x: 0.907, Loss s: 0.031, Loss w_fp: 0.007, Mask: 0.314
[2023-02-12 21:02:29,794][ INFO] Iters: 43, Total loss: 0.517, Loss x: 1.018, Loss s: 0.026, Loss w_fp: 0.006, Mask: 0.319
[2023-02-12 21:04:11,651][ INFO] Iters: 86, Total loss: 0.491, Loss x: 0.968, Loss s: 0.022, Loss w_fp: 0.006, Mask: 0.294
[2023-02-12 21:05:53,638][ INFO] Iters: 129, Total loss: 0.489, Loss x: 0.964, Loss s: 0.020, Loss w_fp: 0.006, Mask: 0.292
[2023-02-12 21:07:35,457][ INFO] Iters: 172, Total loss: 0.480, Loss x: 0.948, Loss s: 0.020, Loss w_fp: 0.006, Mask: 0.307
[2023-02-12 21:09:17,309][ INFO] Iters: 215, Total loss: 0.477, Loss x: 0.941, Loss s: 0.022, Loss w_fp: 0.006, Mask: 0.309
[2023-02-12 21:10:59,020][ INFO] Iters: 258, Total loss: 0.482, Loss x: 0.950, Loss s: 0.022, Loss w_fp: 0.006, Mask: 0.327
[2023-02-12 21:12:40,740][ INFO] Iters: 301, Total loss: 0.485, Loss x: 0.957, Loss s: 0.022, Loss w_fp: 0.006, Mask: 0.342
[2023-02-12 21:14:22,545][ INFO] Iters: 344, Total loss: 0.485, Loss x: 0.957, Loss s: 0.022, Loss w_fp: 0.006, Mask: 0.348
[2023-02-12 21:14:48,305][ INFO] ***** Evaluation center_crop ***** >>>> meanIOU: 59.79
[2023-02-12 21:14:51,517][ INFO] ===========> Epoch: 4, LR: 0.0049, Previous best: 59.79
[2023-02-12 21:14:55,091][ INFO] Iters: 0, Total loss: 0.522, Loss x: 1.038, Loss s: 0.008, Loss w_fp: 0.004, Mask: 0.250
[2023-02-12 21:16:36,610][ INFO] Iters: 43, Total loss: 0.495, Loss x: 0.973, Loss s: 0.027, Loss w_fp: 0.006, Mask: 0.410
[2023-02-12 21:18:18,172][ INFO] Iters: 86, Total loss: 0.486, Loss x: 0.956, Loss s: 0.028, Loss w_fp: 0.006, Mask: 0.381
[2023-02-12 21:19:59,824][ INFO] Iters: 129, Total loss: 0.479, Loss x: 0.942, Loss s: 0.026, Loss w_fp: 0.006, Mask: 0.378
[2023-02-12 21:21:41,471][ INFO] Iters: 172, Total loss: 0.471, Loss x: 0.924, Loss s: 0.027, Loss w_fp: 0.006, Mask: 0.396
[2023-02-12 21:23:23,140][ INFO] Iters: 215, Total loss: 0.463, Loss x: 0.911, Loss s: 0.025, Loss w_fp: 0.007, Mask: 0.409
[2023-02-12 21:25:04,784][ INFO] Iters: 258, Total loss: 0.461, Loss x: 0.907, Loss s: 0.025, Loss w_fp: 0.007, Mask: 0.416
[2023-02-12 21:26:46,563][ INFO] Iters: 301, Total loss: 0.461, Loss x: 0.905, Loss s: 0.026, Loss w_fp: 0.007, Mask: 0.420
[2023-02-12 21:28:28,218][ INFO] Iters: 344, Total loss: 0.460, Loss x: 0.903, Loss s: 0.025, Loss w_fp: 0.007, Mask: 0.430
[2023-02-12 21:28:54,274][ INFO] ***** Evaluation center_crop ***** >>>> meanIOU: 56.80
[2023-02-12 21:28:56,343][ INFO] ===========> Epoch: 5, LR: 0.0049, Previous best: 59.79
[2023-02-12 21:28:59,775][ INFO] Iters: 0, Total loss: 0.445, Loss x: 0.873, Loss s: 0.030, Loss w_fp: 0.003, Mask: 0.285
[2023-02-12 21:30:41,321][ INFO] Iters: 43, Total loss: 0.491, Loss x: 0.965, Loss s: 0.026, Loss w_fp: 0.007, Mask: 0.444
[2023-02-12 21:32:22,979][ INFO] Iters: 86, Total loss: 0.473, Loss x: 0.929, Loss s: 0.028, Loss w_fp: 0.007, Mask: 0.469
[2023-02-12 21:34:04,582][ INFO] Iters: 129, Total loss: 0.469, Loss x: 0.919, Loss s: 0.031, Loss w_fp: 0.008, Mask: 0.475
[2023-02-12 21:35:46,186][ INFO] Iters: 172, Total loss: 0.487, Loss x: 0.954, Loss s: 0.032, Loss w_fp: 0.008, Mask: 0.462
[2023-02-12 21:37:27,891][ INFO] Iters: 215, Total loss: 0.477, Loss x: 0.935, Loss s: 0.031, Loss w_fp: 0.008, Mask: 0.465
[2023-02-12 21:39:09,620][ INFO] Iters: 258, Total loss: 0.475, Loss x: 0.930, Loss s: 0.034, Loss w_fp: 0.008, Mask: 0.463
[2023-02-12 21:40:51,318][ INFO] Iters: 301, Total loss: 0.468, Loss x: 0.915, Loss s: 0.035, Loss w_fp: 0.009, Mask: 0.470
[2023-02-12 21:42:32,988][ INFO] Iters: 344, Total loss: 0.466, Loss x: 0.909, Loss s: 0.035, Loss w_fp: 0.009, Mask: 0.483
[2023-02-12 21:42:58,965][ INFO] ***** Evaluation center_crop ***** >>>> meanIOU: 61.75
[2023-02-12 21:43:02,027][ INFO] ===========> Epoch: 6, LR: 0.0049, Previous best: 61.75
[2023-02-12 21:43:05,378][ INFO] Iters: 0, Total loss: 0.504, Loss x: 0.973, Loss s: 0.056, Loss w_fp: 0.015, Mask: 0.582
[2023-02-12 21:44:46,937][ INFO] Iters: 43, Total loss: 0.446, Loss x: 0.872, Loss s: 0.033, Loss w_fp: 0.008, Mask: 0.551
[2023-02-12 21:46:28,642][ INFO] Iters: 86, Total loss: 0.436, Loss x: 0.853, Loss s: 0.031, Loss w_fp: 0.008, Mask: 0.539
[2023-02-12 21:48:10,332][ INFO] Iters: 129, Total loss: 0.443, Loss x: 0.864, Loss s: 0.034, Loss w_fp: 0.009, Mask: 0.541
[2023-02-12 21:49:52,002][ INFO] Iters: 172, Total loss: 0.448, Loss x: 0.872, Loss s: 0.036, Loss w_fp: 0.010, Mask: 0.550
[2023-02-12 21:51:33,672][ INFO] Iters: 215, Total loss: 0.442, Loss x: 0.861, Loss s: 0.035, Loss w_fp: 0.010, Mask: 0.565
[2023-02-12 21:53:15,459][ INFO] Iters: 258, Total loss: 0.443, Loss x: 0.864, Loss s: 0.036, Loss w_fp: 0.009, Mask: 0.560
[2023-02-12 21:54:57,305][ INFO] Iters: 301, Total loss: 0.436, Loss x: 0.850, Loss s: 0.036, Loss w_fp: 0.009, Mask: 0.565
[2023-02-12 21:56:39,116][ INFO] Iters: 344, Total loss: 0.435, Loss x: 0.847, Loss s: 0.037, Loss w_fp: 0.009, Mask: 0.567
[2023-02-12 21:57:05,077][ INFO] ***** Evaluation center_crop ***** >>>> meanIOU: 61.71
[2023-02-12 21:57:07,247][ INFO] ===========> Epoch: 7, LR: 0.0049, Previous best: 61.75
[2023-02-12 21:57:10,768][ INFO] Iters: 0, Total loss: 0.448, Loss x: 0.883, Loss s: 0.021, Loss w_fp: 0.006, Mask: 0.665
[2023-02-12 21:58:52,386][ INFO] Iters: 43, Total loss: 0.450, Loss x: 0.877, Loss s: 0.037, Loss w_fp: 0.010, Mask: 0.624
[2023-02-12 22:00:33,994][ INFO] Iters: 86, Total loss: 0.440, Loss x: 0.856, Loss s: 0.039, Loss w_fp: 0.010, Mask: 0.613
[2023-02-12 22:02:15,673][ INFO] Iters: 129, Total loss: 0.430, Loss x: 0.836, Loss s: 0.041, Loss w_fp: 0.009, Mask: 0.617
[2023-02-12 22:03:57,410][ INFO] Iters: 172, Total loss: 0.423, Loss x: 0.822, Loss s: 0.040, Loss w_fp: 0.009, Mask: 0.614
[2023-02-12 22:05:39,340][ INFO] Iters: 215, Total loss: 0.424, Loss x: 0.824, Loss s: 0.039, Loss w_fp: 0.008, Mask: 0.619
[2023-02-12 22:07:21,190][ INFO] Iters: 258, Total loss: 0.425, Loss x: 0.827, Loss s: 0.037, Loss w_fp: 0.008, Mask: 0.614
[2023-02-12 22:09:03,019][ INFO] Iters: 301, Total loss: 0.425, Loss x: 0.828, Loss s: 0.037, Loss w_fp: 0.008, Mask: 0.614
[2023-02-12 22:10:44,888][ INFO] Iters: 344, Total loss: 0.419, Loss x: 0.816, Loss s: 0.038, Loss w_fp: 0.008, Mask: 0.617
[2023-02-12 22:11:10,992][ INFO] ***** Evaluation center_crop ***** >>>> meanIOU: 62.56
[2023-02-12 22:11:14,471][ INFO] ===========> Epoch: 8, LR: 0.0048, Previous best: 62.56
[2023-02-12 22:11:17,823][ INFO] Iters: 0, Total loss: 0.441, Loss x: 0.871, Loss s: 0.018, Loss w_fp: 0.005, Mask: 0.563
[2023-02-12 22:12:59,536][ INFO] Iters: 43, Total loss: 0.454, Loss x: 0.890, Loss s: 0.031, Loss w_fp: 0.005, Mask: 0.564
[2023-02-12 22:14:41,376][ INFO] Iters: 86, Total loss: 0.419, Loss x: 0.819, Loss s: 0.032, Loss w_fp: 0.006, Mask: 0.575
[2023-02-12 22:16:23,199][ INFO] Iters: 129, Total loss: 0.413, Loss x: 0.806, Loss s: 0.035, Loss w_fp: 0.006, Mask: 0.608
from unimatch.
LiheYoung
commented on September 25, 2024
According to the log, the difference is that I use 1 image per card, while you use 2 images.
The OHEM min_kept hyper-parameter is conditioned on the number of labeled images in each card. Since you have doubled the labeled images per card, I recommend also doubling the min_kept to 200,000. Nevertheless, if you want to fully reproduce our logs, the best practice may be using 1 image per card with 8 cards.
from unimatch.
JoyHuYY1412
commented on September 25, 2024
According to the log, the difference is that I use 1 image per card, while you use 2 images.
The OHEM
min_kepthyper-parameter is conditioned on the number of labeled images in each card. Since you have doubled the labeled images per card, I recommend also doubling themin_keptto200,000. Nevertheless, if you want to fully reproduce our logs, the best practice may be using 1 image per card with 8 cards.
Hi Young. Thank you for your advice. After I tried with 8 GPUs, the initial mask ratio seemed to be normal. However, the final result with 8 GPU is 74.70% (comparable with 4GPU results of 74.49% FYI). I checked the logs and found the highest accuracy before sliding window evaluation is 70.45%, which is also lower than your log.
/home/anaconda3/envs/unimatch2/lib/python3.6/site-packages/torch/distributed/launch.py:186: FutureWarning: The module torch.distributed.launch is deprecated
and will be removed in future. Use torchrun.
Note that --use_env is set by default in torchrun.
If your script expects --local_rank argument to be set, please
change it to read from os.environ['LOCAL_RANK'] instead. See
https://pytorch.org/docs/stable/distributed.html#launch-utility for
further instructions
FutureWarning,
WARNING:torch.distributed.run:
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
[2023-02-15 17:22:10,644][ INFO] {'backbone': 'resnet101',
'batch_size': 1,
'conf_thresh': 0.95,
'criterion': {'kwargs': {'ignore_index': 255,
'min_kept': 100000,
'thresh': 0.7},
'name': 'OHEM'},
'crop_size': 769,
'data_root': '/BS/SETR/work/Projects/UniMatch_ICON/data/cityscapes',
'dataset': 'cityscapes',
'dilations': [12, 24, 36],
'epochs': 240,
'lr': 0.005,
'lr_multi': 1.0,
'multi_grid': True,
'nclass': 19,
'replace_stride_with_dilation': [False, True, True]}
[2023-02-15 17:22:13,955][ INFO] Total params: 59.5M
[2023-02-15 17:22:27,051][ INFO] ===========> Epoch: 217, LR: 0.0006, Previous best: 70.45
[2023-02-15 17:23:11,050][ INFO] Iters: 0, Total loss: 0.011, Loss x: 0.020, Loss s: 0.001, Loss w_fp: 0.000, Mask: 0.995
[2023-02-15 17:24:08,730][ INFO] Iters: 43, Total loss: 0.060, Loss x: 0.105, Loss s: 0.027, Loss w_fp: 0.005, Mask: 0.947
[2023-02-15 17:25:05,775][ INFO] Iters: 86, Total loss: 0.063, Loss x: 0.109, Loss s: 0.031, Loss w_fp: 0.005, Mask: 0.944
[2023-02-15 17:26:02,629][ INFO] Iters: 129, Total loss: 0.065, Loss x: 0.111, Loss s: 0.034, Loss w_fp: 0.004, Mask: 0.943
[2023-02-15 17:26:59,569][ INFO] Iters: 172, Total loss: 0.066, Loss x: 0.114, Loss s: 0.033, Loss w_fp: 0.005, Mask: 0.942
[2023-02-15 17:27:56,447][ INFO] Iters: 215, Total loss: 0.068, Loss x: 0.115, Loss s: 0.037, Loss w_fp: 0.004, Mask: 0.943
[2023-02-15 17:28:53,283][ INFO] Iters: 258, Total loss: 0.068, Loss x: 0.116, Loss s: 0.036, Loss w_fp: 0.004, Mask: 0.944
[2023-02-15 17:29:50,094][ INFO] Iters: 301, Total loss: 0.067, Loss x: 0.115, Loss s: 0.035, Loss w_fp: 0.004, Mask: 0.943
[2023-02-15 17:30:47,222][ INFO] Iters: 344, Total loss: 0.066, Loss x: 0.114, Loss s: 0.034, Loss w_fp: 0.004, Mask: 0.943
[2023-02-15 17:31:01,883][ INFO] ***** Evaluation center_crop ***** >>>> meanIOU: 69.90
[2023-02-15 17:31:03,703][ INFO] ===========> Epoch: 218, LR: 0.0006, Previous best: 70.45
[2023-02-15 17:31:05,826][ INFO] Iters: 0, Total loss: 0.092, Loss x: 0.176, Loss s: 0.013, Loss w_fp: 0.002, Mask: 0.952
[2023-02-15 17:32:02,543][ INFO] Iters: 43, Total loss: 0.063, Loss x: 0.109, Loss s: 0.030, Loss w_fp: 0.003, Mask: 0.937
[2023-02-15 17:32:59,171][ INFO] Iters: 86, Total loss: 0.067, Loss x: 0.114, Loss s: 0.037, Loss w_fp: 0.004, Mask: 0.940
[2023-02-15 17:33:55,954][ INFO] Iters: 129, Total loss: 0.067, Loss x: 0.115, Loss s: 0.037, Loss w_fp: 0.004, Mask: 0.944
[2023-02-15 17:34:52,928][ INFO] Iters: 172, Total loss: 0.065, Loss x: 0.110, Loss s: 0.036, Loss w_fp: 0.004, Mask: 0.943
[2023-02-15 17:35:49,831][ INFO] Iters: 215, Total loss: 0.067, Loss x: 0.114, Loss s: 0.037, Loss w_fp: 0.004, Mask: 0.943
[2023-02-15 17:36:46,368][ INFO] Iters: 258, Total loss: 0.067, Loss x: 0.114, Loss s: 0.036, Loss w_fp: 0.004, Mask: 0.942
[2023-02-15 17:37:43,108][ INFO] Iters: 301, Total loss: 0.068, Loss x: 0.116, Loss s: 0.035, Loss w_fp: 0.004, Mask: 0.942
[2023-02-15 17:38:40,064][ INFO] Iters: 344, Total loss: 0.067, Loss x: 0.115, Loss s: 0.035, Loss w_fp: 0.004, Mask: 0.942
[2023-02-15 17:38:54,548][ INFO] ***** Evaluation center_crop ***** >>>> meanIOU: 69.73
[2023-02-15 17:38:56,438][ INFO] ===========> Epoch: 219, LR: 0.0006, Previous best: 70.45
[2023-02-15 17:38:58,564][ INFO] Iters: 0, Total loss: 0.124, Loss x: 0.247, Loss s: 0.000, Loss w_fp: 0.000, Mask: 1.000
[2023-02-15 17:39:55,383][ INFO] Iters: 43, Total loss: 0.058, Loss x: 0.100, Loss s: 0.029, Loss w_fp: 0.003, Mask: 0.948
[2023-02-15 17:40:52,321][ INFO] Iters: 86, Total loss: 0.061, Loss x: 0.105, Loss s: 0.029, Loss w_fp: 0.004, Mask: 0.948
[2023-02-15 17:41:49,140][ INFO] Iters: 129, Total loss: 0.063, Loss x: 0.109, Loss s: 0.030, Loss w_fp: 0.003, Mask: 0.946
[2023-02-15 17:42:45,886][ INFO] Iters: 172, Total loss: 0.062, Loss x: 0.108, Loss s: 0.030, Loss w_fp: 0.004, Mask: 0.944
[2023-02-15 17:43:42,740][ INFO] Iters: 215, Total loss: 0.062, Loss x: 0.107, Loss s: 0.030, Loss w_fp: 0.004, Mask: 0.942
[2023-02-15 17:44:39,465][ INFO] Iters: 258, Total loss: 0.062, Loss x: 0.108, Loss s: 0.030, Loss w_fp: 0.004, Mask: 0.942
[2023-02-15 17:45:36,309][ INFO] Iters: 301, Total loss: 0.063, Loss x: 0.109, Loss s: 0.030, Loss w_fp: 0.004, Mask: 0.942
[2023-02-15 17:46:33,310][ INFO] Iters: 344, Total loss: 0.063, Loss x: 0.109, Loss s: 0.029, Loss w_fp: 0.004, Mask: 0.942
[2023-02-15 17:46:47,653][ INFO] ***** Evaluation center_crop ***** >>>> meanIOU: 69.68
[2023-02-15 17:46:49,536][ INFO] ===========> Epoch: 220, LR: 0.0005, Previous best: 70.45
[2023-02-15 17:46:51,657][ INFO] Iters: 0, Total loss: 0.037, Loss x: 0.052, Loss s: 0.041, Loss w_fp: 0.003, Mask: 0.923
[2023-02-15 17:47:48,588][ INFO] Iters: 43, Total loss: 0.064, Loss x: 0.111, Loss s: 0.031, Loss w_fp: 0.004, Mask: 0.949
[2023-02-15 17:48:45,578][ INFO] Iters: 86, Total loss: 0.064, Loss x: 0.110, Loss s: 0.030, Loss w_fp: 0.004, Mask: 0.945
[2023-02-15 17:49:42,558][ INFO] Iters: 129, Total loss: 0.062, Loss x: 0.108, Loss s: 0.028, Loss w_fp: 0.004, Mask: 0.946
[2023-02-15 17:50:39,713][ INFO] Iters: 172, Total loss: 0.062, Loss x: 0.108, Loss s: 0.028, Loss w_fp: 0.004, Mask: 0.942
[2023-02-15 17:51:36,758][ INFO] Iters: 215, Total loss: 0.061, Loss x: 0.105, Loss s: 0.031, Loss w_fp: 0.004, Mask: 0.942
[2023-02-15 17:52:33,654][ INFO] Iters: 258, Total loss: 0.061, Loss x: 0.105, Loss s: 0.030, Loss w_fp: 0.004, Mask: 0.943
[2023-02-15 17:53:30,614][ INFO] Iters: 301, Total loss: 0.060, Loss x: 0.104, Loss s: 0.029, Loss w_fp: 0.004, Mask: 0.944
[2023-02-15 17:54:27,575][ INFO] Iters: 344, Total loss: 0.060, Loss x: 0.103, Loss s: 0.029, Loss w_fp: 0.003, Mask: 0.945
[2023-02-15 17:55:44,076][ INFO] ***** Evaluation sliding_window ***** >>>> meanIOU: 74.34
[2023-02-15 17:55:46,811][ INFO] ===========> Epoch: 221, LR: 0.0005, Previous best: 74.34
[2023-02-15 17:55:49,350][ INFO] Iters: 0, Total loss: 0.068, Loss x: 0.110, Loss s: 0.045, Loss w_fp: 0.007, Mask: 0.937
[2023-02-15 17:56:46,078][ INFO] Iters: 43, Total loss: 0.070, Loss x: 0.126, Loss s: 0.024, Loss w_fp: 0.005, Mask: 0.941
[2023-02-15 17:57:42,938][ INFO] Iters: 86, Total loss: 0.068, Loss x: 0.120, Loss s: 0.029, Loss w_fp: 0.004, Mask: 0.943
[2023-02-15 17:58:39,813][ INFO] Iters: 129, Total loss: 0.066, Loss x: 0.115, Loss s: 0.029, Loss w_fp: 0.005, Mask: 0.944
[2023-02-15 17:59:36,931][ INFO] Iters: 172, Total loss: 0.064, Loss x: 0.112, Loss s: 0.027, Loss w_fp: 0.004, Mask: 0.944
[2023-02-15 18:00:33,852][ INFO] Iters: 215, Total loss: 0.065, Loss x: 0.112, Loss s: 0.029, Loss w_fp: 0.004, Mask: 0.943
[2023-02-15 18:01:30,735][ INFO] Iters: 258, Total loss: 0.063, Loss x: 0.109, Loss s: 0.028, Loss w_fp: 0.004, Mask: 0.944
[2023-02-15 18:02:27,547][ INFO] Iters: 301, Total loss: 0.063, Loss x: 0.109, Loss s: 0.028, Loss w_fp: 0.004, Mask: 0.944
[2023-02-15 18:03:24,454][ INFO] Iters: 344, Total loss: 0.063, Loss x: 0.110, Loss s: 0.029, Loss w_fp: 0.004, Mask: 0.943
[2023-02-15 18:04:15,368][ INFO] ***** Evaluation sliding_window ***** >>>> meanIOU: 74.55
[2023-02-15 18:04:18,163][ INFO] ===========> Epoch: 222, LR: 0.0005, Previous best: 74.55
[2023-02-15 18:04:20,287][ INFO] Iters: 0, Total loss: 0.078, Loss x: 0.144, Loss s: 0.017, Loss w_fp: 0.007, Mask: 0.959
[2023-02-15 18:05:16,800][ INFO] Iters: 43, Total loss: 0.063, Loss x: 0.104, Loss s: 0.039, Loss w_fp: 0.004, Mask: 0.943
[2023-02-15 18:06:13,388][ INFO] Iters: 86, Total loss: 0.068, Loss x: 0.117, Loss s: 0.036, Loss w_fp: 0.004, Mask: 0.939
[2023-02-15 18:07:10,166][ INFO] Iters: 129, Total loss: 0.067, Loss x: 0.115, Loss s: 0.035, Loss w_fp: 0.004, Mask: 0.940
[2023-02-15 18:08:06,995][ INFO] Iters: 172, Total loss: 0.065, Loss x: 0.110, Loss s: 0.036, Loss w_fp: 0.004, Mask: 0.941
[2023-02-15 18:09:03,605][ INFO] Iters: 215, Total loss: 0.064, Loss x: 0.109, Loss s: 0.032, Loss w_fp: 0.004, Mask: 0.944
[2023-02-15 18:10:00,253][ INFO] Iters: 258, Total loss: 0.065, Loss x: 0.112, Loss s: 0.032, Loss w_fp: 0.004, Mask: 0.944
[2023-02-15 18:10:56,983][ INFO] Iters: 301, Total loss: 0.066, Loss x: 0.114, Loss s: 0.033, Loss w_fp: 0.004, Mask: 0.942
[2023-02-15 18:11:53,954][ INFO] Iters: 344, Total loss: 0.065, Loss x: 0.112, Loss s: 0.034, Loss w_fp: 0.004, Mask: 0.942
[2023-02-15 18:12:45,473][ INFO] ***** Evaluation sliding_window ***** >>>> meanIOU: 74.12
[2023-02-15 18:12:47,395][ INFO] ===========> Epoch: 223, LR: 0.0005, Previous best: 74.55
[2023-02-15 18:12:49,464][ INFO] Iters: 0, Total loss: 0.051, Loss x: 0.052, Loss s: 0.095, Loss w_fp: 0.004, Mask: 0.960
[2023-02-15 18:13:46,068][ INFO] Iters: 43, Total loss: 0.060, Loss x: 0.101, Loss s: 0.033, Loss w_fp: 0.004, Mask: 0.941
[2023-02-15 18:14:43,095][ INFO] Iters: 86, Total loss: 0.061, Loss x: 0.105, Loss s: 0.032, Loss w_fp: 0.004, Mask: 0.941
[2023-02-15 18:15:40,011][ INFO] Iters: 129, Total loss: 0.061, Loss x: 0.103, Loss s: 0.032, Loss w_fp: 0.004, Mask: 0.940
[2023-02-15 18:16:36,947][ INFO] Iters: 172, Total loss: 0.063, Loss x: 0.108, Loss s: 0.031, Loss w_fp: 0.004, Mask: 0.941
[2023-02-15 18:17:33,660][ INFO] Iters: 215, Total loss: 0.062, Loss x: 0.107, Loss s: 0.031, Loss w_fp: 0.004, Mask: 0.942
[2023-02-15 18:18:30,418][ INFO] Iters: 258, Total loss: 0.061, Loss x: 0.105, Loss s: 0.031, Loss w_fp: 0.004, Mask: 0.943
[2023-02-15 18:19:27,148][ INFO] Iters: 301, Total loss: 0.061, Loss x: 0.104, Loss s: 0.030, Loss w_fp: 0.004, Mask: 0.943
[2023-02-15 18:20:23,818][ INFO] Iters: 344, Total loss: 0.061, Loss x: 0.104, Loss s: 0.032, Loss w_fp: 0.004, Mask: 0.942
[2023-02-15 18:21:14,834][ INFO] ***** Evaluation sliding_window ***** >>>> meanIOU: 74.48
[2023-02-15 18:21:16,952][ INFO] ===========> Epoch: 224, LR: 0.0004, Previous best: 74.55
[2023-02-15 18:21:19,051][ INFO] Iters: 0, Total loss: 0.046, Loss x: 0.087, Loss s: 0.010, Loss w_fp: 0.002, Mask: 0.973
[2023-02-15 18:22:15,669][ INFO] Iters: 43, Total loss: 0.069, Loss x: 0.121, Loss s: 0.031, Loss w_fp: 0.003, Mask: 0.941
[2023-02-15 18:23:12,323][ INFO] Iters: 86, Total loss: 0.066, Loss x: 0.115, Loss s: 0.030, Loss w_fp: 0.003, Mask: 0.944
[2023-02-15 18:24:09,013][ INFO] Iters: 129, Total loss: 0.066, Loss x: 0.115, Loss s: 0.029, Loss w_fp: 0.004, Mask: 0.944
[2023-02-15 18:25:05,655][ INFO] Iters: 172, Total loss: 0.066, Loss x: 0.115, Loss s: 0.029, Loss w_fp: 0.004, Mask: 0.942
[2023-02-15 18:26:02,636][ INFO] Iters: 215, Total loss: 0.066, Loss x: 0.115, Loss s: 0.030, Loss w_fp: 0.004, Mask: 0.943
[2023-02-15 18:26:59,342][ INFO] Iters: 258, Total loss: 0.067, Loss x: 0.117, Loss s: 0.029, Loss w_fp: 0.004, Mask: 0.942
[2023-02-15 18:27:56,125][ INFO] Iters: 301, Total loss: 0.068, Loss x: 0.118, Loss s: 0.031, Loss w_fp: 0.004, Mask: 0.941
[2023-02-15 18:28:53,092][ INFO] Iters: 344, Total loss: 0.067, Loss x: 0.116, Loss s: 0.031, Loss w_fp: 0.004, Mask: 0.941
[2023-02-15 18:29:44,014][ INFO] ***** Evaluation sliding_window ***** >>>> meanIOU: 74.45
[2023-02-15 18:29:46,033][ INFO] ===========> Epoch: 225, LR: 0.0004, Previous best: 74.55
[2023-02-15 18:29:48,120][ INFO] Iters: 0, Total loss: 0.039, Loss x: 0.066, Loss s: 0.020, Loss w_fp: 0.002, Mask: 0.974
[2023-02-15 18:30:44,912][ INFO] Iters: 43, Total loss: 0.066, Loss x: 0.109, Loss s: 0.043, Loss w_fp: 0.004, Mask: 0.937
[2023-02-15 18:31:41,894][ INFO] Iters: 86, Total loss: 0.063, Loss x: 0.106, Loss s: 0.037, Loss w_fp: 0.004, Mask: 0.938
[2023-02-15 18:32:38,868][ INFO] Iters: 129, Total loss: 0.062, Loss x: 0.103, Loss s: 0.036, Loss w_fp: 0.004, Mask: 0.939
[2023-02-15 18:33:35,892][ INFO] Iters: 172, Total loss: 0.063, Loss x: 0.107, Loss s: 0.036, Loss w_fp: 0.004, Mask: 0.936
[2023-02-15 18:34:32,888][ INFO] Iters: 215, Total loss: 0.065, Loss x: 0.110, Loss s: 0.035, Loss w_fp: 0.004, Mask: 0.936
[2023-02-15 18:35:29,913][ INFO] Iters: 258, Total loss: 0.064, Loss x: 0.108, Loss s: 0.035, Loss w_fp: 0.004, Mask: 0.937
[2023-02-15 18:36:26,858][ INFO] Iters: 301, Total loss: 0.064, Loss x: 0.108, Loss s: 0.036, Loss w_fp: 0.004, Mask: 0.936
[2023-02-15 18:37:23,814][ INFO] Iters: 344, Total loss: 0.063, Loss x: 0.107, Loss s: 0.034, Loss w_fp: 0.004, Mask: 0.937
[2023-02-15 18:38:15,571][ INFO] ***** Evaluation sliding_window ***** >>>> meanIOU: 74.38
[2023-02-15 18:38:17,509][ INFO] ===========> Epoch: 226, LR: 0.0004, Previous best: 74.55
[2023-02-15 18:38:19,647][ INFO] Iters: 0, Total loss: 0.046, Loss x: 0.077, Loss s: 0.027, Loss w_fp: 0.004, Mask: 0.969
[2023-02-15 18:39:16,248][ INFO] Iters: 43, Total loss: 0.065, Loss x: 0.116, Loss s: 0.024, Loss w_fp: 0.003, Mask: 0.950
[2023-02-15 18:40:12,942][ INFO] Iters: 86, Total loss: 0.054, Loss x: 0.092, Loss s: 0.028, Loss w_fp: 0.003, Mask: 0.948
[2023-02-15 18:41:09,830][ INFO] Iters: 129, Total loss: 0.056, Loss x: 0.097, Loss s: 0.026, Loss w_fp: 0.003, Mask: 0.947
[2023-02-15 18:42:06,715][ INFO] Iters: 172, Total loss: 0.058, Loss x: 0.099, Loss s: 0.030, Loss w_fp: 0.004, Mask: 0.944
[2023-02-15 18:43:03,686][ INFO] Iters: 215, Total loss: 0.060, Loss x: 0.102, Loss s: 0.032, Loss w_fp: 0.004, Mask: 0.945
[2023-02-15 18:44:00,430][ INFO] Iters: 258, Total loss: 0.061, Loss x: 0.104, Loss s: 0.032, Loss w_fp: 0.004, Mask: 0.944
[2023-02-15 18:44:57,014][ INFO] Iters: 301, Total loss: 0.062, Loss x: 0.107, Loss s: 0.031, Loss w_fp: 0.004, Mask: 0.944
[2023-02-15 18:45:53,783][ INFO] Iters: 344, Total loss: 0.063, Loss x: 0.107, Loss s: 0.034, Loss w_fp: 0.004, Mask: 0.943
[2023-02-15 18:46:44,627][ INFO] ***** Evaluation sliding_window ***** >>>> meanIOU: 74.42
[2023-02-15 18:46:46,535][ INFO] ===========> Epoch: 227, LR: 0.0004, Previous best: 74.55
[2023-02-15 18:46:48,709][ INFO] Iters: 0, Total loss: 0.152, Loss x: 0.292, Loss s: 0.022, Loss w_fp: 0.002, Mask: 0.973
[2023-02-15 18:47:45,475][ INFO] Iters: 43, Total loss: 0.074, Loss x: 0.129, Loss s: 0.032, Loss w_fp: 0.004, Mask: 0.947
[2023-02-15 18:48:42,514][ INFO] Iters: 86, Total loss: 0.063, Loss x: 0.110, Loss s: 0.028, Loss w_fp: 0.004, Mask: 0.945
[2023-02-15 18:49:39,331][ INFO] Iters: 129, Total loss: 0.063, Loss x: 0.109, Loss s: 0.031, Loss w_fp: 0.004, Mask: 0.944
[2023-02-15 18:50:36,100][ INFO] Iters: 172, Total loss: 0.066, Loss x: 0.113, Loss s: 0.032, Loss w_fp: 0.004, Mask: 0.945
[2023-02-15 18:51:32,916][ INFO] Iters: 215, Total loss: 0.066, Loss x: 0.113, Loss s: 0.035, Loss w_fp: 0.004, Mask: 0.943
from unimatch.
LiheYoung
commented on September 25, 2024
My officially reported result is 75.7, while as I re-run the code, the obtained result is 76.0 (as present in the log). And your result is 74.5. I am sorry that I currently can not think of other reasons for this except randomness and different environments. So could you try other splits? such as 1/8 or 1/4, which will be more stable. If resources are allowed, you can also give a second try to the 1/16 split. Thank you.
from unimatch.
LiheYoung
commented on September 25, 2024
According to our later explorations, there is another practice to improve performance on Cityscapes by 1% generally. Please modify the Cityscapes config:
- conf_thresh: 0.95 -> 0
And if you hope to decrease training time and use less GPU memory, please modify the Cityscapes config:
- replace_stride_with_dilation: [False, True, True] -> [False, False, True]
- dilations: [12, 24, 36] -> [6, 12, 18]
- multi_grid: True -> False
from unimatch.
JoyHuYY1412
commented on September 25, 2024
According to our later explorations, there is another practice to improve performance on Cityscapes by 1% generally. Please modify the Cityscapes config:
- conf_thresh: 0.95 -> 0
And if you hope to decrease training time and use less GPU memory, please modify the Cityscapes config:
- replace_stride_with_dilation: [False, True, True] -> [False, False, True]
- dilations: [12, 24, 36] -> [6, 12, 18]
- multi_grid: True -> False
Thank you for your reply! I will check other splits and try your suggestions. Thanks again.
from unimatch.
Related Issues (20)
- 关于医学图像分割场景下的结果 HOT 4
- Image-level Perturbations in LEVIR HOT 1
- Pascal Voc 数据集的checkpoint HOT 19
- 关于特征空间扰动 HOT 4
- 关于第四页的Algorithm 1 HOT 2
- Question about the batch size
- Reproducability of PASCAL VOC HQ dataset results HOT 4
- 关于在Transformer系列模型使用 HOT 2
- Can't find Resnet 101
- How much memory its necessary? HOT 1
- 轻量级backbone及训练问题 HOT 1
- Question about crop size HOT 2
- 关于sliding_window HOT 1
- 复现训练代码
- 256×256的分辨率需要多少的GPU运行内存 HOT 1
- 评估和可视化代码 HOT 2
- color_map function in the utils.py file HOT 1
- Minor Typo HOT 1
- 关于自己数据集的问题 HOT 2
- 关于您该项目的实验 HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from unimatch.