samsunglabs/fcaf3d

[ECCV2022] FCAF3D: Fully Convolutional Anchor-Free 3D Object Detection

FCAF3D: Fully Convolutional Anchor-Free 3D Object Detection

News:

• 🚀 June, 2023. We add ScanNet-pretrained S3DIS model and log significantly pushing forward state-of-the-art.
• 🔥 February, 2023. Feel free to visit our new FCAF3D-based 3D instance segmentation TD3D and real-time 3D object detection TR3D.
• 🔥 August, 2022. FCAF3D is now fully supported in mmdetection3d.
• 🔥 July, 2022. Our paper is accepted at ECCV 2022.
• 🔥 March, 2022. We have updated the preprint adding more comparison with fully convolutional GSDN baseline.
• 🔥 December, 2021. FCAF3D is now state-of-the-art on paperswithcode on ScanNet, SUN RGB-D, and S3DIS.

This repository contains an implementation of FCAF3D, a 3D object detection method introduced in our paper:

FCAF3D: Fully Convolutional Anchor-Free 3D Object Detection
Danila Rukhovich, Anna Vorontsova, Anton Konushin
Samsung Research
https://arxiv.org/abs/2112.00322

Installation

For convenience, we provide a Dockerfile.

Alternatively, you can install all required packages manually. This implementation is based on mmdetection3d framework. Please refer to the original installation guide getting_started.md, replacing open-mmlab/mmdetection3d with samsunglabs/fcaf3d. Also, MinkowskiEngine and rotated_iou should be installed with these commands.

Most of the FCAF3D-related code locates in the following files: detectors/single_stage_sparse.py, dense_heads/fcaf3d_neck_with_head.py, backbones/me_resnet.py.

Getting Started

Please see getting_started.md for basic usage examples. We follow the mmdetection3d data preparation protocol described in scannet, sunrgbd, and s3dis. The only difference is that we do not sample 50,000 points from each point cloud in SUN RGB-D, using all points.

Training

To start training, run dist_train with FCAF3D configs:

bash tools/dist_train.sh configs/fcaf3d/fcaf3d_scannet-3d-18class.py 2

Testing

Test pre-trained model using dist_test with FCAF3D configs:

bash tools/dist_test.sh configs/fcaf3d/fcaf3d_scannet-3d-18class.py \
    work_dirs/fcaf3d_scannet-3d-18class/latest.pth 2 --eval mAP

Visualization

Visualizations can be created with test script. For better visualizations, you may set score_thr in configs to 0.20:

python tools/test.py configs/fcaf3d/fcaf3d_scannet-3d-18class.py \
    work_dirs/fcaf3d_scannet-3d-18class/latest.pth --eval mAP --show \
    --show-dir work_dirs/fcaf3d_scannet-3d-18class

Models

The metrics are obtained in 5 training runs followed by 5 test runs. We report both the best and the average values (the latter are given in round brackets).

For VoteNet and ImVoteNet, we provide the configs and checkpoints with our Mobius angle parametrization. For ImVoxelNet, please refer to the imvoxelnet repository as it is not currently supported in mmdetection3d for indoor datasets. Inference speed (scenes per second) is measured on a single NVidia GTX1080Ti. Please, note that ScanNet-pretrained S3DIS model was actually trained in the original open-mmlab/mmdetection3d codebase, so it can be inferenced only in their repo.

FCAF3D

Dataset	[email protected]	[email protected]	Download
ScanNet	71.5 (70.7)	57.3 (56.0)	model | log | config
SUN RGB-D	64.2 (63.8)	48.9 (48.2)	model | log | config
S3DIS	66.7 (64.9)	45.9 (43.8)	model | log | config
S3DIS ScanNet-pretrained	75.7 (74.1)	58.2 (56.1)	model | log | config

Faster FCAF3D on ScanNet

Backbone	Voxel size	[email protected]	[email protected]	Scenes per sec.	Download
HDResNet34	0.01	70.7	56.0	8.0	see table above
HDResNet34:3	0.01	69.8	53.6	12.2	model | log | config
HDResNet34:2	0.02	63.1	46.8	31.5	model | log | config

VoteNet on SUN RGB-D

Source	[email protected]	[email protected]	Download
mmdetection3d	59.1	35.8	instruction
ours	61.1 (60.5)	40.4 (39.5)	model | log | config

ImVoteNet on SUN RGB-D

Source	[email protected]	[email protected]	Download
mmdetection3d	64.0	37.8	instruction
ours	64.6 (64.1)	40.8 (39.8)	model | log | config

Comparison with state-of-the-art on ScanNet

Example Detections

Citation

If you find this work useful for your research, please cite our paper:

@inproceedings{rukhovich2022fcaf3d,
  title={FCAF3D: fully convolutional anchor-free 3D object detection},
  author={Rukhovich, Danila and Vorontsova, Anna and Konushin, Anton},
  booktitle={European Conference on Computer Vision},
  pages={477--493},
  year={2022},
  organization={Springer}
}

fcaf3d's Issues

log information

Hello,I want to know which .py can generate the following log information? I have been looking for a long time,but I have not found anything related.Thank you for your reply

+---------+---------+---------+---------+---------+
| classes | AP_0.25 | AR_0.25 | AP_0.50 | AR_0.50 |
+---------+---------+---------+---------+---------+
| chair | 0.2800 | 0.2800 | 0.2400 | 0.2400 |
+---------+---------+---------+---------+---------+
| Overall | 0.2800 | 0.2800 | 0.2400 | 0.2400 |
+---------+---------+---------+---------+---------+
2022-06-01 15:01:57,371 - mmdet - INFO - Exp name: fcaf3d_s3dis-3d-5class.py
2022-06-01 15:01:57,371 - mmdet - INFO - Epoch(val) [8][8] chair_AP_0.25: 0.2800, mAP_0.25: 0.2800, chair_rec_0.25: 0.2800, mAR_0.25: 0.2800, chair_AP_0.50: 0.2400, mAP_0.50: 0.2400, chair_rec_0.50: 0.2400, mAR_0.50: 0.2400
2022-06-01 15:02:14,503 - mmdet - INFO - Epoch [9][50/72] lr: 1.000e-04, eta: 0:01:02, time: 0.343, data_time: 0.060, memory: 1532, loss_centerness: 0.4974, loss_bbox: 0.3062, loss_cls: 0.1228, loss: 0.9264, grad_norm: 4.4644
2022-06-01 15:02:20,639 - mmdet - INFO - Saving checkpoint at 9 epochs

Can you please share the docker file for CUDA 11.2

I tried to set up the environment to training FCAF3D model. I followed your guide to set up mmdet & mmdet3d library with CUDA 11.2, but the last step for installing MinkowskiEngine, I couldn't install this library with CUDA 11.2 . If anybody have already done for training FCAF3D with CUDA 11.2, please share with me your docker. Thank you so much.

some issues for the evaluation results of fcaf3d

Hello：
There is a obvious difference between the actual evaluation results of the algorithm and those described in the paper, the cause of the problem has not been found yet, please give me some suggestions.

1.Evaluation results for the model provided in github of SamsungLabs/fcaf3d：(12 epoches)
1.1 sunrgbd

---

Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.

---

Use load_from_local loader
Use load_from_local loader
[ ] 0/5050, elapsed: [>>>>>>>>>>>>>>>>] 5050/5050, 15.9 task/s, elapsed: 318s, ETA: 0s
+-------------+---------+---------+---------+---------+
| classes | AP_0.25 | AR_0.25 | AP_0.50 | AR_0.50 |
+-------------+---------+---------+---------+---------+
| bed | 0.0496 | 0.7612 | 0.0094 | 0.5883 |
| table | 0.1570 | 0.8514 | 0.1222 | 0.7909 |
| sofa | 0.2261 | 0.8469 | 0.1440 | 0.7735 |
| chair | 0.1985 | 0.7662 | 0.1671 | 0.7272 |
| toilet | 0.3586 | 0.9379 | 0.2778 | 0.8552 |
| desk | 0.0970 | 0.7641 | 0.0665 | 0.6982 |
| dresser | 0.2063 | 0.9128 | 0.1956 | 0.8670 |
| night_stand | 0.4174 | 0.9608 | 0.4127 | 0.9451 |
| bookshelf | 0.1585 | 0.8865 | 0.1489 | 0.8511 |
| bathtub | 0.3658 | 0.9592 | 0.3561 | 0.8571 |
+-------------+---------+---------+---------+---------+
| Overall | 0.2235 | 0.8647 | 0.1900 | 0.7954 |
+-------------+---------+---------+---------+---------+
{'bed_AP_0.25': 0.049614179879426956, 'table_AP_0.25': 0.15698249638080597, 'sofa_AP_0.25': 0.2261233627796173, 'chair_AP_0.25': 0.19846875965595245, 'toilet_AP_0.25': 0.35864582657814026, 'desk_AP_0.25': 0.09701932221651077, 'dresser_AP_0.25': 0.20626720786094666, 'night_stand_AP_0.25': 0.41742968559265137, 'bookshelf_AP_0.25': 0.15852327644824982, 'bathtub_AP_0.25': 0.3657997250556946, 'mAP_0.25': 0.22348737716674805, 'bed_rec_0.25': 0.7611650485436893, 'table_rec_0.25': 0.8513628620102215, 'sofa_rec_0.25': 0.84688995215311, 'chair_rec_0.25': 0.7661741214057508, 'toilet_rec_0.25': 0.9379310344827586, 'desk_rec_0.25': 0.7640807651434643, 'dresser_rec_0.25': 0.9128440366972477, 'night_stand_rec_0.25': 0.9607843137254902, 'bookshelf_rec_0.25': 0.8865248226950354, 'bathtub_rec_0.25': 0.9591836734693877, 'mAR_0.25': 0.8646940630326154, 'bed_AP_0.50': 0.009411284700036049, 'table_AP_0.50': 0.12223674356937408, 'sofa_AP_0.50': 0.14403118193149567, 'chair_AP_0.50': 0.1670922487974167, 'toilet_AP_0.50': 0.27775609493255615, 'desk_AP_0.50': 0.06645272672176361, 'dresser_AP_0.50': 0.19564399123191833, 'night_stand_AP_0.50': 0.41268840432167053, 'bookshelf_AP_0.50': 0.14886672794818878, 'bathtub_AP_0.50': 0.35609957575798035, 'mAP_0.50': 0.19002790749073029, 'bed_rec_0.50': 0.5883495145631068, 'table_rec_0.50': 0.790885860306644, 'sofa_rec_0.50': 0.773524720893142, 'chair_rec_0.50': 0.7272364217252396, 'toilet_rec_0.50': 0.8551724137931035, 'desk_rec_0.50': 0.6981934112646121, 'dresser_rec_0.50': 0.8669724770642202, 'night_stand_rec_0.50': 0.9450980392156862, 'bookshelf_rec_0.50': 0.851063829787234, 'bathtub_rec_0.50': 0.8571428571428571, 'mAR_0.50': 0.7953639545755846}

1.2 scannet

---

Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune
the variable for optimal performance in your application as needed.

---

Use load_from_local loader
Use load_from_local loader
[ ] 0/312, elapsed: 0[>>>>>>>>>>>>>>>>>>>] 312/312, 10.4 task/s, elapsed: 30s, ETA: 0s
+----------------+---------+---------+---------+---------+
| classes | AP_0.25 | AR_0.25 | AP_0.50 | AR_0.50 |
+----------------+---------+---------+---------+---------+
| cabinet | 0.0077 | 0.2742 | 0.0052 | 0.2419 |
| bed | 0.0003 | 0.0370 | 0.0000 | 0.0247 |
| chair | 0.0193 | 0.2661 | 0.0138 | 0.2354 |
| sofa | 0.0015 | 0.0825 | 0.0005 | 0.0825 |
| table | 0.0016 | 0.1429 | 0.0011 | 0.1229 |
| door | 0.0104 | 0.2441 | 0.0079 | 0.2313 |
| window | 0.0028 | 0.1773 | 0.0021 | 0.1631 |
| bookshelf | 0.0081 | 0.1558 | 0.0042 | 0.1429 |
| picture | 0.0028 | 0.2658 | 0.0025 | 0.2568 |
| counter | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| desk | 0.0012 | 0.0866 | 0.0010 | 0.0709 |
| curtain | 0.0000 | 0.0448 | 0.0000 | 0.0448 |
| refrigerator | 0.0000 | 0.0351 | 0.0000 | 0.0000 |
| showercurtrain | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| toilet | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| sink | 0.0000 | 0.0102 | 0.0000 | 0.0000 |
| bathtub | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| garbagebin | 0.0048 | 0.2113 | 0.0039 | 0.1811 |
+----------------+---------+---------+---------+---------+
| Overall | 0.0034 | 0.1130 | 0.0024 | 0.0999 |
+----------------+---------+---------+---------+---------+
{'cabinet_AP_0.25': 0.007672073319554329, 'bed_AP_0.25': 0.0003270927118137479, 'chair_AP_0.25': 0.019300922751426697, 'sofa_AP_0.25': 0.0015425414312630892, 'table_AP_0.25': 0.0016294954111799598, 'door_AP_0.25': 0.010358083993196487, 'window_AP_0.25': 0.0027569225057959557, 'bookshelf_AP_0.25': 0.008131678216159344, 'picture_AP_0.25': 0.002778079127892852, 'counter_AP_0.25': 0.0, 'desk_AP_0.25': 0.0011750475969165564, 'curtain_AP_0.25': 2.2466720110969618e-05, 'refrigerator_AP_0.25': 1.3386352293309756e-05, 'showercurtrain_AP_0.25': 0.0, 'toilet_AP_0.25': 0.0, 'sink_AP_0.25': 1.440034111510613e-06, 'bathtub_AP_0.25': 0.0, 'garbagebin_AP_0.25': 0.00475729675963521, 'mAP_0.25': 0.003359251655638218, 'cabinet_rec_0.25': 0.27419354838709675, 'bed_rec_0.25': 0.037037037037037035, 'chair_rec_0.25': 0.26608187134502925, 'sofa_rec_0.25': 0.08247422680412371, 'table_rec_0.25': 0.14285714285714285, 'door_rec_0.25': 0.24411134903640258, 'window_rec_0.25': 0.1773049645390071, 'bookshelf_rec_0.25': 0.15584415584415584, 'picture_rec_0.25': 0.26576576576576577, 'counter_rec_0.25': 0.0, 'desk_rec_0.25': 0.08661417322834646, 'curtain_rec_0.25': 0.04477611940298507, 'refrigerator_rec_0.25': 0.03508771929824561, 'showercurtrain_rec_0.25': 0.0, 'toilet_rec_0.25': 0.0, 'sink_rec_0.25': 0.01020408163265306, 'bathtub_rec_0.25': 0.0, 'garbagebin_rec_0.25': 0.21132075471698114, 'mAR_0.25': 0.11298182832749847, 'cabinet_AP_0.50': 0.005249418318271637, 'bed_AP_0.50': 7.024568276392529e-06, 'chair_AP_0.50': 0.013795703649520874, 'sofa_AP_0.50': 0.0005232820985838771, 'table_AP_0.50': 0.001064108102582395, 'door_AP_0.50': 0.007914243265986443, 'window_AP_0.50': 0.0021169877145439386, 'bookshelf_AP_0.50': 0.0042017120867967606, 'picture_AP_0.50': 0.0025250331964343786, 'counter_AP_0.50': 0.0, 'desk_AP_0.50': 0.0010463166981935501, 'curtain_AP_0.50': 2.2466720110969618e-05, 'refrigerator_AP_0.50': 0.0, 'showercurtrain_AP_0.50': 0.0, 'toilet_AP_0.50': 0.0, 'sink_AP_0.50': 0.0, 'bathtub_AP_0.50': 0.0, 'garbagebin_AP_0.50': 0.003941362723708153, 'mAP_0.50': 0.0023559811525046825, 'cabinet_rec_0.
50': 0.24193548387096775, 'bed_rec_0.50': 0.024691358024691357, 'chair_rec_0.50': 0.23538011695906433, 'sofa_rec_0.50': 0.08247422680412371,
'table_rec_0.50': 0.12285714285714286, 'door_rec_0.50': 0.23126338329764454, 'window_rec_0.50': 0.16312056737588654, 'bookshelf_rec_0.50':
0.14285714285714285, 'picture_rec_0.50': 0.25675675675675674, 'counter_rec_0.50': 0.0, 'desk_rec_0.50': 0.07086614173228346, 'curtain_rec_0.
50': 0.04477611940298507, 'refrigerator_rec_0.50': 0.0, 'showercurtrain_rec_0.50': 0.0, 'toilet_rec_0.50': 0.0, 'sink_rec_0.50': 0.0, 'batht
ub_rec_0.50': 0.0, 'garbagebin_rec_0.50': 0.1811320754716981, 'mAR_0.50': 0.0998950286339104}

2.Evaluation results for the model trained by myself：(12 epoches)
2.1 sunrgbd

---

Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.

---

Use load_from_local loader
Use load_from_local loader
[ ] 0/5050, elapsed: [>>>>>>>>>>>>>>>] 5050/5050, 0.3 task/s, elapsed: 16300s, ETA: 0s
+-------------+---------+---------+---------+---------+
| classes | AP_0.25 | AR_0.25 | AP_0.50 | AR_0.50 |
+-------------+---------+---------+---------+---------+
| bed | 0.5560 | 0.9903 | 0.4873 | 0.8738 |
| table | 0.3110 | 0.9817 | 0.2184 | 0.7785 |
| sofa | 0.4583 | 0.9841 | 0.4016 | 0.8740 |
| chair | 0.5615 | 0.9633 | 0.4967 | 0.8643 |
| toilet | 0.6838 | 0.9931 | 0.6078 | 0.8966 |
| desk | 0.1938 | 0.9644 | 0.0907 | 0.6791 |
| dresser | 0.2366 | 0.9541 | 0.1876 | 0.7752 |
| night_stand | 0.4379 | 0.9765 | 0.3945 | 0.8824 |
| bookshelf | 0.2321 | 0.9184 | 0.0987 | 0.5426 |
| bathtub | 0.3912 | 0.9796 | 0.3296 | 0.8367 |
+-------------+---------+---------+---------+---------+
| Overall | 0.4062 | 0.9705 | 0.3313 | 0.8003 |
+-------------+---------+---------+---------+---------+
{'bed_AP_0.25': 0.5559849143028259, 'table_AP_0.25': 0.3109968602657318, 'sofa_AP_0.25': 0.4582793712615967, 'chair_AP_0.25': 0.5614634156227112, 'toilet_AP_0.25': 0.6837515830993652, 'desk_AP_0.25': 0.1938285529613495, 'dresser_AP_0.25': 0.23655451834201813, 'night_stand_AP_0.25': 0.43787485361099243, 'bookshelf_AP_0.25': 0.2321251481771469, 'bathtub_AP_0.25': 0.3912363648414612, 'mAP_0.25': 0.4062095582485199, 'bed_rec_0.25': 0.9902912621359223, 'table_rec_0.25': 0.981686541737649, 'sofa_rec_0.25': 0.9840510366826156, 'chair_rec_0.25': 0.963258785942492, 'toilet_rec_0.25': 0.993103448275862, 'desk_rec_0.25': 0.9643995749202976, 'dresser_rec_0.25': 0.9541284403669725, 'night_stand_rec_0.25': 0.9764705882352941, 'bookshelf_rec_0.25': 0.9184397163120568, 'bathtub_rec_0.25': 0.9795918367346939, 'mAR_0.25': 0.9705421231343856, 'bed_AP_0.50': 0.4872896671295166, 'table_AP_0.50': 0.21839171648025513, 'sofa_AP_0.50': 0.40156134963035583, 'chair_AP_0.50': 0.4966593384742737, 'toilet_AP_0.50': 0.607752799987793, 'desk_AP_0.50': 0.09067006409168243, 'dresser_AP_0.50': 0.18762624263763428, 'night_stand_AP_0.50': 0.39452967047691345, 'bookshelf_AP_0.50': 0.09874459356069565, 'bathtub_AP_0.50': 0.32958272099494934, 'mAP_0.50': 0.33128082752227783, 'bed_rec_0.50': 0.8737864077669902, 'table_rec_0.50': 0.778534923339012, 'sofa_rec_0.50': 0.8740031897926634, 'chair_rec_0.50': 0.8643170926517572, 'toilet_rec_0.50': 0.896551724137931, 'desk_rec_0.50': 0.6790648246546227, 'dresser_rec_0.50': 0.7752293577981652, 'night_stand_rec_0.50': 0.8823529411764706, 'bookshelf_rec_0.50': 0.5425531914893617, 'bathtub_rec_0.50': 0.8367346938775511, 'mAR_0.50': 0.8003128346684527}

2.2 scannet

---

Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tun$
the variable for optimal performance in your application as needed.

---

Use load_from_local loader
Use load_from_local loader
[ ] 0/312, elapsed: 0[>>>>>>> ] 116/312, 0.2 task/s, elapsed[ ] 117/312, 0.2 t$
sk/s, elapsed: 547s,[ ] 118/312, 0.2 task/s, elapsed: 547s, ETA: 89[>>>>>>>>>>>>> ] 240/312, [> ] 241/312, 0.2 task/s, elapsed: 1262$
[> ] 242/312, 0.2 task/s, elapsed: 1262s[> ] 243/312, 0.2 task/s, elap[>>>>>>>>>>>>>>>>>>] 312/312, 0.2 task/s, elapsed: 1531s, ETA: 0s
+----------------+---------+---------+---------+---------+
| classes | AP_0.25 | AR_0.25 | AP_0.50 | AR_0.50 |
+----------------+---------+---------+---------+---------+
| cabinet | 0.0089 | 0.2500 | 0.0071 | 0.2419 |
| bed | 0.0022 | 0.0494 | 0.0005 | 0.0123 |
| chair | 0.0195 | 0.2661 | 0.0131 | 0.2383 |
| sofa | 0.0009 | 0.0825 | 0.0000 | 0.0309 |
| table | 0.0030 | 0.1400 | 0.0021 | 0.1257 |
| door | 0.0072 | 0.2698 | 0.0066 | 0.2548 |
| window | 0.0030 | 0.1844 | 0.0020 | 0.1525 |
| bookshelf | 0.0041 | 0.1429 | 0.0031 | 0.1429 |
| picture | 0.0049 | 0.2523 | 0.0045 | 0.2342 |
| counter | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| desk | 0.0041 | 0.0709 | 0.0041 | 0.0551 |
| curtain | 0.0001 | 0.0597 | 0.0001 | 0.0597 |
| refrigerator | 0.0001 | 0.0877 | 0.0001 | 0.0877 |
| showercurtrain | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| toilet | 0.0005 | 0.0345 | 0.0001 | 0.0345 |
| sink | 0.0000 | 0.0408 | 0.0000 | 0.0408 |
| bathtub | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| garbagebin | 0.0076 | 0.2038 | 0.0063 | 0.1887 |
+----------------+---------+---------+---------+---------+
| Overall | 0.0037 | 0.1186 | 0.0028 | 0.1056 |
+----------------+---------+---------+---------+---------+
{'cabinet_AP_0.25': 0.0088879419490695, 'bed_AP_0.25': 0.002196365036070347, 'chair_AP_0.25': 0.019513951614499092, 'sofa_AP_0.25': 0.00092$
0919025614858, 'table_AP_0.25': 0.0029890856239944696, 'door_AP_0.25': 0.007219596300274134, 'window_AP_0.25': 0.002968191634863615, 'books$
elf_AP_0.25': 0.0041231452487409115, 'picture_AP_0.25': 0.004887890536338091, 'counter_AP_0.25': 0.0, 'desk_AP_0.25': 0.004088526125997305,
'curtain_AP_0.25': 0.0001449216069886461, 'refrigerator_AP_0.25': 0.00010905037925112993, 'showercurtrain_AP_0.25': 0.0, 'toilet_AP_0.25': $
.0005089639453217387, 'sink_AP_0.25': 3.358132016728632e-05, 'bathtub_AP_0.25': 0.0, 'garbagebin_AP_0.25': 0.007584888022392988, 'mAP_0.25'$
0.003676677355542779, 'cabinet_rec_0.25': 0.25, 'bed_rec_0.25': 0.04938271604938271, 'chair_rec_0.25': 0.26608187134502925, 'sofa_rec_0.25$
: 0.08247422680412371, 'table_rec_0.25': 0.14, 'door_rec_0.25': 0.2698072805139186, 'window_rec_0.25': 0.18439716312056736, 'bookshelf_rec_0
.25': 0.14285714285714285, 'picture_rec_0.25': 0.25225225225225223, 'counter_rec_0.25': 0.0, 'desk_rec_0.25': 0.07086614173228346, 'curtain_
rec_0.25': 0.05970149253731343, 'refrigerator_rec_0.25': 0.08771929824561403, 'showercurtrain_rec_0.25': 0.0, 'toilet_rec_0.25': 0.034482758
620689655, 'sink_rec_0.25': 0.04081632653061224, 'bathtub_rec_0.25': 0.0, 'garbagebin_rec_0.25': 0.2037735849056604, 'mAR_0.25': 0.118589569
75081056, 'cabinet_AP_0.50': 0.007079910486936569, 'bed_AP_0.50': 0.0005367686389945447, 'chair_AP_0.50': 0.013134335167706013, 'sofa_AP_0.5
0': 4.525698022916913e-05, 'table_AP_0.50': 0.0021134275011718273, 'door_AP_0.50': 0.00658177025616169, 'window_AP_0.50': 0.0019997931085526
943, 'bookshelf_AP_0.50': 0.0031221157405525446, 'picture_AP_0.50': 0.004526467993855476, 'counter_AP_0.50': 0.0, 'desk_AP_0.50': 0.00407800
916582346, 'curtain_AP_0.50': 0.0001449216069886461, 'refrigerator_AP_0.50': 0.0001018800976453349, 'showercurtrain_AP_0.50': 0.0, 'toilet_A
P_0.50': 5.409148434409872e-05, 'sink_AP_0.50': 3.358132016728632e-05, 'bathtub_AP_0.50': 0.0, 'garbagebin_AP_0.50': 0.006303954869508743, '
mAP_0.50': 0.0027697933837771416, 'cabinet_rec_0.50': 0.24193548387096775, 'bed_rec_0.50': 0.012345679012345678, 'chair_rec_0.50': 0.2383040
9356725146, 'sofa_rec_0.50': 0.030927835051546393, 'table_rec_0.50': 0.12571428571428572, 'door_rec_0.50': 0.25481798715203424, 'window_rec_
0.50': 0.1524822695035461, 'bookshelf_rec_0.50': 0.14285714285714285, 'picture_rec_0.50': 0.23423423423423423, 'counter_rec_0.50': 0.0, 'des
k_rec_0.50': 0.05511811023622047, 'curtain_rec_0.50': 0.05970149253731343, 'refrigerator_rec_0.50': 0.08771929824561403, 'showercurtrain_rec

mAP result question

Hi,

I have run the code on my local server, but get a little different mAP results compared to the values reported in the paper. I guess maybe this is due to the dataset and dataloader? Here is the experiment.

I eval your released model on ScanNet V2 Val and get this result:

On ScanNet, The [email protected] in paper is 0.715 but the eval result is 0.7063 and [email protected] in paper is 0.573 but the eval result is 0.5661. I have run many times, the [email protected] is always around 0.707 and [email protected] is always around 0.566. Maybe this is due to the dataset and dataloader. I want to figure out why there is a gap.

My experiment enviroment is:

• GeForce RTX 3090
• cuda 11.3.1
• python 3.7.11
• pytorch 1.10.0
• mmcv 1.4.0
• mmdet 2.14.0
• mmseg 0.14.1
• mmdet3d 0.15.0
• MinkowskiEngine 0.5.4

Merge with main mmdetection3d repo?

Given this work is modified based on the mmdetection3d repo, are there any plans to update this code to the original mmdetection3d repo?

I run the trained model on a single data and encountered buffer size issue

I want to run the trained model on my own data.
I followed the pcd_demo.py requirments.

(base) root@01d3ccee40b8:/mmdetection3d/demo# python pcd_demo.py /mmdetection3d/demo/apple1.pcd /mmdetection3d/configs/fcaf3d/fcaf3d_sunrgbd-3d-10class.py /mmdetection3d/work_dirs/fcaf3d_sunrgbd-3d-10class/latest.pth --score-thr 0.1 --show Use load_from_local loader Traceback (most recent call last): File "pcd_demo.py", line 41, in <module> main() File "pcd_demo.py", line 28, in main result, data = inference_detector(model, args.pcd) File "/mmdetection3d/mmdet3d/apis/inference.py", line 104, in inference_detector data = test_pipeline(data) File "/opt/conda/lib/python3.7/site-packages/mmdet/datasets/pipelines/compose.py", line 40, in __call__ data = t(data) File "/mmdetection3d/mmdet3d/datasets/pipelines/loading.py", line 415, in __call__ points = self._load_points(pts_filename) File "/mmdetection3d/mmdet3d/datasets/pipelines/loading.py", line 391, in _load_points points = np.frombuffer(pts_bytes, dtype=np.float32) ValueError: buffer size must be a multiple of element size
I searched in Google but they didn't deal with pointcloud and i didnt get a useful advice.

terminate called after throwing an instance of 'c10::CUDAError'

/home/miniconda3/envs/indoor/lib/python3.8/site-packages/torch/distributed/launch.py:163: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
  logger.warn(
The module torch.distributed.launch is deprecated and going to be removed in future.Migrate to torch.distributed.run
*****************************************
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
*****************************************
WARNING:torch.distributed.run:--use_env is deprecated and will be removed in future releases.
 Please read local_rank from `os.environ('LOCAL_RANK')` instead.
INFO:torch.distributed.launcher.api:Starting elastic_operator with launch configs:
  entrypoint       : tools/train.py
  min_nodes        : 1
  max_nodes        : 1
  nproc_per_node   : 2
  run_id           : none
  rdzv_backend     : static
  rdzv_endpoint    : 127.0.0.1:29500
  rdzv_configs     : {'rank': 0, 'timeout': 900}
  max_restarts     : 3
  monitor_interval : 5
  log_dir          : None
  metrics_cfg      : {}

INFO:torch.distributed.elastic.agent.server.local_elastic_agent:log directory set to: /tmp/torchelastic_poy3g6dh/none_du2bhj2l
INFO:torch.distributed.elastic.agent.server.api:[default] starting workers for entrypoint: python3
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous'ing worker group
/home/miniconda3/envs/indoor/lib/python3.8/site-packages/torch/distributed/elastic/utils/store.py:52: FutureWarning: This is an experimental API and will be changed in future.
  warnings.warn(
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous complete for workers. Result:
  restart_count=0
  master_addr=127.0.0.1
  master_port=29500
  group_rank=0
  group_world_size=1
  local_ranks=[0, 1]
  role_ranks=[0, 1]
  global_ranks=[0, 1]
  role_world_sizes=[2, 2]
  global_world_sizes=[2, 2]

INFO:torch.distributed.elastic.agent.server.api:[default] Starting worker group
INFO:torch.distributed.elastic.multiprocessing:Setting worker0 reply file to: /tmp/torchelastic_poy3g6dh/none_du2bhj2l/attempt_0/0/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker1 reply file to: /tmp/torchelastic_poy3g6dh/none_du2bhj2l/attempt_0/1/error.json
fatal: not a git repository (or any parent up to mount point /opt/data)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
2022-12-23 16:16:27,421 - mmdet - INFO - Environment info:
------------------------------------------------------------
sys.platform: linux
Python: 3.8.13 (default, Oct 21 2022, 23:50:54) [GCC 11.2.0]
CUDA available: True
GPU 0,1,2,3: GeForce RTX 3090
CUDA_HOME: /usr/local/cuda-11.2
NVCC: Build cuda_11.2.r11.2/compiler.29618528_0
GCC: gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
PyTorch: 1.9.0+cu111
PyTorch compiling details: PyTorch built with:
  - GCC 7.3
  - C++ Version: 201402
  - Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v2.1.2 (Git Hash 98be7e8afa711dc9b66c8ff3504129cb82013cdb)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 11.1
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86
  - CuDNN 8.0.5
  - Magma 2.5.2
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.1, CUDNN_VERSION=8.0.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.9.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON,

TorchVision: 0.10.0+cu111
OpenCV: 4.6.0
MMCV: 1.4.0
MMCV Compiler: GCC 7.3
MMCV CUDA Compiler: 11.1
MMDetection: 2.14.0
MMSegmentation: 0.14.1
MMDetection3D: 0.15.0+
------------------------------------------------------------

fatal: not a git repository (or any parent up to mount point /opt/data)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
2022-12-23 16:16:28,108 - mmdet - INFO - Distributed training: True
2022-12-23 16:16:28,735 - mmdet - INFO - Config:
voxel_size = 0.01
model = dict(
    type='SingleStageSparse3DDetector',
    voxel_size=0.01,
    backbone=dict(type='MEResNet3D', in_channels=3, depth=34),
    neck_with_head=dict(
        type='Fcaf3DNeckWithHead',
        in_channels=(64, 128, 256, 512),
        out_channels=128,
        pts_threshold=100000,
        n_classes=18,
        n_reg_outs=6,
        voxel_size=0.01,
        assigner=dict(type='Fcaf3DAssigner', limit=27, topk=18, n_scales=4),
        loss_bbox=dict(type='IoU3DLoss', loss_weight=1.0, with_yaw=False)),
    train_cfg=dict(),
    test_cfg=dict(nms_pre=1000, iou_thr=0.5, score_thr=0.01))
optimizer = dict(type='AdamW', lr=0.001, weight_decay=0.0001)
optimizer_config = dict(grad_clip=dict(max_norm=10, norm_type=2))
lr_config = dict(policy='step', warmup=None, step=[8, 11])
runner = dict(type='EpochBasedRunner', max_epochs=12)
custom_hooks = [dict(type='EmptyCacheHook', after_iter=True)]
checkpoint_config = dict(interval=1, max_keep_ckpts=1)
log_config = dict(interval=50, hooks=[dict(type='TextLoggerHook')])
dist_params = dict(backend='nccl')
log_level = 'INFO'
work_dir = './work_dirs/fcaf3d_scannet-3d-18class'
load_from = None
resume_from = None
workflow = [('train', 1)]
n_points = 100000
dataset_type = 'ScanNetDataset'
data_root = '../all_data/scannet_v2/'
class_names = ('cabinet', 'bed', 'chair', 'sofa', 'table', 'door', 'window',
               'bookshelf', 'picture', 'counter', 'desk', 'curtain',
               'refrigerator', 'showercurtrain', 'toilet', 'sink', 'bathtub',
               'garbagebin')
train_pipeline = [
    dict(
        type='LoadPointsFromFile',
        coord_type='DEPTH',
        shift_height=False,
        load_dim=6,
        use_dim=[0, 1, 2, 3, 4, 5]),
    dict(type='LoadAnnotations3D'),
    dict(type='GlobalAlignment', rotation_axis=2),
    dict(type='IndoorPointSample', num_points=100000),
    dict(
        type='RandomFlip3D',
        sync_2d=False,
        flip_ratio_bev_horizontal=0.5,
        flip_ratio_bev_vertical=0.5),
    dict(
        type='GlobalRotScaleTrans',
        rot_range=[-0.087266, 0.087266],
        scale_ratio_range=[0.9, 1.1],
        translation_std=[0.1, 0.1, 0.1],
        shift_height=False),
    dict(
        type='DefaultFormatBundle3D',
        class_names=('cabinet', 'bed', 'chair', 'sofa', 'table', 'door',
                     'window', 'bookshelf', 'picture', 'counter', 'desk',
                     'curtain', 'refrigerator', 'showercurtrain', 'toilet',
                     'sink', 'bathtub', 'garbagebin')),
    dict(type='Collect3D', keys=['points', 'gt_bboxes_3d', 'gt_labels_3d'])
]
test_pipeline = [
    dict(
        type='LoadPointsFromFile',
        coord_type='DEPTH',
        shift_height=False,
        load_dim=6,
        use_dim=[0, 1, 2, 3, 4, 5]),
    dict(type='GlobalAlignment', rotation_axis=2),
    dict(
        type='MultiScaleFlipAug3D',
        img_scale=(1333, 800),
        pts_scale_ratio=1,
        flip=False,
        transforms=[
            dict(
                type='GlobalRotScaleTrans',
                rot_range=[0, 0],
                scale_ratio_range=[1.0, 1.0],
                translation_std=[0, 0, 0]),
            dict(
                type='RandomFlip3D',
                sync_2d=False,
                flip_ratio_bev_horizontal=0.5,
                flip_ratio_bev_vertical=0.5),
            dict(type='IndoorPointSample', num_points=100000),
            dict(
                type='DefaultFormatBundle3D',
                class_names=('cabinet', 'bed', 'chair', 'sofa', 'table',
                             'door', 'window', 'bookshelf', 'picture',
                             'counter', 'desk', 'curtain', 'refrigerator',
                             'showercurtrain', 'toilet', 'sink', 'bathtub',
                             'garbagebin'),
                with_label=False),
            dict(type='Collect3D', keys=['points'])
        ])
]
data = dict(
    samples_per_gpu=8,
    workers_per_gpu=4,
    train=dict(
        type='RepeatDataset',
        times=10,
        dataset=dict(
            type='ScanNetDataset',
            data_root='../all_data/scannet_v2/',
            ann_file='../all_data/scannet_v2/scannet_infos_train.pkl',
            pipeline=[
                dict(
                    type='LoadPointsFromFile',
                    coord_type='DEPTH',
                    shift_height=False,
                    load_dim=6,
                    use_dim=[0, 1, 2, 3, 4, 5]),
                dict(type='LoadAnnotations3D'),
                dict(type='GlobalAlignment', rotation_axis=2),
                dict(type='IndoorPointSample', num_points=100000),
                dict(
                    type='RandomFlip3D',
                    sync_2d=False,
                    flip_ratio_bev_horizontal=0.5,
                    flip_ratio_bev_vertical=0.5),
                dict(
                    type='GlobalRotScaleTrans',
                    rot_range=[-0.087266, 0.087266],
                    scale_ratio_range=[0.9, 1.1],
                    translation_std=[0.1, 0.1, 0.1],
                    shift_height=False),
                dict(
                    type='DefaultFormatBundle3D',
                    class_names=('cabinet', 'bed', 'chair', 'sofa', 'table',
                                 'door', 'window', 'bookshelf', 'picture',
                                 'counter', 'desk', 'curtain', 'refrigerator',
                                 'showercurtrain', 'toilet', 'sink', 'bathtub',
                                 'garbagebin')),
                dict(
                    type='Collect3D',
                    keys=['points', 'gt_bboxes_3d', 'gt_labels_3d'])
            ],
            filter_empty_gt=True,
            classes=('cabinet', 'bed', 'chair', 'sofa', 'table', 'door',
                     'window', 'bookshelf', 'picture', 'counter', 'desk',
                     'curtain', 'refrigerator', 'showercurtrain', 'toilet',
                     'sink', 'bathtub', 'garbagebin'),
            box_type_3d='Depth')),
    val=dict(
        type='ScanNetDataset',
        data_root='../all_data/scannet_v2/',
        ann_file='../all_data/scannet_v2/scannet_infos_val.pkl',
        pipeline=[
            dict(
                type='LoadPointsFromFile',
                coord_type='DEPTH',
                shift_height=False,
                load_dim=6,
                use_dim=[0, 1, 2, 3, 4, 5]),
            dict(type='GlobalAlignment', rotation_axis=2),
            dict(
                type='MultiScaleFlipAug3D',
                img_scale=(1333, 800),
                pts_scale_ratio=1,
                flip=False,
                transforms=[
                    dict(
                        type='GlobalRotScaleTrans',
                        rot_range=[0, 0],
                        scale_ratio_range=[1.0, 1.0],
                        translation_std=[0, 0, 0]),
                    dict(
                        type='RandomFlip3D',
                        sync_2d=False,
                        flip_ratio_bev_horizontal=0.5,
                        flip_ratio_bev_vertical=0.5),
                    dict(type='IndoorPointSample', num_points=100000),
                    dict(
                        type='DefaultFormatBundle3D',
                        class_names=('cabinet', 'bed', 'chair', 'sofa',
                                     'table', 'door', 'window', 'bookshelf',
                                     'picture', 'counter', 'desk', 'curtain',
                                     'refrigerator', 'showercurtrain',
                                     'toilet', 'sink', 'bathtub',
                                     'garbagebin'),
                        with_label=False),
                    dict(type='Collect3D', keys=['points'])
                ])
        ],
        classes=('cabinet', 'bed', 'chair', 'sofa', 'table', 'door', 'window',
                 'bookshelf', 'picture', 'counter', 'desk', 'curtain',
                 'refrigerator', 'showercurtrain', 'toilet', 'sink', 'bathtub',
                 'garbagebin'),
        test_mode=True,
        box_type_3d='Depth'),
    test=dict(
        type='ScanNetDataset',
        data_root='../all_data/scannet_v2/',
        ann_file='../all_data/scannet_v2/scannet_infos_val.pkl',
        pipeline=[
            dict(
                type='LoadPointsFromFile',
                coord_type='DEPTH',
                shift_height=False,
                load_dim=6,
                use_dim=[0, 1, 2, 3, 4, 5]),
            dict(type='GlobalAlignment', rotation_axis=2),
            dict(
                type='MultiScaleFlipAug3D',
                img_scale=(1333, 800),
                pts_scale_ratio=1,
                flip=False,
                transforms=[
                    dict(
                        type='GlobalRotScaleTrans',
                        rot_range=[0, 0],
                        scale_ratio_range=[1.0, 1.0],
                        translation_std=[0, 0, 0]),
                    dict(
                        type='RandomFlip3D',
                        sync_2d=False,
                        flip_ratio_bev_horizontal=0.5,
                        flip_ratio_bev_vertical=0.5),
                    dict(type='IndoorPointSample', num_points=100000),
                    dict(
                        type='DefaultFormatBundle3D',
                        class_names=('cabinet', 'bed', 'chair', 'sofa',
                                     'table', 'door', 'window', 'bookshelf',
                                     'picture', 'counter', 'desk', 'curtain',
                                     'refrigerator', 'showercurtrain',
                                     'toilet', 'sink', 'bathtub',
                                     'garbagebin'),
                        with_label=False),
                    dict(type='Collect3D', keys=['points'])
                ])
        ],
        classes=('cabinet', 'bed', 'chair', 'sofa', 'table', 'door', 'window',
                 'bookshelf', 'picture', 'counter', 'desk', 'curtain',
                 'refrigerator', 'showercurtrain', 'toilet', 'sink', 'bathtub',
                 'garbagebin'),
        test_mode=True,
        box_type_3d='Depth'))
gpu_ids = range(0, 2)

2022-12-23 16:16:28,735 - mmdet - INFO - Set random seed to 0, deterministic: False
2022-12-23 16:16:29,843 - mmdet - INFO - Model:
SingleStageSparse3DDetector(
  (backbone): MEResNet3D(
    (conv1): Sequential(
      (0): MinkowskiConvolution(in=3, out=64, kernel_size=[3, 3, 3], stride=[2, 2, 2], dilation=[1, 1, 1])
      (1): MinkowskiInstanceNorm(nchannels=64)
      (2): MinkowskiReLU()
      (3): MinkowskiMaxPooling(kernel_size=[2, 2, 2], stride=[2, 2, 2], dilation=[1, 1, 1])
    )
    (layer1): Sequential(
      (0): BasicBlock(
        (conv1): MinkowskiConvolution(in=64, out=64, kernel_size=[3, 3, 3], stride=[2, 2, 2], dilation=[1, 1, 1])
        (norm1): MinkowskiBatchNorm(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): MinkowskiConvolution(in=64, out=64, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
        (norm2): MinkowskiBatchNorm(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): MinkowskiReLU()
        (downsample): Sequential(
          (0): MinkowskiConvolution(in=64, out=64, kernel_size=[1, 1, 1], stride=[2, 2, 2], dilation=[1, 1, 1])
          (1): MinkowskiBatchNorm(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): MinkowskiConvolution(in=64, out=64, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
        (norm1): MinkowskiBatchNorm(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): MinkowskiConvolution(in=64, out=64, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
        (norm2): MinkowskiBatchNorm(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): MinkowskiReLU()
      )
      (2): BasicBlock(
        (conv1): MinkowskiConvolution(in=64, out=64, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
        (norm1): MinkowskiBatchNorm(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): MinkowskiConvolution(in=64, out=64, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
        (norm2): MinkowskiBatchNorm(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): MinkowskiReLU()
      )
    )
    (layer2): Sequential(
      (0): BasicBlock(
        (conv1): MinkowskiConvolution(in=64, out=128, kernel_size=[3, 3, 3], stride=[2, 2, 2], dilation=[1, 1, 1])
        (norm1): MinkowskiBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): MinkowskiConvolution(in=128, out=128, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
        (norm2): MinkowskiBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): MinkowskiReLU()
        (downsample): Sequential(
          (0): MinkowskiConvolution(in=64, out=128, kernel_size=[1, 1, 1], stride=[2, 2, 2], dilation=[1, 1, 1])
          (1): MinkowskiBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): MinkowskiConvolution(in=128, out=128, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
        (norm1): MinkowskiBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): MinkowskiConvolution(in=128, out=128, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
        (norm2): MinkowskiBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): MinkowskiReLU()
      )
      (2): BasicBlock(
        (conv1): MinkowskiConvolution(in=128, out=128, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
        (norm1): MinkowskiBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): MinkowskiConvolution(in=128, out=128, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
        (norm2): MinkowskiBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): MinkowskiReLU()
      )
      (3): BasicBlock(
        (conv1): MinkowskiConvolution(in=128, out=128, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
        (norm1): MinkowskiBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): MinkowskiConvolution(in=128, out=128, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
        (norm2): MinkowskiBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): MinkowskiReLU()
      )
    )
    (layer3): Sequential(
      (0): BasicBlock(
        (conv1): MinkowskiConvolution(in=128, out=256, kernel_size=[3, 3, 3], stride=[2, 2, 2], dilation=[1, 1, 1])
        (norm1): MinkowskiBatchNorm(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): MinkowskiConvolution(in=256, out=256, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
        (norm2): MinkowskiBatchNorm(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): MinkowskiReLU()
        (downsample): Sequential(
          (0): MinkowskiConvolution(in=128, out=256, kernel_size=[1, 1, 1], stride=[2, 2, 2], dilation=[1, 1, 1])
          (1): MinkowskiBatchNorm(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): MinkowskiConvolution(in=256, out=256, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
        (norm1): MinkowskiBatchNorm(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): MinkowskiConvolution(in=256, out=256, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
        (norm2): MinkowskiBatchNorm(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): MinkowskiReLU()
      )
      (2): BasicBlock(
        (conv1): MinkowskiConvolution(in=256, out=256, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
        (norm1): MinkowskiBatchNorm(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): MinkowskiConvolution(in=256, out=256, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
        (norm2): MinkowskiBatchNorm(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): MinkowskiReLU()
      )
      (3): BasicBlock(
        (conv1): MinkowskiConvolution(in=256, out=256, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
        (norm1): MinkowskiBatchNorm(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): MinkowskiConvolution(in=256, out=256, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
        (norm2): MinkowskiBatchNorm(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): MinkowskiReLU()
      )
      (4): BasicBlock(
        (conv1): MinkowskiConvolution(in=256, out=256, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
        (norm1): MinkowskiBatchNorm(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): MinkowskiConvolution(in=256, out=256, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
        (norm2): MinkowskiBatchNorm(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): MinkowskiReLU()
      )
      (5): BasicBlock(
        (conv1): MinkowskiConvolution(in=256, out=256, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
        (norm1): MinkowskiBatchNorm(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): MinkowskiConvolution(in=256, out=256, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
        (norm2): MinkowskiBatchNorm(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): MinkowskiReLU()
      )
    )
    (layer4): Sequential(
      (0): BasicBlock(
        (conv1): MinkowskiConvolution(in=256, out=512, kernel_size=[3, 3, 3], stride=[2, 2, 2], dilation=[1, 1, 1])
        (norm1): MinkowskiBatchNorm(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): MinkowskiConvolution(in=512, out=512, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
        (norm2): MinkowskiBatchNorm(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): MinkowskiReLU()
        (downsample): Sequential(
          (0): MinkowskiConvolution(in=256, out=512, kernel_size=[1, 1, 1], stride=[2, 2, 2], dilation=[1, 1, 1])
          (1): MinkowskiBatchNorm(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        )
      )
      (1): BasicBlock(
        (conv1): MinkowskiConvolution(in=512, out=512, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
        (norm1): MinkowskiBatchNorm(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): MinkowskiConvolution(in=512, out=512, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
        (norm2): MinkowskiBatchNorm(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): MinkowskiReLU()
      )
      (2): BasicBlock(
        (conv1): MinkowskiConvolution(in=512, out=512, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
        (norm1): MinkowskiBatchNorm(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (conv2): MinkowskiConvolution(in=512, out=512, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
        (norm2): MinkowskiBatchNorm(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): MinkowskiReLU()
      )
    )
  )
  (neck_with_head): Fcaf3DNeckWithHead(
    (loss_centerness): CrossEntropyLoss()
    (loss_bbox): IoU3DLoss()
    (loss_cls): FocalLoss()
    (pruning): MinkowskiPruning()
    (out_block_0): Sequential(
      (0): MinkowskiConvolution(in=64, out=128, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
      (1): MinkowskiBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): MinkowskiELU()
    )
    (up_block_1): Sequential(
      (0): MinkowskiGenerativeConvolutionTranspose(in=128, out=64, kernel_size=[2, 2, 2], stride=[2, 2, 2], dilation=[1, 1, 1])
      (1): MinkowskiBatchNorm(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): MinkowskiELU()
      (3): MinkowskiConvolution(in=64, out=64, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
      (4): MinkowskiBatchNorm(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (5): MinkowskiELU()
    )
    (out_block_1): Sequential(
      (0): MinkowskiConvolution(in=128, out=128, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
      (1): MinkowskiBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): MinkowskiELU()
    )
    (up_block_2): Sequential(
      (0): MinkowskiGenerativeConvolutionTranspose(in=256, out=128, kernel_size=[2, 2, 2], stride=[2, 2, 2], dilation=[1, 1, 1])
      (1): MinkowskiBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): MinkowskiELU()
      (3): MinkowskiConvolution(in=128, out=128, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
      (4): MinkowskiBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (5): MinkowskiELU()
    )
    (out_block_2): Sequential(
      (0): MinkowskiConvolution(in=256, out=128, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
      (1): MinkowskiBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): MinkowskiELU()
    )
    (up_block_3): Sequential(
      (0): MinkowskiGenerativeConvolutionTranspose(in=512, out=256, kernel_size=[2, 2, 2], stride=[2, 2, 2], dilation=[1, 1, 1])
      (1): MinkowskiBatchNorm(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): MinkowskiELU()
      (3): MinkowskiConvolution(in=256, out=256, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
      (4): MinkowskiBatchNorm(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (5): MinkowskiELU()
    )
    (out_block_3): Sequential(
      (0): MinkowskiConvolution(in=512, out=128, kernel_size=[3, 3, 3], stride=[1, 1, 1], dilation=[1, 1, 1])
      (1): MinkowskiBatchNorm(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      (2): MinkowskiELU()
    )
    (centerness_conv): MinkowskiConvolution(in=128, out=1, kernel_size=[1, 1, 1], stride=[1, 1, 1], dilation=[1, 1, 1])
    (reg_conv): MinkowskiConvolution(in=128, out=6, kernel_size=[1, 1, 1], stride=[1, 1, 1], dilation=[1, 1, 1])
    (cls_conv): MinkowskiConvolution(in=128, out=18, kernel_size=[1, 1, 1], stride=[1, 1, 1], dilation=[1, 1, 1])
    (scales): ModuleList(
      (0): Scale()
      (1): Scale()
      (2): Scale()
      (3): Scale()
    )
  )
)
2022-12-23 16:16:42,207 - mmdet - INFO - Start running, host: root@zy20221217, work_dir: /opt/data/private/indoor/work_dirs/fcaf3d_scannet-3d-18class
2022-12-23 16:16:42,208 - mmdet - INFO - Hooks will be executed in the following order:
before_run:
(VERY_HIGH   ) StepLrUpdaterHook
(NORMAL      ) CheckpointHook
(NORMAL      ) DistEvalHook
(VERY_LOW    ) TextLoggerHook
 --------------------
before_train_epoch:
(VERY_HIGH   ) StepLrUpdaterHook
(NORMAL      ) DistSamplerSeedHook
(NORMAL      ) DistEvalHook
(NORMAL      ) EmptyCacheHook
(LOW         ) IterTimerHook
(VERY_LOW    ) TextLoggerHook
 --------------------
before_train_iter:
(VERY_HIGH   ) StepLrUpdaterHook
(NORMAL      ) DistEvalHook
(LOW         ) IterTimerHook
 --------------------
after_train_iter:
(ABOVE_NORMAL) OptimizerHook
(NORMAL      ) CheckpointHook
(NORMAL      ) DistEvalHook
(NORMAL      ) EmptyCacheHook
(LOW         ) IterTimerHook
(VERY_LOW    ) TextLoggerHook
 --------------------
after_train_epoch:
(NORMAL      ) CheckpointHook
(NORMAL      ) DistEvalHook
(NORMAL      ) EmptyCacheHook
(VERY_LOW    ) TextLoggerHook
 --------------------
before_val_epoch:
(NORMAL      ) DistSamplerSeedHook
(NORMAL      ) EmptyCacheHook
(LOW         ) IterTimerHook
(VERY_LOW    ) TextLoggerHook
 --------------------
before_val_iter:
(LOW         ) IterTimerHook
 --------------------
after_val_iter:
(NORMAL      ) EmptyCacheHook
(LOW         ) IterTimerHook
 --------------------
after_val_epoch:
(NORMAL      ) EmptyCacheHook
(VERY_LOW    ) TextLoggerHook
 --------------------
after_run:
(VERY_LOW    ) TextLoggerHook
 --------------------
2022-12-23 16:16:42,209 - mmdet - INFO - workflow: [('train', 1)], max: 12 epochs
2022-12-23 16:16:42,209 - mmdet - INFO - Checkpoints will be saved to /opt/data/private/indoor/work_dirs/fcaf3d_scannet-3d-18class by HardDiskBackend.
Traceback (most recent call last):
  File "tools/train.py", line 223, in <module>
    main()
  File "tools/train.py", line 212, in main
    train_model(
  File "/opt/data/private/indoor/mmdet3d/apis/train.py", line 27, in train_model
    train_detector(
  File "/home/miniconda3/envs/indoor/lib/python3.8/site-packages/mmdet/apis/train.py", line 170, in train_detector
    runner.run(data_loaders, cfg.workflow)
  File "/home/miniconda3/envs/indoor/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 127, in run
    epoch_runner(data_loaders[i], **kwargs)
  File "/home/miniconda3/envs/indoor/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 51, in train
    self.call_hook('after_train_iter')
  File "/home/miniconda3/envs/indoor/lib/python3.8/site-packages/mmcv/runner/base_runner.py", line 307, in call_hook
    getattr(hook, fn_name)(self)
  File "/home/miniconda3/envs/indoor/lib/python3.8/site-packages/mmcv/runner/hooks/optimizer.py", line 35, in after_train_iter
    runner.outputs['loss'].backward()
  File "/home/miniconda3/envs/indoor/lib/python3.8/site-packages/torch/_tensor.py", line 255, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/home/miniconda3/envs/indoor/lib/python3.8/site-packages/torch/autograd/__init__.py", line 147, in backward
    Variable._execution_engine.run_backward(
RuntimeError: merge_sort: failed to synchronize: cudaErrorIllegalAddress: an illegal memory access was encountered
Traceback (most recent call last):
  File "tools/train.py", line 223, in <module>
    main()
  File "tools/train.py", line 212, in main
    train_model(
  File "/opt/data/private/indoor/mmdet3d/apis/train.py", line 27, in train_model
    train_detector(
  File "/home/miniconda3/envs/indoor/lib/python3.8/site-packages/mmdet/apis/train.py", line 170, in train_detector
    runner.run(data_loaders, cfg.workflow)
  File "/home/miniconda3/envs/indoor/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 127, in run
    epoch_runner(data_loaders[i], **kwargs)
  File "/home/miniconda3/envs/indoor/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 51, in train
    self.call_hook('after_train_iter')
  File "/home/miniconda3/envs/indoor/lib/python3.8/site-packages/mmcv/runner/base_runner.py", line 307, in call_hook
    getattr(hook, fn_name)(self)
  File "/home/miniconda3/envs/indoor/lib/python3.8/site-packages/mmcv/runner/hooks/optimizer.py", line 35, in after_train_iter
    runner.outputs['loss'].backward()
  File "/home/miniconda3/envs/indoor/lib/python3.8/site-packages/torch/_tensor.py", line 255, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/home/miniconda3/envs/indoor/lib/python3.8/site-packages/torch/autograd/__init__.py", line 147, in backward
    Variable._execution_engine.run_backward(
RuntimeError: merge_sort: failed to synchronize: cudaErrorIllegalAddress: an illegal memory access was encountered
terminate called after throwing an instance of 'c10::CUDAError'
  what():  CUDA error: an illegal memory access was encountered
Exception raised from create_event_internal at /pytorch/c10/cuda/CUDACachingAllocator.cpp:1055 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x42 (0x7f1a89cdba22 in /home/miniconda3/envs/indoor/lib/python3.8/site-packages/torch/lib/libc10.so)
frame #1: <unknown function> + 0x10aa3 (0x7f1a89f3caa3 in /home/miniconda3/envs/indoor/lib/python3.8/site-packages/torch/lib/libc10_cuda.so)
frame #2: c10::cuda::CUDACachingAllocator::raw_delete(void*) + 0x1a7 (0x7f1a89f3e147 in /home/miniconda3/envs/indoor/lib/python3.8/site-packages/torch/lib/libc10_cuda.so)
frame #3: c10::TensorImpl::release_resources() + 0x54 (0x7f1a89cc55a4 in /home/miniconda3/envs/indoor/lib/python3.8/site-packages/torch/lib/libc10.so)
frame #4: <unknown function> + 0xe568e9 (0x7f1a8afb08e9 in /home/miniconda3/envs/indoor/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so)
frame #5: <unknown function> + 0x2f45656 (0x7f1a8d09f656 in /home/miniconda3/envs/indoor/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so)
frame #6: <unknown function> + 0x355bcc2 (0x7f1a8d6b5cc2 in /home/miniconda3/envs/indoor/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so)
frame #7: torch::autograd::deleteNode(torch::autograd::Node*) + 0x7f (0x7f1a8d6b5d6f in /home/miniconda3/envs/indoor/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so)
frame #8: <unknown function> + 0x3545ad8 (0x7f1a8d69fad8 in /home/miniconda3/envs/indoor/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so)
frame #9: c10::TensorImpl::release_resources() + 0x20 (0x7f1a89cc5570 in /home/miniconda3/envs/indoor/lib/python3.8/site-packages/torch/lib/libc10.so)
frame #10: <unknown function> + 0xa2822a (0x7f1b2eae322a in /home/miniconda3/envs/indoor/lib/python3.8/site-packages/torch/lib/libtorch_python.so)
frame #11: <unknown function> + 0xa282c1 (0x7f1b2eae32c1 in /home/miniconda3/envs/indoor/lib/python3.8/site-packages/torch/lib/libtorch_python.so)
frame #12: /home/miniconda3/envs/indoor/bin/python3() [0x4f0e66]
frame #13: /home/miniconda3/envs/indoor/bin/python3() [0x4c179f]
frame #14: /home/miniconda3/envs/indoor/bin/python3() [0x4c15f3]
frame #15: /home/miniconda3/envs/indoor/bin/python3() [0x4c16da]
frame #16: /home/miniconda3/envs/indoor/bin/python3() [0x4f0d27]
frame #17: /home/miniconda3/envs/indoor/bin/python3() [0x4d07e8]
frame #18: /home/miniconda3/envs/indoor/bin/python3() [0x4e3c38]
frame #19: /home/miniconda3/envs/indoor/bin/python3() [0x4e3c4b]
frame #20: /home/miniconda3/envs/indoor/bin/python3() [0x4e3c4b]
frame #21: /home/miniconda3/envs/indoor/bin/python3() [0x4e3c4b]
frame #22: /home/miniconda3/envs/indoor/bin/python3() [0x4b4c97]
frame #23: PyDict_SetItemString + 0x99 (0x4bc2c9 in /home/miniconda3/envs/indoor/bin/python3)
frame #24: PyImport_Cleanup + 0x93 (0x58c5e3 in /home/miniconda3/envs/indoor/bin/python3)
frame #25: Py_FinalizeEx + 0x71 (0x5881e1 in /home/miniconda3/envs/indoor/bin/python3)
frame #26: Py_RunMain + 0x1b6 (0x57f406 in /home/miniconda3/envs/indoor/bin/python3)
frame #27: Py_BytesMain + 0x39 (0x55cbe9 in /home/miniconda3/envs/indoor/bin/python3)
frame #28: __libc_start_main + 0xe7 (0x7f1b362ccc87 in /lib/x86_64-linux-gnu/libc.so.6)
frame #29: /home/miniconda3/envs/indoor/bin/python3() [0x55ca9e]

terminate called after throwing an instance of 'c10::CUDAError'
  what():  CUDA error: an illegal memory access was encountered
Exception raised from create_event_internal at /pytorch/c10/cuda/CUDACachingAllocator.cpp:1055 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x42 (0x7f38aa109a22 in /home/miniconda3/envs/indoor/lib/python3.8/site-packages/torch/lib/libc10.so)
frame #1: <unknown function> + 0x10aa3 (0x7f38aa36aaa3 in /home/miniconda3/envs/indoor/lib/python3.8/site-packages/torch/lib/libc10_cuda.so)
frame #2: c10::cuda::CUDACachingAllocator::raw_delete(void*) + 0x1a7 (0x7f38aa36c147 in /home/miniconda3/envs/indoor/lib/python3.8/site-packages/torch/lib/libc10_cuda.so)
frame #3: c10::TensorImpl::release_resources() + 0x54 (0x7f38aa0f35a4 in /home/miniconda3/envs/indoor/lib/python3.8/site-packages/torch/lib/libc10.so)
frame #4: <unknown function> + 0xe568e9 (0x7f38ab3de8e9 in /home/miniconda3/envs/indoor/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so)
frame #5: <unknown function> + 0x2f45656 (0x7f38ad4cd656 in /home/miniconda3/envs/indoor/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so)
frame #6: <unknown function> + 0x355bcc2 (0x7f38adae3cc2 in /home/miniconda3/envs/indoor/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so)
frame #7: torch::autograd::deleteNode(torch::autograd::Node*) + 0x7f (0x7f38adae3d6f in /home/miniconda3/envs/indoor/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so)
frame #8: <unknown function> + 0x3545ad8 (0x7f38adacdad8 in /home/miniconda3/envs/indoor/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so)
frame #9: c10::TensorImpl::release_resources() + 0x20 (0x7f38aa0f3570 in /home/miniconda3/envs/indoor/lib/python3.8/site-packages/torch/lib/libc10.so)
frame #10: <unknown function> + 0xa2822a (0x7f394ef1122a in /home/miniconda3/envs/indoor/lib/python3.8/site-packages/torch/lib/libtorch_python.so)
frame #11: <unknown function> + 0xa282c1 (0x7f394ef112c1 in /home/miniconda3/envs/indoor/lib/python3.8/site-packages/torch/lib/libtorch_python.so)
frame #12: /home/miniconda3/envs/indoor/bin/python3() [0x4f0e66]
frame #13: /home/miniconda3/envs/indoor/bin/python3() [0x4c179f]
frame #14: /home/miniconda3/envs/indoor/bin/python3() [0x4c15f3]
frame #15: /home/miniconda3/envs/indoor/bin/python3() [0x4c16da]
frame #16: /home/miniconda3/envs/indoor/bin/python3() [0x4f0d27]
frame #17: /home/miniconda3/envs/indoor/bin/python3() [0x4d07e8]
frame #18: /home/miniconda3/envs/indoor/bin/python3() [0x4e3c38]
frame #19: /home/miniconda3/envs/indoor/bin/python3() [0x4e3c4b]
frame #20: /home/miniconda3/envs/indoor/bin/python3() [0x4e3c4b]
frame #21: /home/miniconda3/envs/indoor/bin/python3() [0x4e3c4b]
frame #22: /home/miniconda3/envs/indoor/bin/python3() [0x4b4c97]
frame #23: PyDict_SetItemString + 0x99 (0x4bc2c9 in /home/miniconda3/envs/indoor/bin/python3)
frame #24: PyImport_Cleanup + 0x93 (0x58c5e3 in /home/miniconda3/envs/indoor/bin/python3)
frame #25: Py_FinalizeEx + 0x71 (0x5881e1 in /home/miniconda3/envs/indoor/bin/python3)
frame #26: Py_RunMain + 0x1b6 (0x57f406 in /home/miniconda3/envs/indoor/bin/python3)
frame #27: Py_BytesMain + 0x39 (0x55cbe9 in /home/miniconda3/envs/indoor/bin/python3)
frame #28: __libc_start_main + 0xe7 (0x7f39566fac87 in /lib/x86_64-linux-gnu/libc.so.6)
frame #29: /home/miniconda3/envs/indoor/bin/python3() [0x55ca9e]

hello, can you help me slove this problem.

Welcome update to OpenMMLab 2.0

Welcome update to OpenMMLab 2.0

I am Vansin, the technical operator of OpenMMLab. In September of last year, we announced the release of OpenMMLab 2.0 at the World Artificial Intelligence Conference in Shanghai. We invite you to upgrade your algorithm library to OpenMMLab 2.0 using MMEngine, which can be used for both research and commercial purposes. If you have any questions, please feel free to join us on the OpenMMLab Discord at https://discord.gg/amFNsyUBvm or add me on WeChat (van-sin) and I will invite you to the OpenMMLab WeChat group.

Here are the OpenMMLab 2.0 repos branches:

	OpenMMLab 1.0 branch	OpenMMLab 2.0 branch
MMEngine		0.x
MMCV	1.x	2.x
MMDetection	0.x 、1.x、2.x	3.x
MMAction2	0.x	1.x
MMClassification	0.x	1.x
MMSegmentation	0.x	1.x
MMDetection3D	0.x	1.x
MMEditing	0.x	1.x
MMPose	0.x	1.x
MMDeploy	0.x	1.x
MMTracking	0.x	1.x
MMOCR	0.x	1.x
MMRazor	0.x	1.x
MMSelfSup	0.x	1.x
MMRotate	1.x	1.x
MMYOLO		0.x

Attention: please create a new virtual environment for OpenMMLab 2.0.

mAP depending on score_thr

Hi! me again
I've designed my own detector for persimmon Thanks to your advice.
it's mAP was 0.7 wow! your model is amazing.
btw, when I set score_thr =0.1, it decreased upto 0.58
if I set score_thr=0.3(because 0.3 is higher than 0.25 IOU), it would be right.
but I set only 0.1
why this phenomenon occured? and which mAP is real?

FCAF3D evaluation result mAP question

Hi ! Thanks for you wonderful work! I have learned a lot from your job，but i have a simple question
For evaluation，you say you report the best metrics accross 5*5 trails，do you mean
1、the best mAP0.25 and mAP0.5 in your paper is come from the same pth and the same test result or
2、select the best mAP0.25 and mAP0.5 respectively from the 25 results，they are not come from one result

how should I understand results?

I made 3 .obj files through your model checkfile. and I visualized that in mashlab.
(left is a gt and right is a predition) but while gt has only 2boxes , predtion had so many bounding boxes (looks like many anchor boxes)
how should I understand the results? please explain that to me

run pcd_demo.py

I want to use the trained model to test point cloud,but I meet following question:
Traceback (most recent call last):
File "/media/ketizu/086C03DE086C03DE/project1/fcaf3d-master/demo/pcd_demo.py", line 102, in
main()
File "/media/ketizu/086C03DE086C03DE/project1/fcaf3d-master/demo/pcd_demo.py", line 90, in main
show_result_meshlab(
File "/media/ketizu/086C03DE086C03DE/project1/fcaf3d-master/mmdet3d/apis/inference.py", line 481, in show_result_meshlab
file_name = show_det_result_meshlab(data, result, out_dir, score_thr,
File "/media/ketizu/086C03DE086C03DE/project1/fcaf3d-master/mmdet3d/apis/inference.py", line 324, in show_det_result_meshlab
show_result(
TypeError: show_result() missing 2 required positional arguments: 'out_dir' and 'filename'

Process finished with exit code 1

another version fcaf3d

hello, Danila, very glad to contact you, writing to ask you a question, thanks for you help.
you have done a grate job about "FCAF3D: Fully Convolutional Anchor-Free 3D Object Detection" and given a released code,
I just want to know do you have another version source code which is not based on mmdetection3d, mmdetection3d is hard to modification, so, could you release a code which is easy to apply, look for you reply, thanks very much.

how can I increase small objects'mAP?

Hi
I selected 10 small objects in SUNRGBD and I made those trained in this model.
but, result showed only 0.1mAP( learning_rate = 0.001, epoch = 12, Objects were cup,bottle,bowl,book etc)
I guess this model was designed for big objects not small things, right?
then, how can I increase small objects' mAP? are there any parameters to do that?

Concatenate different feature levels to one level before passing data to the heads

Thanks for the wonderful work! I learned a lot from reading the paper and going over the code.

I was wondering whether there would be a chance to use an FPN to produce one feature map of size WxHxC instead of going over the different levels for each box prediction? I guess that would be done here:

fcaf3d/mmdet3d/models/dense_heads/fcaf3d_neck_with_head.py

Line 104 in ad5985d

out = self.__getattr__(f'out_block_{i}')(x)

or ? To the Minkowski convolution permit the transformation from a spare set to a feature image of fixed height and width? Do you have any idea what the impact on the accuracy would be?

How does demo predict gravity direction without calibration data?

Hi
I know that calibration information is very important in training.
but pcd_demo.py only required pcd file(binary format). I searched that format and it just contained depth data without calibration.
I wonder how this model predicted the gravity direction without calibration data.

Training with Custom Dataset

Prerequisites

• I have searched Issues and Discussions but cannot get the expected help. I have also searched through issues on this github page.

• I have read the (https://mmdetection3d.readthedocs.io/en/latest/2_new_data_model.html) but cannot get the expected help.

Task
I am trying to train fcaf3d with a custom dataset that emulates the sunrgbd dataset.

Environment
Docker Image Build from DockerFile.
GPU: 2080ti

Steps and Issues
To emulate the SUNRGB-D Dataset:

• I generated the binary file as a numpy ndarray with 6 dimensions. (x, y, z, r, g, b) e.g in human readable format (point_cloudcustom.txt). Which is supposed to emulate the binary file from the create_data.py in human readable format ( point_cloud.txt
  *Github doesn't like me attaching a binary file. Also I only have 733 pointclouds, significantly less than the sunrgbd dataset.

• I also generated the corresponding pkl file, based of the details provided from; https://mmdetection3d.readthedocs.io/en/latest/datasets/sunrgbd_det.html. The only difference is you have to add two classes at minimum otherwise it gives me a Assertion error like so:

File "/mmdetection3d/mmdet3d/core/bbox/structures/base_box3d.py", line 47, in __init__
assert tensor.dim() == 2 and tensor.size(-1) == box_dim, tensor.size()
AssertionError: torch.Size([7])

Example pkl file in human readable format: pklhumanreadable.txt

Reproduces the problem - Command or script
python tools/train.py configs/fcaf3d/fcaf3d_sunrgbd-3d-10class.py

Reproduces the problem - issue on 2080 ti
issue as log:
2080ti_train_issue.txt

  File "/mmdetection3d/mmdet3d/models/backbones/me_resnet.py", line 89, in forward
    x = self.layer2(x)
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/container.py", line 119, in forward
    input = module(input)
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/MinkowskiEngine/modules/resnet_block.py", line 59, in forward
    out = self.conv2(out)
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/MinkowskiEngine/MinkowskiConvolution.py", line 321, in forward
    input._manager,
  File "/opt/conda/lib/python3.7/site-packages/MinkowskiEngine/MinkowskiConvolution.py", line 84, in forward
    coordinate_manager._manager,
RuntimeError: CUDA out of memory. Tried to allocate 392.00 MiB (GPU 0; 10.75 GiB total capacity; 8.93 GiB already allocated; 201.06 MiB free; 9.09 GiB reserved in total by PyTorch)

Reproduces the problem - issue on A100
I also launched an instance with a A100 to verify if it genuinely needed more memory.

Thoughts
I am quite confused as to why my dataset is more memory hungry, given that it is only 733 points. Also it trained fine with the SUNRGB-D Dataset on a 2080ti.

Alternative Voxel Based Backbones?

Thank you for your wonderful work.
I was wondering, what other voxel backbones were you considering, and why did you select a modified ResNet with spare 3D Convolutions as your backbone? I wanted to experiment with swapping out backbones, but I'm not sure where to start.

resume-from

--resume-from option in train.py works only once due to the bug regarding epoch update in mmcv-full=1.3.8. The bug is solved in 1.3.9. so I suggest to update requirement in mminstal.txt accordingly.

I encounter the error KeyError: 'gt_bboxes_3d' when i try to use visualize_results.py

The command i used:
python tools/misc/visualize_results.py configs/fcaf3d/fcaf3d_sunrgbd-3d-10class.py --result data/sunrgbd/sunrgbd_infos_val.pkl --show-dir workdir

The result i get:
Traceback (most recent call last):
File "tools/misc/visualize_results.py", line 48, in
main()
File "tools/misc/visualize_results.py", line 40, in main
dataset.show(results, args.show_dir) # use default pipeline
File "/home/intern/Downloads/Samsunglabs/fcaf3d/mmdet3d/datasets/sunrgbd_dataset.py", line 207, in show
pred_bboxes = result['boxes_3d']
KeyError: 'boxes_3d'

train.py: error: unrecognized arguments: --load-from fully_conv_s3dis.pth

Hi，it is me again.I want to use the your offered pretrained model of s3dis to train my own dataset.So I need to use the 'load-from' to fine tune the model.But when i use the 'load-from',it throws the error:
train.py: error: unrecognized arguments: --load-from fully_conv_s3dis.pth
How to deal with this situation?
Thank you very much!

Some question on code (About the origin in class DepthInstance3DBoxes )

Thanks for your great work, when I read the code, I have some confuse.

In the file mmdet3d/models/dense_heads/fcaf3d_neck_with_head.py, we can see function loss_single for train, and get_box_single for evaluation. I just want to know, the origin of the box predicated by network is (0.5, 0.5, 0) or (0.5, 0.5, 0.5).
Because I see that when we get the box_loss in function '_loss_single', the gt_box is convert to the class with origin(0.5, 0.5, 0.5), and the box predicated by the network do nothing(after _bbox_pred_to_bbox), then we get the loss_bbox. So we can think the p red box's origin is (0.5, 0.5, 0.5).
But when we evaluate the network, the predicated box in function '_get_box_single' is convert to box with origin(0.5, 0.5, 0.5), then the boxes are evaluated by the function('indoor_eval') with gt_box which was convert to box with origin(0.5, 0.5, 0,5)

So I confused with above code, thus, when I using the box to test some others tasks, I have no idea to use the original box or convert it to the box with origin(0.5, 0.5, 0.5)

Nothing happened when I visualized my latest.pth

Thank you for your Greak work!
I followed your dockerfile and everything was well-done.
training and evaluation were great.
but when i visualized my latest.pth
there is nothing happened
it seemed like processing my checkpoint file but i cannot understand what was done(any file didn't be produced).

should i additionally install more program to visualize something?
here is my problem.

root@0dea488ce090:/mmdetection3d# python tools/test.py configs/fcaf3d/fcaf3d_sunrgbd-3d-10class.py /mmdetection3d/work_dirs/fcaf3d_sunrgbd-3d-10class/latest.pth --show --show-dir /mmdetection3d/work_dirs/fcaf3d_sunrgbd-3d-10class Use load_from_local loader [>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 5050/5050, 8.0 task/s, elapsed: 632s, ETA: 0sroot@0dea488ce090:/mmdetection3d# root@0dea488ce090:/mmdetection3d#

I'd like to detect small objects

sorry for bothering you.
I have been studying this project and I have some questions.

1. I'd like to detect small objects like cups, lamps and such things.
   when i trained and evaluated sunrgbd datas including those. it reported good results. like below
   
   but when I visualized three object files, ground truth and prediction both ignored all the small objects that I included.
   they only displayed relatively big objects like table,chair,bed,desk (I set score_thr = 0.1)
   
   how can I modify to obtain the things desired?

2. how can I feed my own datas?
   finally I'd like to train and predict my own datas(i.g apple.)
   I'm likely to know how to train and evaluate and visualize my datas (just adding my datas to sunrgbd pipelines.)
   my question is that If i train well, how could I feed my real time a single data to model in order to predict centroids or bounding boxes.
   (format might be .ply or .pcd)
   is there any commands to do this work?

ModuleNotFoundError: No module named 'mmdet3d.ops.pcdet_nms'

visualization question

hello,I have generate the .obj ，but，when i open it with meshlab .It shows like the picture.

The error about training the S3DIS dataset

Hello, an error ocurred when I tried to train the S3DIS dataset:

Traceback (most recent call last):
File "tools/train.py", line 223, in
main()
File "tools/train.py", line 212, in main
train_model(
File "/mmdetection3d/mmdet3d/apis/train.py", line 27, in train_model
train_detector(
File "/opt/conda/lib/python3.8/site-packages/mmdet/apis/train.py", line 170, in train_detector
runner.run(data_loaders, cfg.workflow)
File "/opt/conda/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 127, in run
epoch_runner(data_loaders[i], **kwargs)
File "/opt/conda/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 50, in train
self.run_iter(data_batch, train_mode=True, **kwargs)
File "/opt/conda/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 29, in run_iter
outputs = self.model.train_step(data_batch, self.optimizer,
File "/opt/conda/lib/python3.8/site-packages/mmcv/parallel/data_parallel.py", line 67, in train_step
return self.module.train_step(*inputs[0], **kwargs[0])
File "/opt/conda/lib/python3.8/site-packages/mmdet/models/detectors/base.py", line 237, in train_step
losses = self(**data)
File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/opt/conda/lib/python3.8/site-packages/mmcv/runner/fp16_utils.py", line 98, in new_func
return old_func(*args, **kwargs)
File "/mmdetection3d/mmdet3d/models/detectors/base.py", line 58, in forward
return self.forward_train(**kwargs)
File "/mmdetection3d/mmdet3d/models/detectors/single_stage_sparse.py", line 48, in forward_train
x = self.extract_feat(points, img_metas)
File "/mmdetection3d/mmdet3d/models/detectors/single_stage_sparse.py", line 40, in extract_feat
x = self.neck_with_head(x)
File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/mmdetection3d/mmdet3d/models/dense_heads/fcaf3d_neck_with_head.py", line 102, in forward
x = self._prune(x, scores)
File "/mmdetection3d/mmdet3d/models/dense_heads/fcaf3d_neck_with_head.py", line 124, in _prune
prune_mask[permutation[mask]] = True
RuntimeError: invalid shape dimension -255

I don't know how to solve it. Looking forward to your reply!

Performance on SUN RGB-D

I get some trouble when reproducing the results on the SUN RGB-Ddataset. When I train FCAF3D with the following command for 3 times:

CUDA_VISIBLE_DEVICES=5,6 bash tools/dist_train.sh configs/fcaf3d/fcaf3d_sunrgbd-3d-10class.py 2

I get results as follows:

+-------------+---------+---------+---------+---------+
| classes     | AP_0.25 | AR_0.25 | AP_0.50 | AR_0.50 |
+-------------+---------+---------+---------+---------+
| bed         | 0.8719  | 0.9845  | 0.6141  | 0.7456  |
| table       | 0.4641  | 0.9672  | 0.2555  | 0.6465  |
| sofa        | 0.6754  | 0.9745  | 0.4773  | 0.7703  |
| chair       | 0.7990  | 0.9602  | 0.6536  | 0.8373  |
| toilet      | 0.9059  | 0.9931  | 0.6496  | 0.8138  |
| desk        | 0.2875  | 0.9431  | 0.0774  | 0.4819  |
| dresser     | 0.3682  | 0.9358  | 0.1909  | 0.5780  |
| night_stand | 0.6696  | 0.9647  | 0.5330  | 0.8118  |
| bookshelf   | 0.2164  | 0.8156  | 0.0430  | 0.2376  |
| bathtub     | 0.7385  | 0.9388  | 0.4736  | 0.6939  |
+-------------+---------+---------+---------+---------+
| Overall     | 0.5997  | 0.9477  | 0.3968  | 0.6617  |
+-------------+---------+---------+---------+---------+

+-------------+---------+---------+---------+---------+
| classes     | AP_0.25 | AR_0.25 | AP_0.50 | AR_0.50 |
+-------------+---------+---------+---------+---------+
| bed         | 0.8724  | 0.9883  | 0.6265  | 0.7553  |
| table       | 0.4751  | 0.9706  | 0.2678  | 0.6520  |
| sofa        | 0.6932  | 0.9856  | 0.4864  | 0.7352  |
| chair       | 0.8149  | 0.9672  | 0.6757  | 0.8485  |
| toilet      | 0.9139  | 1.0000  | 0.6214  | 0.8000  |
| desk        | 0.2901  | 0.9511  | 0.0759  | 0.4862  |
| dresser     | 0.3544  | 0.9633  | 0.1908  | 0.6147  |
| night_stand | 0.6701  | 0.9608  | 0.5355  | 0.8118  |
| bookshelf   | 0.2548  | 0.8794  | 0.0505  | 0.2872  |
| bathtub     | 0.7533  | 0.9796  | 0.5332  | 0.6735  |
+-------------+---------+---------+---------+---------+
| Overall     | 0.6092  | 0.9646  | 0.4064  | 0.6665  |
+-------------+---------+---------+---------+---------+

+-------------+---------+---------+---------+---------+
| classes     | AP_0.25 | AR_0.25 | AP_0.50 | AR_0.50 |
+-------------+---------+---------+---------+---------+
| bed         | 0.8681  | 0.9553  | 0.5910  | 0.7146  |
| table       | 0.4706  | 0.9497  | 0.2596  | 0.6427  |
| sofa        | 0.6792  | 0.9665  | 0.4032  | 0.7225  |
| chair       | 0.7962  | 0.9533  | 0.6424  | 0.8232  |
| toilet      | 0.9123  | 0.9931  | 0.6393  | 0.7862  |
| desk        | 0.2962  | 0.9283  | 0.0804  | 0.4718  |
| dresser     | 0.3405  | 0.9083  | 0.1785  | 0.5596  |
| night_stand | 0.6685  | 0.9294  | 0.5227  | 0.7529  |
| bookshelf   | 0.1860  | 0.7730  | 0.0451  | 0.2376  |
| bathtub     | 0.7124  | 0.9592  | 0.3878  | 0.6327  |
+-------------+---------+---------+---------+---------+
| Overall     | 0.5930  | 0.9316  | 0.3750  | 0.6344  |
+-------------+---------+---------+---------+---------+

Generalization of FCAF3D

Thanks for your sharing. The FCAF3D is concise and effective. Have you ever tried to apply it to outdoor scenes?

Can MinkowskiGenerativeConvolutionTranspose from fcaf3d_neck_with_head.py replaced?

Hi, thank you for the great work!

I noticed that the code uses MinkowskiEngine as the backend. I am trying to replace the backend with Spconv, which is one of the different sparse convolution libraries for a research purpose.

I found out that fcaf3d is using MinkowkiGenerativeConvolutionTranspose in fcaf3d_neck_with_head.py. I am not sure how I could replace that function with spconv apis.
How is MinkowkiGenerativeConvolutionTranspose different from MinkowskiConvolutionTranspose? The documentation says it produces new coordinates, but is this necessary, or can be replaced with MinkowskiConvolutionTranspose?
If not, is there any workaround for this?

Thanks!

I follow the step of file 'read me' except that i use only a single gpu and it errors: "RuntimeError: merge_sort: failed to synchronize: cudaErrorIllegalAddress: an illegal memory access was encountered"

RuntimeError: merge_sort: failed to synchronize: cudaErrorIllegalAddress: an illegal memory access was encountered

output all predict bbox coordinate position

Hello,I successfully run your code,but I want to output all predict bbox coordinate position.I have tried print the bbox_list in single_stage_mono3d.py,but when i run the tools/test.py ,I found it did not call the single_stage_mono3d.py.Could you please help me ?

How to solve the error reported during the training trial

on3d$ bash tools/dist_train.sh configs/fcaf3d/fcaf3d_8x2_sunrgbd-3d-10class.py 2*****************************************
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.

---

/home/air/anaconda3/envs/mmde/lib/python3.7/site-packages/mmdet/utils/setup_env.py:49: UserWarning: Setting MKL_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
f'Setting MKL_NUM_THREADS environment variable for each process '
/home/air/anaconda3/envs/mmde/lib/python3.7/site-packages/mmdet/utils/setup_env.py:49: UserWarning: Setting MKL_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
f'Setting MKL_NUM_THREADS environment variable for each process '

Performance on ScanNet

Hi, awesome work! But I get some trouble when reproducing the results on the ScanNet dataset. Could you give me some advice?
When I train FCAF3D with the following command:

bash tools/dist_train.sh configs/fcaf3d/fcaf3d_scannet-3d-18class.py 2

I get results as follows

+----------------+---------+---------+---------+---------+
| classes | AP_0.25 | AR_0.25 | AP_0.50 | AR_0.50 |
+----------------+---------+---------+---------+---------+
| cabinet | 0.5393 | 0.9059 | 0.3704 | 0.7366 |
| bed | 0.8674 | 0.9259 | 0.7992 | 0.8519 |
| chair | 0.9542 | 0.9854 | 0.8964 | 0.9415 |
| sofa | 0.9118 | 0.9794 | 0.7914 | 0.9381 |
| table | 0.7015 | 0.8971 | 0.6479 | 0.8029 |
| door | 0.6186 | 0.9229 | 0.4389 | 0.7195 |
| window | 0.5750 | 0.8865 | 0.3336 | 0.5993 |
| bookshelf | 0.5852 | 0.8831 | 0.5157 | 0.7922 |
| picture | 0.2571 | 0.5721 | 0.1564 | 0.3649 |
| counter | 0.6106 | 0.8846 | 0.2338 | 0.5385 |
| desk | 0.7157 | 0.9685 | 0.5472 | 0.8819 |
| curtain | 0.5929 | 0.8955 | 0.4346 | 0.7015 |
| refrigerator | 0.4901 | 0.8772 | 0.4010 | 0.8246 |
| showercurtrain | 0.8306 | 0.9643 | 0.4340 | 0.7857 |
| toilet | 1.0000 | 1.0000 | 0.9378 | 0.9655 |
| sink | 0.8543 | 0.9592 | 0.5197 | 0.6837 |
| bathtub | 0.8684 | 0.9032 | 0.8299 | 0.8710 |
| garbagebin | 0.6439 | 0.8717 | 0.5762 | 0.7604 |
+----------------+---------+---------+---------+---------+
| Overall | 0.7009 | 0.9046 | 0.5480 | 0.7644 |
+----------------+---------+---------+---------+---------+

There is about 0.6 mAP gap between my result (70.09) and paper result (70.7) on AP 0.25 metric, and there is about 1.2 mAP gap between my result (54.8) and paper result (56.0) on AP 0.5 metric. Is this variance? or do I need to modify something to achieve comparable results with paper? Thanks ~

Training on KITTI

@filaPro hi thanks for opensourcing code base , does this architecture been trained on KITTI , ARGOVerse, Nuscense dataset? if not what changes have to be made for the training on those dataset
Thanks in advance

How can I generate per point classification?

Hi, thanks for sharing your excellent work first! I have several questions.

I met the following error while testing.
For my understanding, there are no annotations available while testing. So after a hard debug, I found that the second composer of test_pipeline called the 'annos' key of my test.pkl data, so I comment out this line dict(type='GlobalAlignment', rotation_axis=2), in the config file so that I can get a result. But actually I'm not sure will it cause any alignment problem on the results.

File "tools/test.py", line 214, in
main()
File "tools/test.py", line 184, in main
outputs = single_gpu_test(model, data_loader, args.show, args.show_dir)
File "/media/yangtian/SATA3/Workspace/fcaf3d-master/mmdet3d/apis/test.py", line 36, in single_gpu_test
for i, data in enumerate(data_loader):
File "/home/yangtian/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 517, in next
data = self._next_data()
File "/home/yangtian/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1199, in _next_data
return self._process_data(data)
File "/home/yangtian/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1225, in _process_data
data.reraise()
File "/home/yangtian/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/torch/_utils.py", line 429, in reraise
raise self.exc_type(msg)
KeyError: Caught KeyError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/yangtian/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 202, in _worker_loop
data = fetcher.fetch(index)
File "/home/yangtian/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/yangtian/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/media/yangtian/SATA3/Workspace/fcaf3d-master/mmdet3d/datasets/custom_3d.py", line 354, in getitem
return self.prepare_test_data(idx)
File "/media/yangtian/SATA3/Workspace/fcaf3d-master/mmdet3d/datasets/scannet_dataset.py", line 135, in prepare_test_data
example = self.pipeline(input_dict)
File "/home/yangtian/anaconda3/envs/open-mmlab/lib/python3.8/site-packages/mmdet/datasets/pipelines/compose.py", line 41, in call
data = t(data)
File "/media/yangtian/SATA3/Workspace/fcaf3d-master/mmdet3d/datasets/pipelines/transforms_3d.py", line 475, in call
assert 'axis_align_matrix' in input_dict['ann_info'].keys(),
KeyError: 'ann_info'

And after that, I wanna visualize the results, so I added--show --show_dir ./results/custom_test/ . However met another error:

File "tools/test.py", line 214, in
main()
File "tools/test.py", line 184, in main
outputs = single_gpu_test(model, data_loader, args.show, args.show_dir)
File "/media/yangtian/SATA3/Workspace/fcaf3d-master/mmdet3d/apis/test.py", line 46, in single_gpu_test
model.module.show_results(data, result, out_dir)
File "/media/yangtian/SATA3/Workspace/fcaf3d-master/mmdet3d/models/detectors/base.py", line 107, in show_results
show_result(points, None, pred_bboxes, out_dir, file_name)
TypeError: show_result() missing 2 required positional arguments: 'out_dir' and 'filename'

Not fixed yet. But I found that in show_result.py, the function show_result() takes gt_labels and pred_labels as well while in the testing procedure it doesn't pass this two values. I'm gonna continue debugging on it.

Finally, I'm wondering if there's a convenience way to get per point classification results?

RuntimeError: NCCL error in: /opt/conda/conda-bld/pytorch_1603729009598/work/torch/lib/c10d/ProcessGroupNCCL.cpp:784, invalid usage, NCCL version 2.7.8

hello，when i use your s3dis pretrained model to test,it throws the following errors.Could you please help me ?
RuntimeError: NCCL error in: /opt/conda/conda-bld/pytorch_1603729009598/work/torch/lib/c10d/ProcessGroupNCCL.cpp:784, invalid usage, NCCL version 2.7.8