luoyadan/crb-active-3ddet

The official PyTorch implementation of "Exploring Active 3D Object Detection from a Generalization Perspective" (ICLR Spotlight 2023).

---

This repository is the official Pytorch implementation of our work:

[ICLR 2023] CRB: Exploring Active 3D Object Detection from a Generalization Perspective.

[OpenReview] [arXiv] [Supplementary Material]

[In Submission] Open-CRB: Towards Open World Active Learning for 3D Object Detection.

[Open-CRB Branch] [arXiv]

🔥 11/23 updates: release the code and the preprint of Open-CRB

🔥 02/23 updates: checkpoints available at https://drive.google.com/drive/folders/1PMb6tu84AIw66vCRrMBCHpnBeL5WMkuv?usp=sharing

Framework

To alleviate the high annotation cost in LiDAR-based 3D object detection, active learning is a promising solution that learns to select only a small portion of unlabeled data to annotate, without compromising model performance. Our empirical study, however, suggests that mainstream uncertainty-based and diversity-based active learning policies are not effective when applied in the 3D detection task, as they fail to balance the trade-off between point cloud informativeness and box-level annotation costs. To overcome this limitation, we jointly investigate three novel criteria in our framework CRB for point cloud acquisition - label conciseness, feature representativeness and geometric balance, which hierarchically filters out the point clouds of redundant 3D bounding box labels, latent features and geometric characteristics (e.g., point cloud density) from the unlabeled sample pool and greedily selects informative ones with fewer objects to annotate. Our theoretical analysis demonstrates that the proposed criteria aligns the marginal distributions of the selected subset and the prior distributions of the unseen test set, and minimizes the upper bound of the generalization error. To validate the effectiveness and applicability of CRB, we conduct extensive experiments on the two benchmark 3D object detection datasets of KITTI and Waymo and examine both one-stage (i.e., SECOND) and two-stage 3D detectors (i.e., PV-RCNN). Experiments evidence that the proposed approach outperforms existing active learning strategies and achieves fully supervised performance requiring 1% and 8% annotations of bounding boxes and point clouds, respectively.

---

Contents

• Installation
  • Requirements
  • pcdet v0.5
• Getting Started
  • Requirements
  • KITTI Dataset
  • Waymo Open Dataset
  • Training & Testing

Installation

Requirements

All the codes are tested in the following environment:

• Python 3.6+
• PyTorch 1.10.1
• CUDA 11.3
• wandb 0.12.11
• spconv-cu113 v2.1.21

Install pcdet v0.5

Our implementations of 3D detectors are based on the lastest OpenPCDet. To install this pcdet library and its dependent libraries, please run the following command:

python setup.py develop

NOTE: Please re-install even if you have already installed pcdet previoursly.

Getting Started

The active learning configs are located at tools/cfgs/active-kitti_models and /tools/cfgs/active-waymo_models for different AL methods. The dataset configs are located within tools/cfgs/dataset_configs, and the model configs are located within tools/cfgs for different datasets.

Dataset Preparation

Currently we provide the dataloader of KITTI dataset and Waymo dataset, and the supporting of more datasets are on the way.

KITTI Dataset

• Please download the official KITTI 3D object detection dataset and organize the downloaded files as follows (the road planes could be downloaded from [road plane], which are optional for data augmentation in the training):
• If you would like to train CaDDN, download the precomputed depth maps for the KITTI training set
• NOTE: if you already have the data infos from pcdet v0.1, you can choose to use the old infos and set the DATABASE_WITH_FAKELIDAR option in tools/cfgs/dataset_configs/kitti_dataset.yaml as True. The second choice is that you can create the infos and gt database again and leave the config unchanged.

CRB-active-3Ddet
├── data
│   ├── kitti
│   │   │── ImageSets
│   │   │── training
│   │   │   ├──calib & velodyne & label_2 & image_2 & (optional: planes) & (optional: depth_2)
│   │   │── testing
│   │   │   ├──calib & velodyne & image_2
├── pcdet
├── tools

• Generate the data infos by running the following command:

python -m pcdet.datasets.kitti.kitti_dataset create_kitti_infos tools/cfgs/dataset_configs/kitti_dataset.yaml

Waymo Open Dataset

• Please download the official Waymo Open Dataset, including the training data training_0000.tar~training_0031.tar and the validation data validation_0000.tar~validation_0007.tar.
• Unzip all the above xxxx.tar files to the directory of data/waymo/raw_data as follows (You could get 798 train tfrecord and 202 val tfrecord ):

CRB-active-3Ddet
├── data
│   ├── waymo
│   │   │── ImageSets
│   │   │── raw_data
│   │   │   │── segment-xxxxxxxx.tfrecord
|   |   |   |── ...
|   |   |── waymo_processed_data_v0_5_0
│   │   │   │── segment-xxxxxxxx/
|   |   |   |── ...
│   │   │── waymo_processed_data_v0_5_0_gt_database_train_sampled_1/
│   │   │── waymo_processed_data_v0_5_0_waymo_dbinfos_train_sampled_1.pkl
│   │   │── waymo_processed_data_v0_5_0_gt_database_train_sampled_1_global.npy (optional)
│   │   │── waymo_processed_data_v0_5_0_infos_train.pkl (optional)
│   │   │── waymo_processed_data_v0_5_0_infos_val.pkl (optional)
├── pcdet
├── tools

• Install the official waymo-open-dataset by running the following command:

pip3 install --upgrade pip
pip3 install waymo-open-dataset-tf-2-0-0==1.2.0 --user

Waymo version in our project is 1.2.0

• Extract point cloud data from tfrecord and generate data infos by running the following command (it takes several hours, and you could refer to data/waymo/waymo_processed_data_v0_5_0 to see how many records that have been processed):

python -m pcdet.datasets.waymo.waymo_dataset --func create_waymo_infos \
    --cfg_file tools/cfgs/dataset_configs/waymo_dataset.yaml

Note that you do not need to install waymo-open-dataset if you have already processed the data before and do not need to evaluate with official Waymo Metrics.

Training & Testing

Test and evaluate the pretrained models

The weights of our pre-trained model will be released upon acceptance.

• Test with a pretrained model:

python test.py --cfg_file ${CONFIG_FILE} --batch_size ${BATCH_SIZE} --ckpt ${CKPT}

• To test all the saved checkpoints of a specific training setting and draw the performance curve on the Tensorboard, add the --eval_all argument:

python test.py --cfg_file ${CONFIG_FILE} --batch_size ${BATCH_SIZE} --eval_all

• To test with multiple GPUs:

sh scripts/dist_test.sh ${NUM_GPUS} \
    --cfg_file ${CONFIG_FILE} --batch_size ${BATCH_SIZE}

# or

sh scripts/slurm_test_mgpu.sh ${PARTITION} ${NUM_GPUS} \
    --cfg_file ${CONFIG_FILE} --batch_size ${BATCH_SIZE}

Train a backbone

In our active learning setting, the 3D detector will be pre-trained with a small labeled set $\mathcal{D}_L$ which is randomly sampled from the trainig set. To train such a backbone, please run

sh scripts/${DATASET}/train_${DATASET}_backbone.sh

Train with different active learning strategies

We provide several options for active learning algorithms, including

• random selection [random]
• confidence sample [confidence]
• entropy sampling [entropy]
• MC-Reg sampling [montecarlo]
• greedy coreset [coreset]
• learning loss [llal]
• BADGE sampling [badge]
• CRB sampling [crb]

You could optionally add extra command line parameters --batch_size ${BATCH_SIZE} and --epochs ${EPOCHS} to specify your preferred parameters.

• Train:

python train.py --cfg_file ${CONFIG_FILE}

crb-active-3ddet's Issues

Apply CRB active learning to other models

Hello! Thank you for your amazing work!
Recently I am trying to apply CRB to other models since PV RCNN requires to much GPU memories.
I chose Pointpillar and kitti dataset but it comes out this error:

Traceback (most recent call last):
File "train.py", line 313, in
main()
File "train.py", line 255, in main
dist_train=dist_train
File "/home/CRB-active-3Ddet/tools/train_utils/train_active_utils.py", line 313, in train_model_active
accumulated_iter=accumulated_iter
File "../pcdet/utils/active_training_utils.py", line 271, in select_active_labels
selected_frames = strategy.query(leave_pbar, cur_epoch)
File "../pcdet/query_strategies/crb_sampling.py", line 80, in query
pred_dicts, _ = self.model(unlabelled_batch)
File "/opt/conda/envs/crb/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "../pcdet/models/detectors/pointpillar.py", line 21, in forward
pred_dicts, recall_dicts = self.post_processing(batch_dict)
File "../pcdet/models/detectors/detector3d_template.py", line 232, in post_processing
full_cls_scores = batch_dict['full_cls_scores'][batch_mask]
KeyError: 'full_cls_scores'

I will put my config on the next comment.

Is there anything wrong in my config file？Or is it possible to apply CRB to Pointpillar?
Thanks so much and looking forward to your reply.

Thank you for providing such excellent work. Is the code of OWAL in this project?

Thank you for providing such excellent work. Is the code of OWAL in this project?

Could not install waymo-open-dataset

Dear author,

at the read me there is a line: pip3 install waymo-open-dataset-tf-2-0-0==1.2.0 --user; but when I run it it return: ERROR: Could not find a version that satisfies the requirement waymo-open-dataset-tf-2-0-0==1.2.0 (from versions: none)
ERROR: No matching distribution found for waymo-open-dataset-tf-2-0-0==1.2.0

Request for Detailed Data Release from CRB-active-3Ddet (Figure 2, Waymo Dataset)

Dear Yadan Luo,

I would like to express my gratitude once again for your work on CRB-active-3Ddet and for your previous responses. Your contributions have been invaluable. I am writing to inquire if you could release the specific data used in Figure 2, particularly the section pertaining to the Waymo dataset, in a table format. I am interested in comparing the results I have obtained through my own runs with those presented in the figure. This data would greatly assist in furthering my understanding and analysis.

Thank you for considering this request.

Best regards,
Ruiyu Mao

how to calculate entropy of boxes we don't know the confidence of all classes?

Hi, thanks for your great work.

I have a question about Concise Label Sampling stage.
When we calculate the entropy of a box, confidence of each box of all classes must be known. However, for some anchor-free detection models (such as CenterPoint), we only know the confidence of the class the model predicts. In this case, how can we get the entropy?

Thanks.

Accuracy up and down while doing with CRB-active

Hi Team,

great work,

i was running the code on kitti dataset, i added the evaluation code after saving the model so that i can capture the metrics after saving the model. Unfortunately the metrics are going up and down as shown below.

<style type="text/css"></style>

epoch - 41
class Name		bbox				bev				3d				aos
Car [email protected]: 0.70: 0.70:	21.9936	18.138	17.108		10.5826	10.379	8.013		0.9788	0.9091	0.909		16.54	14.24	13.3
Car [email protected]: 0.70: 0.70:	17.9106	12.9167	11.359		9.0408	7.3997	5.615		0.4922	0.3353	0.285		10.91	7.84	6.5
Car [email protected]: 0.50: 0.50:	21.9936	18.138	17.108		61.1646	47.4221	39.889		38.5874	28.9249	27.177		16.54	14.24	13.3
Car [email protected]: 0.50: 0.50:	17.9106	12.9167	11.359		59.3017	43.534	36.76		36.3289	27.0235	21.639		10.91	7.84	6.5
Pedestrian [email protected]: 0.50: 0.50:	0.0957	0.0955	0.114		0.0645	0.0645	0.064		0	0	0		0.05	0.05	0
Pedestrian [email protected]: 0.50: 0.50:	0.0263	0.0263	0.031		0	0	0		0	0	0		0.01	0.01	0
Pedestrian [email protected]: 0.25: 0.25:	0.0957	0.0955	0.114		0.1843	0.1843	0.184		0.1806	0.1806	0.18		0.05	0.05	0
Pedestrian [email protected]: 0.25: 0.25:	0.0263	0.0263	0.031		0.0507	0.0507	0.05		0.0497	0.0497	0.049		0.01	0.01	0
Cyclist [email protected]: 0.50: 0.50:	0	0	0		0	0	0		0	0	0		0	0	0
Cyclist [email protected]: 0.50: 0.50:	0	0	0		0	0	0		0	0	0		0	0	0
Cyclist [email protected]: 0.25: 0.25:	0	0	0		0	0	0		0	0	0		0	0	0
Cyclist [email protected]: 0.25: 0.25:	0	0	0		0	0	0		0	0	0		0	0	0
epoch - 81
class Name		bbox				bev				3d				aos
Car [email protected]: 0.70: 0.70:	13.1114	11.998	8.736		10.4223	10.5382	9.09		3.0303	3.0303	3.03		7.85	7.53	5.5
Car [email protected]: 0.70: 0.70:	10.5795	8.4826	7.656		2.3562	2.1066	1.569		0.203	0.2809	0.134		6.39	5.51	5
Car [email protected]: 0.50: 0.50:	13.1114	11.998	8.736		40.1075	29.5394	23.139		27.383	20.1661	19.535		7.85	7.53	5.5
Car [email protected]: 0.50: 0.50:	10.5795	8.4826	7.656		38.2718	26.5619	21.072		22.4848	16.8061	13.745		6.39	5.51	5
Pedestrian [email protected]: 0.50: 0.50:	0	0	0		0	0.0298	0.029		0	0.0298	0.029		0	0	0
Pedestrian [email protected]: 0.50: 0.50:	0	0	0		0	0	0		0	0	0		0	0	0
Pedestrian [email protected]: 0.25: 0.25:	0	0	0		0.0155	0.0298	0.029		0.0155	0.0298	0.029		0	0	0
Pedestrian [email protected]: 0.25: 0.25:	0	0	0		0	0.0068	0.006		0	0.0068	0.006		0	0	0
Cyclist [email protected]: 0.50: 0.50:	0	0	0		0	0	0		0	0	0		0	0	0
Cyclist [email protected]: 0.50: 0.50:	0	0	0		0	0	0		0	0	0		0	0	0
Cyclist [email protected]: 0.25: 0.25:	0	0	0		0	0	0		0	0	0		0	0	0
Cyclist [email protected]: 0.25: 0.25:	0	0	0		0	0	0		0	0	0		0	0	0
epoch - 121
class Name		bbox				bev				3d				aos
Car [email protected]: 0.70: 0.70:	38.1252	30.3541	26.062		23.1726	21.3098	17.782		5.9594	5.9046	5.596		19.66	15.88	12.9
Car [email protected]: 0.70: 0.70:	33.1607	25.6126	21.772		17.3449	15.0129	12.887		2.611	2.1974	1.833		18.16	14.02	11.2
Car [email protected]: 0.50: 0.50:	38.1252	30.3541	26.062		75.5587	58.5069	50.252		61.198	46.8493	39.472		19.66	15.88	12.9
Car [email protected]: 0.50: 0.50:	33.1607	25.6126	21.772		77.5033	60.5005	51.584		61.4285	47.2183	39.363		18.16	14.02	11.2
Pedestrian [email protected]: 0.50: 0.50:	0	0.0652	1.298		0.0227	0.0395	0.189		0	0.0344	0.034		0	0.03	1.1
Pedestrian [email protected]: 0.50: 0.50:	0	0.0179	0.044		0.0063	0.0109	0.021		0	0	0		0	0.01	0
Pedestrian [email protected]: 0.25: 0.25:	0	0.0652	1.298		0.04	0.1347	0.324		0.0322	0.112	0.324		0	0.03	1.1
Pedestrian [email protected]: 0.25: 0.25:	0	0.0179	0.044		0.011	0.037	0.057		0.0088	0.0308	0.052		0	0.01	0
Cyclist [email protected]: 0.50: 0.50:	0	0	0		0	0	0		0	0	0		0	0	0
Cyclist [email protected]: 0.50: 0.50:	0	0	0		0	0	0		0	0	0		0	0	0
Cyclist [email protected]: 0.25: 0.25:	0	0	0		0	0	0		0	0	0		0	0	0
Cyclist [email protected]: 0.25: 0.25:	0	0	0		0	0	0		0	0	0		0	0	0
epoch - 161
class Name		bbox				bev				3d				aos
Car [email protected]: 0.70: 0.70:	24.1541	23.1566	19.195		16.0264	15.3192	12.599		5.3866	4.5455	4.545		17.62	16.87	14.5
Car [email protected]: 0.70: 0.70:	17.9364	16.2138	13.954		9.1309	7.9156	6.737		1.1948	1.0228	0.983		10.81	10.06	8.3
Car [email protected]: 0.50: 0.50:	24.1541	23.1566	19.195		54.9687	41.5615	33.856		37.5072	29.3765	28.514		17.62	16.87	14.5
Car [email protected]: 0.50: 0.50:	17.9364	16.2138	13.954		53.3887	40.2478	33.687		33.7264	27.7644	22.683		10.81	10.06	8.3
Pedestrian [email protected]: 0.50: 0.50:	0	0	0		0	0	0		0	0	0		0	0	0
Pedestrian [email protected]: 0.50: 0.50:	0	0	0		0	0	0		0	0	0		0	0	0
Pedestrian [email protected]: 0.25: 0.25:	0	0	0		0	0.0511	0.051		0	0.0511	0.051		0	0	0
Pedestrian [email protected]: 0.25: 0.25:	0	0	0		0	0	0		0	0	0		0	0	0
Cyclist [email protected]: 0.50: 0.50:	0	0	0		0	0	0		0	0	0		0	0	0
Cyclist [email protected]: 0.50: 0.50:	0	0	0		0	0	0		0	0	0		0	0	0
Cyclist [email protected]: 0.25: 0.25:	0	0	0		0	0	0		0	0	0		0	0	0
Cyclist [email protected]: 0.25: 0.25:	0	0	0		0	0	0		0	0	0		0	0	0
epoch - 201
class Name		bbox				bev				3d				aos
Car [email protected]: 0.70: 0.70:	27.6438	23.33	23.126		6.7206	6.5229	6.461		4.5455	4.5455	4.545		20.48	17.91	18.1
Car [email protected]: 0.70: 0.70:	21.2595	18.2876	16.207		2.8905	3.8376	2.9		0.1402	0.2449	0.242		13.31	11.64	10.5
Car [email protected]: 0.50: 0.50:	27.6438	23.33	23.126		60.4661	54.4379	47.19		39.8048	33.637	28.079		20.48	17.91	18.1
Car [email protected]: 0.50: 0.50:	21.2595	18.2876	16.207		60.7818	50.9341	42.812		35.5384	30.7018	26.033		13.31	11.64	10.5
Pedestrian [email protected]: 0.50: 0.50:	0.0196	0.0372	0.037		0.0194	0.0344	0.034		0.0194	0.0344	0.034		0	0.03	0
Pedestrian [email protected]: 0.50: 0.50:	0	0.0102	0.01		0	0.0095	0.009		0	0.0095	0.009		0	0	0
Pedestrian [email protected]: 0.25: 0.25:	0.0196	0.0372	0.037		0.0195	0.037	0.037		0.0194	0.0344	0.034		0	0.03	0
Pedestrian [email protected]: 0.25: 0.25:	0	0.0102	0.01		0	0.0102	0.01		0	0.0095	0.009		0	0	0
Cyclist [email protected]: 0.50: 0.50:	0	0	0		0	0	0		0	0	0		0	0	0
Cyclist [email protected]: 0.50: 0.50:	0	0	0		0	0	0		0	0	0		0	0	0
Cyclist [email protected]: 0.25: 0.25:	0	0	0		0	0	0		0	0	0		0	0	0
Cyclist [email protected]: 0.25: 0.25:	0	0	0		0	0	0		0	0	0		0	0	0
epoch - 241
class Name		bbox				bev				3d				aos
Car [email protected]: 0.70: 0.70:	30.2285	23.9599	24.154		25.0885	21.6854	17.929		7.6825	3.4259	3.438		11.91	9.93	9.5
Car [email protected]: 0.70: 0.70:	27.8177	19.3108	17.067		18.197	14.6932	12.388		3.7072	2.2674	1.873		11.45	8.85	7.3
Car [email protected]: 0.50: 0.50:	30.2285	23.9599	24.154		58.5319	48.2936	40.687		46.1676	37.1298	30.594		11.91	9.93	9.5
Car [email protected]: 0.50: 0.50:	27.8177	19.3108	17.067		56.986	45.7902	39.367		46.7604	36.4906	29.421		11.45	8.85	7.3
Pedestrian [email protected]: 0.50: 0.50:	0	0	0		0	0	0		0	0	0		0	0	0
Pedestrian [email protected]: 0.50: 0.50:	0	0	0		0	0	0		0	0	0		0	0	0
Pedestrian [email protected]: 0.25: 0.25:	0	0	0		0	0	0		0	0	0		0	0	0
Pedestrian [email protected]: 0.25: 0.25:	0	0	0		0	0	0		0	0	0		0	0	0
Cyclist [email protected]: 0.50: 0.50:	0	0	0		0	0	0		0	0	0		0	0	0
Cyclist [email protected]: 0.50: 0.50:	0	0	0		0	0	0		0	0	0		0	0	0
Cyclist [email protected]: 0.25: 0.25:	0	0	0		0	0	0		0	0	0		0	0	0
Cyclist [email protected]: 0.25: 0.25:	0	0	0		0	0	0		0	0	0		0	0	0

attached as pdf too for clear understanding

CRB-Active - dummy.pdf

A simple question of the training schedule

I see the training schedule from active_train_util.py which says that the whole process requires a 40-epochs initialization of backbone. Then what confuse me is that you first select active samples, and then reload initial paramerters and train for one epoch. The trained model is used for active selection, but the model is reloaded back then to initial state. This process costs 200 epochs. I wonder why we need to train the model from original backbone for 200 times, but not 200 epochs on one model? Sorry if I misunderstand something.

Unknown query strategy "greedy_boxbadge"

Hi, thanks for your nice work.
I have the following problem when training with pv_rcnn_active_crb.yaml on kitti dataset

2023-05-03 12:11:58,126   INFO  **********************Start training active-kitti_models/pv_rcnn_active_crb(select-1
00)**********************
2023-05-03 12:11:58,127   INFO  ***** Start Active Pre-train *****
2023-05-03 12:11:58,181   INFO  **init backbone weights saved...**
train: 100%|███████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [
00:16<00:00,  3.07it/s, total_it=2000]
epochs: 100%|███████████████████████████████████████████████████████████████████████████████████████| 40/40 [11:00<0
0:00, 16.51s/it, loss=1.5, lr=1.69e-7]
2023-05-03 12:22:58,595   INFO  ***** Complete Active Pre-train *****
2023-05-03 12:22:58,602   INFO  ***** Start Active Train Loop *****
epochs:   0%|
              | 0/240 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "train.py", line 313, in <module>
    main()
  File "train.py", line 231, in main
    train_func(
  File "/home/coolshan/Project/CMLAB/3d_active_learning/CRB-active-3Ddet/tools/train_utils/train_active_utils.py", l
ine 302, in train_model_active
    = active_training_utils.select_active_labels(
  File "/home/coolshan/Project/CMLAB/3d_active_learning/CRB-active-3Ddet/tools/../pcdet/utils/active_training_utils.
py", line 252, in select_active_labels
    strategy = query_strategies.build_strategy(method=method, model=model, \
  File "/home/coolshan/Project/CMLAB/3d_active_learning/CRB-active-3Ddet/tools/../pcdet/query_strategies/__init__.py
", line 28, in build_strategy
    raise KeyError("Unknown query strategy:", method)
KeyError: ('Unknown query strategy:', 'greedy_boxbadge')

I can't find the greedy_boxbadge strategy in query_strategies/__init__.py too.

__factory = {
    'random': RandomSampling,
    'entropy': EntropySampling,
    'badge': BadgeSampling,
    'coreset': CoresetSampling,
    'llal': LLALSampling,
    'montecarlo': MonteCarloSampling,
    'confidence': ConfidenceSampling,
    'crb': CRBSampling
}

Is there something I'm missing here or an installation error? or how can I fixed.
Thanks

Problem with cumm and spconv.

Question about implementation of MC-Dropout

Hello, I have looked into the code (crb_sampling & montecarlo_sampling)
I checked that dropout layers are enabled for inference, but I couldn't find the stochastic forward passes to measure variances.

When I debugged in crb_sampling, it seemed that it only uses single forward pass after enabling dropout.
Would you please inform me if I have missed something? Thank you very much :D

Checkpoint performance is inconsistent with paper

Hello author,

After downloading the checkpoint provided by you, I re-ran test.py and found a discrepancy between the performance calculated and that presented in the paper. Below is the performance at 1% bbox (1000):

Have I missed any details? How can I achieve the performance mentioned in the paper?
Thanks!

Error occured after pre

Hello! Thanks for your work!
I'm faced with a trouble and hope you could help.
I'm trying to run train a model and I set batch_size=1 and epoches=20 by running

python` train.py --cfg_file cfgs/active-kitti_models/pv_rcnn_active_crb.yaml --batch_size 1 --epochs 20

And my config of active learning is:

ACTIVE_TRAIN:
METHOD: crb
AGGREGATION: mean
PRE_TRAIN_SAMPLE_NUMS: 100
PRE_TRAIN_EPOCH_NUMS: 40
TRAIN_RESUME: False
SELECT_NUMS: 100
SELECT_LABEL_EPOCH_INTERVAL: 40
TOTAL_BUDGET_NUMS: 600
ACTIVE_CONFIG:
K1: 5
K2: 3
BANDWIDTH: 5
CLUSTERING: kmeans++

The pre-train is finished without any error, but after this an error occured before the train start:

2023-11-16 02:15:08,630 INFO Start training active-kitti_models/pv_rcnn_active_crb(select-100)
2023-11-16 02:15:08,631 INFO ***** Start Active Pre-train *****
train: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [02:13<00:00, 1.33s/it, total_it=4000]
epochs: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 40/40 [1:28:31<00:00, 132.79s/it, loss=1.66, lr=0.00252]
2023-11-16 03:43:40,420 INFO ***** Complete Active Pre-train *****
2023-11-16 03:43:40,421 INFO ***** Start Active Train Loop *****
epochs: 0%| | 0/240 [00:00<?, ?it/s*
found and enabled 3 Dropout layers for random sampling* | 0/3612 [00:00<?, ?it/s]
evaluating_unlabelled_set_epoch_40: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3612/3612 [12:53<00:00, 4.67it/s]
inf_grads_unlabelled_set_epoch_40: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 500/500 [10:11<00:00, 1.22s/it]
--- {kmeans++} running time: 54.5201575756073 seconds for fc grads---████████████████████████████████████████████████████████████████████████████████████████| 500/500 [10:11<00:00, 1.18s/it]
--- Build the uniform distribution running time: 0.04799032211303711 seconds ---
global_density_div_for_epoch_40: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [04:52<00:00, 2.92s/it]
successfully saved selected frames for epoch 40 for rank 0███████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [04:52<00:00, 3.49s/it]
wandb: WARNING Step must only increase in log calls. Step 4000 < 4001; dropping {'active_selection/num_bbox_Car': tensor(113, device='cuda:0')}.
wandb: WARNING Step must only increase in log calls. Step 4000 < 4001; dropping {'active_selection/mean_points_Car': tensor(141.4796)}.
wandb: WARNING Step must only increase in log calls. Step 4000 < 4001; dropping {'active_selection/median_points_Car': tensor(99.4700)}.
wandb: WARNING Step must only increase in log calls. Step 4000 < 4001; dropping {'active_selection/variance_points_Car': tensor(22668.2227)}.
wandb: WARNING Step must only increase in log calls. Step 4000 < 4001; dropping {'active_selection/num_bbox_Pedestrian': tensor(119, device='cuda:0')}.
wandb: WARNING Step must only increase in log calls. Step 4000 < 4001; dropping {'active_selection/mean_points_Pedestrian': tensor(171.6909)}.
wandb: WARNING Step must only increase in log calls. Step 4000 < 4001; dropping {'active_selection/median_points_Pedestrian': tensor(163.9500)}.
wandb: WARNING Step must only increase in log calls. Step 4000 < 4001; dropping {'active_selection/variance_points_Pedestrian': tensor(2096.7180)}.
wandb: WARNING Step must only increase in log calls. Step 4000 < 4001; dropping {'active_selection/num_bbox_Cyclist': tensor(46, device='cuda:0')}.
wandb: WARNING Step must only increase in log calls. Step 4000 < 4001; dropping {'active_selection/mean_points_Cyclist': tensor(24.0983)}.
wandb: WARNING Step must only increase in log calls. Step 4000 < 4001; dropping {'active_selection/median_points_Cyclist': tensor(20.5300)}.
wandb: WARNING Step must only increase in log calls. Step 4000 < 4001; dropping {'active_selection/variance_points_Cyclist': tensor(989.8525)}.
wandb: WARNING Step must only increase in log calls. Step 4000 < 4001; dropping {'active_selection/total_bbox_selected': tensor(278, device='cuda:0')}.
Batch_size of a single loader: 1
2023-11-16 04:12:39,231 INFO Database filter by min points Car: 14357 => 13532
2023-11-16 04:12:39,233 INFO Database filter by min points Pedestrian: 2207 => 2168
2023-11-16 04:12:39,233 INFO Database filter by min points Cyclist: 734 => 705
2023-11-16 04:12:39,286 INFO Database filter by difficulty Car: 13532 => 10759
2023-11-16 04:12:39,295 INFO Database filter by difficulty Pedestrian: 2168 => 2075
2023-11-16 04:12:39,298 INFO Database filter by difficulty Cyclist: 705 => 581
2023-11-16 04:12:39,313 INFO Loading KITTI dataset
2023-11-16 04:12:39,547 INFO Total samples for KITTI dataset: 3712
2023-11-16 04:12:39,812 INFO Database filter by min points Car: 14357 => 13532
2023-11-16 04:12:39,814 INFO Database filter by min points Pedestrian: 2207 => 2168
2023-11-16 04:12:39,814 INFO Database filter by min points Cyclist: 734 => 705
2023-11-16 04:12:39,852 INFO Database filter by difficulty Car: 13532 => 10759
2023-11-16 04:12:39,295 INFO Database filter by difficulty Pedestrian: 2168 => 2075
2023-11-16 04:12:39,298 INFO Database filter by difficulty Cyclist: 705 => 581
2023-11-16 04:12:39,313 INFO Loading KITTI dataset
2023-11-16 04:12:39,547 INFO Total samples for KITTI dataset: 3712
2023-11-16 04:12:39,812 INFO Database filter by min points Car: 14357 => 13532
2023-11-16 04:12:39,814 INFO Database filter by min points Pedestrian: 2207 => 2168
2023-11-16 04:12:39,814 INFO Database filter by min points Cyclist: 734 => 705
2023-11-16 04:12:39,852 INFO Database filter by difficulty Car: 13532 => 10759
2023-11-16 04:12:39,860 INFO Database filter by difficulty Pedestrian: 2168 => 2075
2023-11-16 04:12:39,862 INFO Database filter by difficulty Cyclist: 705 => 581
2023-11-16 04:12:39,873 INFO Loading KITTI dataset
2023-11-16 04:12:40,086 INFO Total samples for KITTI dataset: 3712
2023-11-16 04:12:40,088 INFO Loading KITTI dataset
2023-11-16 04:12:40,342 INFO Total samples for KITTI dataset: 3769
2023-11-16 04:12:40,493 INFO **finished selection: reload init weights of the model
epochs: 0%| | 0
/240 [29:00<?, ?it/s]
Traceback (most recent call last):
File "train.py", line 313, in
main()
File "train.py", line 255, in main
dist_train=dist_train
File "/home/CRB-active-3Ddet/tools/train_utils/train_active_utils.py", line 322, in train_model_active
model.load_state_dict(backbone_init_ckpt, strict=cfg.ACTIVE_TRAIN.METHOD!='llal') # strict=cfg.ACTIVE_TRAIN.METHOD!='llal'
File "/opt/conda/envs/crb/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1483, in load_state_dict
self.class.name, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for PVRCNN:
Missing key(s) in state_dict: "backbone_3d.conv_input.0.weight", "backbone_3d.conv_input.1.weight", "backbone_3d.conv_input.1.bias", "backbone_3d.conv_input.1.running_mean", "backbone_3d.conv_input.1.running_var", "backbon
e_3d.conv1.0.0.weight", "backbone_3d.conv1.0.1.weight", "backbone_3d.conv1.0.1.bias", "backbone_3d.conv1.0.1.running_mean", "backbone_3d.conv1.0.1.running_var", "backbone_3d.conv2.0.0.weight", "backbone_3d.conv2.0.1.weight", "back
bone_3d.conv2.0.1.bias", "backbone_3d.conv2.0.1.running_mean", "backbone_3d.conv2.0.1.running_var", "backbone_3d.conv2.1.0.weight", "backbone_3d.conv2.1.1.weight", "backbone_3d.conv2.1.1.bias", "backbone_3d.conv2.1.1.running_mean"
, "backbone_3d.conv2.1.1.running_var", "backbone_3d.conv2.2.0.weight", "backbone_3d.conv2.2.1.weight", "backbone_3d.conv2.2.1.bias", "backbone_3d.conv2.2.1.running_mean", "backbone_3d.conv2.2.1.running_var", "backbone_3d.conv3.0.0
.weight", "backbone_3d.conv3.0.1.weight", "backbone_3d.conv3.0.1.bias", "backbone_3d.conv3.0.1.running_mean", "backbone_3d.conv3.0.1.running_var", "backbone_3d.conv3.1.0.weight", "backbone_3d.conv3.1.1.weight", "backbone_3d.conv3.
1.1.bias", "backbone_3d.conv3.1.1.running_mean", "backbone_3d.conv3.1.1.running_var", "backbone_3d.conv3.2.0.weight", "backbone_3d.conv3.2.1.weight", "backbone_3d.conv3.2.1.bias", "backbone_3d.conv3.2.1.running_mean", "backbone_3d
.conv3.2.1.running_var", "backbone_3d.conv4.0.0.weight", "backbone_3d.conv4.0.1.weight", "backbone_3d.conv4.0.1.bias", "backbone_3d.conv4.0.1.running_mean", "backbone_3d.conv4.0.1.running_var", "backbone_3d.conv4.1.0.weight", "bac
kbone_3d.conv4.1.1.weight", "backbone_3d.conv4.1.1.bias", "backbone_3d.conv4.1.1.running_mean", "backbone_3d.conv4.1.1.running_var", "backbone_3d.conv4.2.0.weight", "backbone_3d.conv4.2.1.weight", "backbone_3d.conv4.2.1.bias", "ba
ckbone_3d.conv4.2.1.running_mean", "backbone_3d.conv4.2.1.running_var", "backbone_3d.conv_out.0.weight", "backbone_3d.conv_out.1.weight", "backbone_3d.conv_out.1.bias", "backbone_3d.conv_out.1.running_mean", "backbone_3d.conv_out.
1.running_var", "pfe.SA_layers.0.mlps.0.0.weight", "pfe.SA_layers.0.mlps.0.1.weight", "pfe.SA_layers.0.mlps.0.1.bias", "pfe.SA_layers.0.mlps.0.1.running_mean", "pfe.SA_layers.0.mlps.0.1.running_var", "pfe.SA_layers.0.mlps.0.3.weig
ht", "pfe.SA_layers.0.mlps.0.4.weight", "pfe.SA_layers.0.mlps.0.4.bias", "pfe.SA_layers.0.mlps.0.4.running_mean", "pfe.SA_layers.0.mlps.0.4.running_var", "pfe.SA_layers.0.mlps.1.0.weight", "pfe.SA_layers.0.mlps.1.1.weight", "pfe.S
A_layers.0.mlps.1.1.bias", "pfe.SA_layers.0.mlps.1.1.running_mean", "pfe.SA_layers.0.mlps.1.1.running_var", "pfe.SA_layers.0.mlps.1.3.weight", "pfe.SA_layers.0.mlps.1.4.weight", "pfe.SA_layers.0.mlps.1.4.bias", "pfe.SA_layers.0.ml
ps.1.4.running_mean", "pfe.SA_layers.0.mlps.1.4.running_var", "pfe.SA_layers.1.mlps.0.0.weight", "pfe.SA_layers.1.mlps.0.1.weight", "pfe.SA_layers.1.mlps.0.1.bias", "pfe.SA_layers.1.mlps.0.1.running_mean", "pfe.SA_layers.1.mlps.0.
1.running_var", "pfe.SA_layers.1.mlps.0.3.weight", "pfe.SA_layers.1.mlps.0.4.weight", "pfe.SA_layers.1.mlps.0.4.bias", "pfe.SA_layers.1.mlps.0.4.running_mean", "pfe.SA_layers.1.mlps.0.4.running_var", "pfe.SA_layers.1.mlps.1.0.weig
ht", "pfe.SA_layers.1.mlps.1.1.weight", "pfe.SA_layers.1.mlps.1.1.bias", "pfe.SA_layers.1.mlps.1.1.running_mean", "pfe.SA_layers.1.mlps.1.1.running_var", "pfe.SA_layers.1.mlps.1.3.weight", "pfe.SA_layers.1.mlps.1.4.weight", "pfe.S
A_layers.1.mlps.1.4.bias", "pfe.SA_layers.1.mlps.1.4.running_mean", "pfe.SA_layers.1.mlps.1.4.running_var", "pfe.SA_layers.2.mlps.0.0.weight", "pfe.SA_layers.2.mlps.0.1.weight", "pfe.SA_layers.2.mlps.0.1.bias", "pfe.SA_layers.2.ml
ps.0.1.running_mean", "pfe.SA_layers.2.mlps.0.1.running_var", "pfe.SA_layers.2.mlps.0.3.weight", "pfe.SA_layers.2.mlps.0.4.weight", "pfe.SA_layers.2.mlps.0.4.bias", "pfe.SA_layers.2.mlps.0.4.running_mean", "pfe.SA_layers.2.mlps.0.
4.running_var", "pfe.SA_layers.2.mlps.1.0.weight", "pfe.SA_layers.2.mlps.1.1.weight", "pfe.SA_layers.2.mlps.1.1.bias", "pfe.SA_layers.2.mlps.1.1.running_mean", "pfe.SA_layers.2.mlps.1.1.running_var", "pfe.SA_layers.2.mlps.1.3.weig
ht", "pfe.SA_layers.2.mlps.1.4.weight", "pfe.SA_layers.2.mlps.1.4.bias", "pfe.SA_layers.2.mlps.1.4.running_mean", "pfe.SA_layers.2.mlps.1.4.running_var", "pfe.SA_layers.3.mlps.0.0.weight", "pfe.SA_layers.3.mlps.0.1.weight", "pfe.S
A_layers.3.mlps.0.1.bias", "pfe.SA_layers.3.mlps.0.1.running_mean", "pfe.SA_layers.3.mlps.0.1.running_var", "pfe.SA_layers.3.mlps.0.3.weight", "pfe.SA_layers.3.mlps.0.4.weight", "pfe.SA_layers.3.mlps.0.4.bias", "pfe.SA_layers.3.ml
ps.0.4.running_mean", "pfe.SA_layers.3.mlps.0.4.running_var", "pfe.SA_layers.3.mlps.1.0.weight", "pfe.SA_layers.3.mlps.1.1.weight", "pfe.SA_layers.3.mlps.1.1.bias", "pfe.SA_layers.3.mlps.1.1.running_mean", "pfe.SA_layers.3.mlps.1.
1.running_var", "pfe.SA_layers.3.mlps.1.3.weight", "pfe.SA_layers.3.mlps.1.4.weight", "pfe.SA_layers.3.mlps.1.4.bias", "pfe.SA_layers.3.mlps.1.4.running_mean", "pfe.SA_layers.3.mlps.1.4.running_var", "pfe.SA_rawpoints.mlps.0.0.wei
ght", "pfe.SA_rawpoints.mlps.0.1.weight", "pfe.SA_rawpoints.mlps.0.1.bias", "pfe.SA_rawpoints.mlps.0.1.running_mean", "pfe.SA_rawpoints.mlps.0.1.running_var", "pfe.SA_rawpoints.mlps.0.3.weight", "pfe.SA_rawpoints.mlps.0.4.weight",
"pfe.SA_rawpoints.mlps.0.4.bias", "pfe.SA_rawpoints.mlps.0.4.running_mean", "pfe.SA_rawpoints.mlps.0.4.running_var", "pfe.SA_rawpoints.mlps.1.0.weight", "pfe.SA_rawpoints.mlps.1.1.weight", "pfe.SA_rawpoints.mlps.1.1.bias", "pfe.S
A_rawpoints.mlps.1.1.running_mean", "pfe.SA_rawpoints.mlps.1.1.running_var", "pfe.SA_rawpoints.mlps.1.3.weight", "pfe.SA_rawpoints.mlps.1.4.weight", "pfe.SA_rawpoints.mlps.1.4.bias", "pfe.SA_rawpoints.mlps.1.4.running_mean", "pfe.
SA_rawpoints.mlps.1.4.running_var", "pfe.vsa_point_feature_fusion.0.weight", "pfe.vsa_point_feature_fusion.1.weight", "pfe.vsa_point_feature_fusion.1.bias", "pfe.vsa_point_feature_fusion.1.running_mean", "pfe.vsa_point_feature_fus
ion.1.running_var", "backbone_2d.blocks.0.13.weight", "backbone_2d.blocks.0.14.weight", "backbone_2d.blocks.0.14.bias", "backbone_2d.blocks.0.14.running_mean", "backbone_2d.blocks.0.14.running_var", "backbone_2d.blocks.0.16.weight
", "backbone_2d.blocks.0.17.weight", "backbone_2d.blocks.0.17.bias", "backbone_2d.blocks.0.17.running_mean", "backbone_2d.blocks.0.17.running_var", "point_head.cls_layers.0.weight", "point_head.cls_layers.1.weight", "point_head.cl
s_layers.1.bias", "point_head.cls_layers.1.running_mean", "point_head.cls_layers.1.running_var", "point_head.cls_layers.3.weight", "point_head.cls_layers.4.weight", "point_head.cls_layers.4.bias", "point_head.cls_layers.4.running_
mean", "point_head.cls_layers.4.running_var", "point_head.cls_layers.6.weight", "point_head.cls_layers.6.bias", "roi_head.roi_grid_pool_layer.mlps.0.0.weight", "roi_head.roi_grid_pool_layer.mlps.0.1.weight", "roi_head.roi_grid_poo
l_layer.mlps.0.1.bias", "roi_head.roi_grid_pool_layer.mlps.0.1.running_mean", "roi_head.roi_grid_pool_layer.mlps.0.1.running_var", "roi_head.roi_grid_pool_layer.mlps.0.3.weight", "roi_head.roi_grid_pool_layer.mlps.0.4.weight", "ro
i_head.roi_grid_pool_layer.mlps.0.4.bias", "roi_head.roi_grid_pool_layer.mlps.0.4.running_mean", "roi_head.roi_grid_pool_layer.mlps.0.4.running_var", "roi_head.roi_grid_pool_layer.mlps.1.0.weight", "roi_head.roi_grid_pool_layer.ml
ps.1.1.weight", "roi_head.roi_grid_pool_layer.mlps.1.1.bias", "roi_head.roi_grid_pool_layer.mlps.1.1.running_mean", "roi_head.roi_grid_pool_layer.mlps.1.1.running_var", "roi_head.roi_grid_pool_layer.mlps.1.3.weight", "roi_head.roi
grid_pool_layer.mlps.1.4.weight", "roi_head.roi_grid_pool_layer.mlps.1.4.bias", "roi_head.roi_grid_pool_layer.mlps.1.4.running_mean", "roi_head.roi_grid_pool_layer.mlps.1.4.running_var", "roi_head.shared_fc_layer.0.weight", "roi
head.shared_fc_layer.1.weight", "roi_head.shared_fc_layer.1.bias", "roi_head.shared_fc_layer.1.running_mean", "roi_head.shared_fc_layer.1.running_var", "roi_head.shared_fc_layer.4.weight", "roi_head.shared_fc_layer.5.weight", "roi
head.shared_fc_layer.5.bias", "roi_head.shared_fc_layer.5.running_mean", "roi_head.shared_fc_layer.5.running_var", "roi_head.cls_layers.0.weight", "roi_head.cls_layers.1.weight", "roi_head.cls_layers.1.bias", "roi_head.cls_layers
.1.running_mean", "roi_head.cls_layers.1.running_var", "roi_head.cls_layers.4.weight", "roi_head.cls_layers.5.weight", "roi_head.cls_layers.5.bias", "roi_head.cls_layers.5.running_mean", "roi_head.cls_layers.5.running_var", "roi_h
ead.cls_layers.7.weight", "roi_head.cls_layers.7.bias", "roi_head.reg_layers.0.weight", "roi_head.reg_layers.1.weight", "roi_head.reg_layers.1.bias", "roi_head.reg_layers.1.running_mean", "roi_head.reg_layers.1.running_var", "roi
head.reg_layers.4.weight", "roi_head.reg_layers.5.weight", "roi_head.reg_layers.5.bias", "roi_head.reg_layers.5.running_mean", "roi_head.reg_layers.5.running_var", "roi_head.reg_layers.7.weight", "roi_head.reg_layers.7.bias".
Unexpected key(s) in state_dict: "vfe.pfn_layers.0.linear.weight", "vfe.pfn_layers.0.norm.weight", "vfe.pfn_layers.0.norm.bias", "vfe.pfn_layers.0.norm.running_mean", "vfe.pfn_layers.0.norm.running_var", "vfe.pfn_layers.0.
norm.num_batches_tracked", "backbone_2d.blocks.2.1.weight", "backbone_2d.blocks.2.2.weight", "backbone_2d.blocks.2.2.bias", "backbone_2d.blocks.2.2.running_mean", "backbone_2d.blocks.2.2.running_var", "backbone_2d.blocks.2.2.num_b
atches_tracked", "backbone_2d.blocks.2.4.weight", "backbone_2d.blocks.2.5.weight", "backbone_2d.blocks.2.5.bias", "backbone_2d.blocks.2.5.running_mean", "backbone_2d.blocks.2.5.running_var", "backbone_2d.blocks.2.5.num_batches_tra
cked", "backbone_2d.blocks.2.7.weight", "backbone_2d.blocks.2.8.weight", "backbone_2d.blocks.2.8.bias", "backbone_2d.blocks.2.8.running_mean", "backbone_2d.blocks.2.8.running_var", "backbone_2d.blocks.2.8.num_batches_tracked", "ba
ckbone_2d.blocks.2.10.weight", "backbone_2d.blocks.2.11.weight", "backbone_2d.blocks.2.11.bias", "backbone_2d.blocks.2.11.running_mean", "backbone_2d.blocks.2.11.running_var", "backbone_2d.blocks.2.11.num_batches_tracked", "backbo
ne_2d.blocks.2.13.weight", "backbone_2d.blocks.2.14.weight", "backbone_2d.blocks.2.14.bias", "backbone_2d.blocks.2.14.running_mean", "backbone_2d.blocks.2.14.running_var", "backbone_2d.blocks.2.14.num_batches_tracked", "backbone_2
d.blocks.2.16.weight", "backbone_2d.blocks.2.17.weight", "backbone_2d.blocks.2.17.bias", "backbone_2d.blocks.2.17.running_mean", "backbone_2d.blocks.2.17.running_var", "backbone_2d.blocks.2.17.num_batches_tracked", "backbone_2d.de
blocks.2.0.weight", "backbone_2d.deblocks.2.1.weight", "backbone_2d.deblocks.2.1.bias", "backbone_2d.deblocks.2.1.running_mean", "backbone_2d.deblocks.2.1.running_var", "backbone_2d.deblocks.2.1.num_batches_tracked".

size mismatch for backbone_2d.blocks.0.1.weight: copying a param with shape torch.Size([64, 64, 3, 3]) from checkpoint, the shape in current model is torch.Size([128, 256, 3, 3]).
size mismatch for backbone_2d.blocks.0.2.weight: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone_2d.blocks.0.2.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone_2d.blocks.0.2.running_mean: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone_2d.blocks.0.2.running_var: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone_2d.blocks.0.4.weight: copying a param with shape torch.Size([64, 64, 3, 3]) from checkpoint, the shape in current model is torch.Size([128, 128, 3, 3]).
size mismatch for backbone_2d.blocks.0.5.weight: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone_2d.blocks.0.5.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone_2d.blocks.0.5.running_mean: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone_2d.blocks.0.5.running_var: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone_2d.blocks.0.7.weight: copying a param with shape torch.Size([64, 64, 3, 3]) from checkpoint, the shape in current model is torch.Size([128, 128, 3, 3]).
size mismatch for backbone_2d.blocks.0.8.weight: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone_2d.blocks.0.8.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone_2d.blocks.0.8.running_mean: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone_2d.blocks.0.8.running_var: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone_2d.blocks.0.10.weight: copying a param with shape torch.Size([64, 64, 3, 3]) from checkpoint, the shape in current model is torch.Size([128, 128, 3, 3]).
size mismatch for backbone_2d.blocks.0.11.weight: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone_2d.blocks.0.11.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone_2d.blocks.0.11.running_mean: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone_2d.blocks.0.11.running_var: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for backbone_2d.blocks.1.1.weight: copying a param with shape torch.Size([128, 64, 3, 3]) from checkpoint, the shape in current model is torch.Size([256, 128, 3, 3]).
size mismatch for backbone_2d.blocks.1.2.weight: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone_2d.blocks.1.2.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone_2d.blocks.1.2.running_mean: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone_2d.blocks.1.2.running_var: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone_2d.blocks.1.4.weight: copying a param with shape torch.Size([128, 128, 3, 3]) from checkpoint, the shape in current model is torch.Size([256, 256, 3, 3]).
size mismatch for backbone_2d.blocks.1.5.weight: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone_2d.blocks.1.5.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone_2d.blocks.1.5.running_mean: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone_2d.blocks.1.5.running_var: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone_2d.blocks.1.7.weight: copying a param with shape torch.Size([128, 128, 3, 3]) from checkpoint, the shape in current model is torch.Size([256, 256, 3, 3]).
size mismatch for backbone_2d.blocks.1.8.weight: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone_2d.blocks.1.8.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone_2d.blocks.1.8.running_mean: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone_2d.blocks.1.8.running_var: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone_2d.blocks.1.10.weight: copying a param with shape torch.Size([128, 128, 3, 3]) from checkpoint, the shape in current model is torch.Size([256, 256, 3, 3]).
size mismatch for backbone_2d.blocks.1.11.weight: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone_2d.blocks.1.11.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone_2d.blocks.1.11.running_mean: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone_2d.blocks.1.11.running_var: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone_2d.blocks.1.13.weight: copying a param with shape torch.Size([128, 128, 3, 3]) from checkpoint, the shape in current model is torch.Size([256, 256, 3, 3]).
size mismatch for backbone_2d.blocks.1.14.weight: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone_2d.blocks.1.14.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone_2d.blocks.1.14.running_mean: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone_2d.blocks.1.14.running_var: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone_2d.blocks.1.16.weight: copying a param with shape torch.Size([128, 128, 3, 3]) from checkpoint, the shape in current model is torch.Size([256, 256, 3, 3]).
size mismatch for backbone_2d.blocks.1.17.weight: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone_2d.blocks.1.17.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone_2d.blocks.1.17.running_mean: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone_2d.blocks.1.17.running_var: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone_2d.deblocks.0.0.weight: copying a param with shape torch.Size([64, 128, 1, 1]) from checkpoint, the shape in current model is torch.Size([128, 256, 1, 1]).
size mismatch for backbone_2d.deblocks.0.1.weight: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone_2d.deblocks.0.1.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone_2d.deblocks.0.1.running_mean: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone_2d.deblocks.0.1.running_var: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone_2d.deblocks.1.0.weight: copying a param with shape torch.Size([128, 128, 2, 2]) from checkpoint, the shape in current model is torch.Size([256, 256, 2, 2]).
size mismatch for backbone_2d.deblocks.1.1.weight: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone_2d.deblocks.1.1.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone_2d.deblocks.1.1.running_mean: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for backbone_2d.deblocks.1.1.running_var: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for dense_head.conv_cls.weight: copying a param with shape torch.Size([18, 384, 1, 1]) from checkpoint, the shape in current model is torch.Size([18, 512, 1, 1]).
size mismatch for dense_head.conv_box.weight: copying a param with shape torch.Size([42, 384, 1, 1]) from checkpoint, the shape in current model is torch.Size([42, 512, 1, 1]).
size mismatch for dense_head.conv_dir_cls.weight: copying a param with shape torch.Size([12, 384, 1, 1]) from checkpoint, the shape in current model is torch.Size([12, 512, 1, 1]).

It seems that the size of some part of the model goes wrong, while i didn't change anything about the model structure, could you please help find out the solution?

Impact of using different models for AL cycles and final training

Thanks for the great work, we are able to reproduce your results on KITTI using PV-RCNN, it is impressive.

I have a few questions.

1. If I use PV-RCNN as the backbone during the AL cycles to select a final pool of data and send for oracle labeling. After that, use the selected data pool to train a totally different model. Will the effect of AL still works in this case?
2. In the paper, you have been using 6 AL cycles for benchmarking, I'm curious why 6 is selected as the number of cycle? If I only use 1 cycle to save the training time, what is the impact?

Hope you can provide some insights, thanks in advanced.

Implementation of Monte Carlo Forward Passes

Dear Authors,

thanks for providing this nice framework.

Unfortunately, the implementation of the 5 Monte Carlo forward passes - mentioned in the paper - confuses me.

In montecarlo_sampling.py, badge_sampling.py and crb_sampling.py there is no indication that a sample is forwarded to the model 5 (or multiple) times. After checking montecarlo_sampling.py in detail it also does not seem to be present in the dataloader or other locations.

CRB-active-3Ddet/pcdet/query_strategies/montecarlo_sampling.py

Line 45 in 9af7e97

pred_dicts, _ = self.model(unlabelled_batch)

Could you elaborate where the multiple forward passes take place?

Additionally, I am wondering why Badge is using Monte Carlo dropout at all, I double checked my knowledge about Badge and it is not mentioning it in the paper or using Monto Carlo dropout in their implementation. I could not find it in your paper. Is there a reason for including it, contrasting to the original paper.

https://openreview.net/pdf?id=ryghZJBKPS
https://github.com/JordanAsh/badge/blob/master/query_strategies/badge_sampling.py

Thanks in advance.

problem with cumm and spconv. installed spconv-cu113==2.1.21 and cumm-cu113 but getting this error.

from pcdet.models import build_network, model_fn_decorator, load_data_to_gpu

File "/ahmed/pcdet/models/init.py", line 6, in
from .detectors import build_detector
File "/ahmed/pcdet/models/detectors/init.py", line 1, in
from .detector3d_template import Detector3DTemplate
File "/ahmed/pcdet/models/detectors/detector3d_template.py", line 7, in
from ...utils.spconv_utils import find_all_spconv_keys
File "/ahmed/pcdet/utils/spconv_utils.py", line 8, in
import spconv.pytorch as spconv
File "/opt/conda/envs/ahmed/lib/python3.7/site-packages/spconv/init.py", line 17, in
from .core import ConvAlgo, AlgoHint
File "/opt/conda/envs/ahmed/lib/python3.7/site-packages/spconv/core.py", line 18, in
from cumm.gemm.algospec.core import TensorOpParams
ImportError: cannot import name 'TensorOpParams' from 'cumm.gemm.algospec.core' (/opt/conda/envs/ahmed/lib/python3.7/site-packages/cumm/gemm/algospec/core.py)

Python 3.7.16

spconv-cu113 2.1.21

cumm-cu113 0.4.9

PLEASE HELP ME! Thanks!

【Reproduce Experiment】How to reproduce experimental results in the paper

I use your default yaml file to train different strategy models, but the results presented are far from the paper experiment.
What should I do to have a similar result as in the paper?

I only set USE_ROAD_PLANE to False in yaml

The following are the results of my experiment:

Thanks

Is it necessary to train a backbone before I train the model?

I wonder if I must train a backbone by runing:

sh scripts/${DATASET}/train_${DATASET}_backbone.sh

before I run the train command:

python train.py --cfg_file ${CONFIG_FILE}

Or, can I just run the second command directly without running the train backbone command at first?

Where to find Supplementary Material?

Appreciating your work very much. But the SM seems to not be available on the arXiv, and I can't find it on the ICLR 2023 website probably for the reason that it has not been set public. Could you tell me where can I get access to the SM of your paper?

ONE-STAGE detector cfg

Thanks for sharing such an excellent work. Currently, there is only the cfg for PVRCNN. Is it possible to also share the configuration of One-stage methods? I tried to configure one, but it seems the current version of crb_sampling.py is not directly appliacable to the one-stage methods since it have a different loss function structure. Thx!

Clarification Needed on Which KITTI Dataset to Download

Hello,

Firstly, I would like to thank you for your efforts in developing and maintaining CRB-active-3Ddet. It is my first time working on this area.

I am currently trying to set up the environment and download the necessary datasets. I noticed in the documentation/readme, it's mentioned to "download the official KITTI 3D object detection dataset and organize the downloaded files as follows." However, upon visiting the official KITTI website, I found that there are multiple subsets available.

Could you please specify which particular subset(s) of the KITTI dataset are required for this project? This information would be greatly beneficial for ensuring I have the correct data for running and utilizing the project effectively.

And if I downloaded the dataset and orgnized in the way you mentioned is it not need to run the following command: python -m pcdet.datasets.kitti.kitti_dataset create_kitti_infos tools/cfgs/dataset_configs/kitti_dataset.yaml

Thank you in advance for your assistance and for the great work on this project!

Best regards,
Ruiyu

different errors occured in crb_sampling.py with both single gpu and multi-gpus

Hi,
Could you please help solve an error?
here is my ACTIVE_TRAIN in pv_rcnn_active_crb.yaml config:

ACTIVE_TRAIN:
    METHOD: crb
    AGGREGATION: mean

    PRE_TRAIN_SAMPLE_NUMS: 100  
    PRE_TRAIN_EPOCH_NUMS: 1
    TRAIN_RESUME: False

    SELECT_NUMS: 100
    SELECT_LABEL_EPOCH_INTERVAL: 1

    TOTAL_BUDGET_NUMS: 200

    ACTIVE_CONFIG:
        K1: 5
        K2: 3
        BANDWIDTH: 5
        CLUSTERING: kmeans++

i run python3 ./train.py --cfg_file cfgs/active-kitti_models/pv_rcnn_active_crb.yaml --batch_size 4 --fix_random_seed --max_ckpt_save_num 200 --ckpt_save_interval 1 and got following error:

Traceback (most recent call last):
  File "./train.py", line 314, in <module>
    main()
  File "./train.py", line 233, in main
    train_func(
  File "/*/CRB-active-3Ddet/tools/train_utils/train_active_utils.py", line 305, in train_model_active
    = active_training_utils.select_active_labels(
  File "/*/CRB-active-3Ddet/tools/../pcdet/utils/active_training_utils.py", line 271, in select_active_labels
    selected_frames = strategy.query(leave_pbar, cur_epoch)
  File "/*/CRB-active-3Ddet/tools/../pcdet/query_strategies/crb_sampling.py", line 264, in query
    x_axis = [np.linspace(-50, int(global_density_max[i])+50, 400) for i in range(num_class)]
  File "/*/CRB-active-3Ddet/tools/../pcdet/query_strategies/crb_sampling.py", line 264, in <listcomp>
    x_axis = [np.linspace(-50, int(global_density_max[i])+50, 400) for i in range(num_class)]
IndexError: list index out of range

I have checked that the variable global_density_max is an empty tensor.

total log is:

...(skip model info)...
2023-02-08 03:57:51,634   INFO  **********************Start training active-kitti_models/pv_rcnn_active_crb(select-100)**********************
2023-02-08 03:57:51,635   INFO  ***** Start Active Pre-train *****
train: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 25/25 [00:47<00:00,  1.92s/it, total_it=25]
epochs: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:49<00:00, 49.74s/it, loss=5.73, lr=0.000432]
2023-02-08 03:58:41,381   INFO  ***** Complete Active Pre-train *****
2023-02-08 03:58:41,381   INFO  ***** Start Active Train Loop *****
epochs:   0%|                                                                                                                                      | 0/2 [00:00<?, ?it/s**found and enabled 3 Dropout layers for random sampling**                                                                                        | 0/903 [00:00<?, ?it/s]
evaluating_unlabelled_set_epoch_1: 100%|███████████████████████████████████████████████████████████████████████████████████████████████| 903/903 [05:16<00:00,  2.85it/s]
inf_grads_unlabelled_set_epoch_1: 100%|████████████████████████████████████████████████████████████████████████████████████████████████| 500/500 [03:43<00:00,  2.24it/s]
--- {kmeans++} running time: 31.196898698806763 seconds for fc grads---████████████████████████████████████████████████████████████████| 500/500 [03:43<00:00,  2.21it/s]
epochs:   0%|                                                                                                                                      | 0/2 [09:32<?, ?it/s]
Traceback (most recent call last):
  File "./train.py", line 314, in <module>
    main()
  File "./train.py", line 233, in main
    train_func(
  File "/*/CRB-active-3Ddet/tools/train_utils/train_active_utils.py", line 305, in train_model_active
    = active_training_utils.select_active_labels(
  File "/*/CRB-active-3Ddet/tools/../pcdet/utils/active_training_utils.py", line 271, in select_active_labels
    selected_frames = strategy.query(leave_pbar, cur_epoch)
  File "*/CRB-active-3Ddet/tools/../pcdet/query_strategies/crb_sampling.py", line 264, in query
    x_axis = [np.linspace(-50, int(global_density_max[i])+50, 400) for i in range(num_class)]
  File "/*/CRB-active-3Ddet/tools/../pcdet/query_strategies/crb_sampling.py", line 264, in <listcomp>
    x_axis = [np.linspace(-50, int(global_density_max[i])+50, 400) for i in range(num_class)]
IndexError: list index out of range

Actually, I got a different error when i run with multi-gpus with the same config:

2023-02-08 08:38:48,753   INFO  **********************Start training active-kitti_models/pv_rcnn_active_crb(select-100)**********************
2023-02-08 08:38:48,753   INFO  ***** Start Active Pre-train *****
train: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:11<00:00,  2.93s/it, total_it=4]
epochs: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:46<00:00, 46.46s/it, loss=118, lr=0.0075]
2023-02-08 08:39:35,218   INFO  ***** Complete Active Pre-train *****
2023-02-08 08:39:35,218   INFO  ***** Start Active Train Loop *****
epochs:   0%|                                                                                                                                      | 0/2 [00:00<?, ?it/s**found and enabled 3 Dropout layers for random sampling**                                                                                        | 0/113 [00:00<?, ?it/s]
**found and enabled 3 Dropout layers for random sampling**
**found and enabled 3 Dropout layers for random sampling**
**found and enabled 3 Dropout layers for random sampling**
**found and enabled 3 Dropout layers for random sampling**
**found and enabled 3 Dropout layers for random sampling**
**found and enabled 3 Dropout layers for random sampling**
**found and enabled 3 Dropout layers for random sampling**
evaluating_unlabelled_set_epoch_1: 100%|███████████████████████████████████████████████████████████████████████████████████████████████| 113/113 [00:41<00:00,  2.74it/s]
Traceback (most recent call last):                                                                                                                                       
  File "train.py", line 314, in <module>                                                                                                 | 1/452 [00:02<15:24,  2.05s/it]
    main()
  File "train.py", line 233, in main
    train_func(
  File "/`/CRB-active-3Ddet/tools/train_utils/train_active_utils.py", line 305, in train_model_active
Traceback (most recent call last):                                                                                                                                       
  File "train.py", line 314, in <module>
    main()
  File "train.py", line 233, in main
    = active_training_utils.select_active_labels(
  File "/`/CRB-active-3Ddet/tools/../pcdet/utils/active_training_utils.py", line 271, in select_active_labels
    train_func(
  File "/`/CRB-active-3Ddet/tools/train_utils/train_active_utils.py", line 305, in train_model_active
epochs:   0%|                                                                                                                                      | 0/2 [01:19<?, ?it/s]
    selected_frames = strategy.query(leave_pbar, cur_epoch)
Traceback (most recent call last):
      File "train.py", line 314, in <module>
  File "`CRB-active-3Ddet/tools/../pcdet/query_strategies/crb_sampling.py", line 184, in query
= active_training_utils.select_active_labels(
  File "/`/CRB-active-3Ddet/tools/../pcdet/utils/active_training_utils.py", line 271, in select_active_labels
    selected_frames = strategy.query(leave_pbar, cur_epoch)
  File "/`/CRB-active-3Ddet/tools/../pcdet/query_strategies/crb_sampling.py", line 184, in query
    pred_dicts, _, _= self.model(unlabelled_batch)
  File "/opt/conda/envs/mining/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    main()
  File "train.py", line 233, in main
    result = self.forward(*input, **kwargs)
  File "/opt/conda/envs/mining/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 692, in forward
    pred_dicts, _, _= self.model(unlabelled_batch)
  File "/opt/conda/envs/mining/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    if self.reducer._rebuild_buckets():
RuntimeError: Expected to have finished reduction in the prior iteration before starting a new one. This error indicates that your module has parameters that were not used in producing loss. You can enable unused parameter detection by (1) passing the keyword argument `find_unused_parameters=True` to `torch.nn.parallel.DistributedDataParallel`; (2) making sure all `forward` function outputs participate in calculating loss. If you already have done the above two steps, then the distributed data parallel module wasn't able to locate the output tensors in the return value of your module's `forward` function. Please include the loss function and the structure of the return value of `forward` of your module when reporting this issue (e.g. list, dict, iterable).

Where can I see the list of frames used for pretrain?

Thanks for your excellent work and codes!
I notice that a small set DL is used to train the backbone.
I can find the data frames selected during active learning loops in folder active_label,
but I can't find a file that record the which data frames were used for backbone pretrain.

Questions about selecting data by boxes

Hello, your work is great! I'm trying to do some works on it. But I have some confusion about your paper and code.

1. How to select data for annoatation by a certrain number of boxes(like 200 boxes)? Is it implemented by just picking a certain number of point clouds(like 100 frames) and count the objects on them?
2. I am confusing about "requiring 1% and 8% annotations of bounding boxes and point clouds" in your paper. When I did some experiment on KITTI, I found that using 8% point clouds(300 frames) can get around 8% boxes.
3. In your code and IMPLEMENTATION DETAILS, you seem to use more data(random 100 + 6*100 CRB), which is much more than 1% and 8% annotations of bounding boxes and point clouds. I'm really confusing about it.
4. The final results vary a lot even using same query strategy(like training CRB with same parameters twice, one reult is only 79map, another one can even be close to 81), how do you solve it.

It would be much appreciated if you could answer my doubts!

montecarlo sampling, Badge Sampling and Confidence Sampling

Dear Authors,

Fisrt of all, Thanks for sharing your great work.

I am studying each active learning methods that you used with your codes.
I have questions for Montecarlo Sampling, Badge Sampling and Conidence Sampling.

I. Montecarlo Sampling
In 'montecarlo_sampling.py' code, pred_dicts is got at the 'eval' mode so it has postprocessing.
But pred_dicts from postprocessing at 'detector3d_template.py' doesn't have keys of rcnn_cls and rcnn_reg.
So 'KeyError' happens:

File "../pcdet/query_strategies/montecarlo_sampling.py", line 53, in query
cls_var = torch.var(torch.sigmoid(pred_dicts[0]['rcnn_cls']), 0).mean()
KeyError: 'rcnn_cls'

Would you let me know how I should change the line 53 and 54 at 'montecarlo_sampling.py?

II. Badge Sampling
When I review 'badge_sampling.py' in the debug mode, I found the order between self.unlabelled_loader and grad_loader are different.
For example,
the first batch's frame ids, unlabelled_batch['frame_id'], that I got are array(['003732', '003508', '004890', '000258', '002447', '006607'], dtype='<U6') from the line 75 with batch_size=6.
But, with the grad_loader, I got array(['003113'], dtype='<U6') first and then array(['001648'], dtype='<U6') from the line 130.
I think if the order of samples are different between two dataloaders, loss and gradients calculated might be wrong.
Am I right? or is there something I missed?

III. Confidence Sampling
In the line 40 of confidence_sampling.py, shape of pred_dicts[batch_inx]['confidence'] would be [number of NMS, 1].
So if we get softmax according to axis 1 in the line 41-43, I think aggregated_values always would be zero in line 48.
Am I right?

Thanks in advance.

unable to run one stage model using CRB

can you give some cfgs code which can help me running CRB with second model on the kitti dataset? thanks