dyzhang09/sam3d

[SCIS] SAM3D: Zero-Shot 3D Object Detection via Segment Anything Model

sam3d's Issues

demo.py

请问，有没有单帧点云的demo.py的推理文件呢？

代码调试问题

当我在终端运行：python tools/test.py projects/configs/sam3d_intensity_bev_waymo_car.py fake.pth
可以成功运行；

但是当我在pycharm中调试时，当代码调试到第121行时，
遇到如下错误：FileNotFoundError: class SAMDet3D in projects/core/sam_det3d.py: class MaskAutoGenerator in projects/core/mask_utils.py: [Errno 2] No such file or directory: 'projects/pretrain_weights/sam_vit_h_4b8939.pth'

针对FlieNotFoundError，实际上类SAMDet3D和MaskAutoGenerator就在上述路径脚本文件里，为什么会报文件没有发现这个错误呢？

数据集版本问题

请问，您的项目用的是waymo哪个版本的数据集？官网现在有好多个版本（如下图）

还有个问题：
您的项目是只用到了验证集吗？测试集和训练集都没有用到对吧？

Can waymo use the Nuscenes dataset instead?

Hi，Can waymo use the Nuscenes dataset instead?

Suggestion - Integrate MobileSAM into the pipeline for lightweight and faster inference

Reference: https://github.com/ChaoningZhang/MobileSAM

Our project performs on par with the original SAM and keeps exactly the same pipeline as the original SAM except for a change on the image encode, therefore, it is easy to Integrate into any project.

MobileSAM is around 60 times smaller and around 50 times faster than original SAM, and it is around 7 times smaller and around 5 times faster than the concurrent FastSAM. The comparison of the whole pipeline is summarzed as follows:

Best Wishes,

Qiao

Please check the typo of the paper.

Please check the typo of the paper.

Waymo data conversion get killed

Hi there,
Really appreciate the work that has been put on. Actually, i wanted help if possible! I am having the same issue https://github.com/open-mmlab/mmdetection3d/issues/2796 where after calling the ools/create_data.py waymo --root-path /workspace/data/waymo/ --out-dir /workspace/data/waymo --workers 1 --extra-tag waymo get killed while generating the info.pkl files.
I wanted to know did you faced the same issue and how it you overcame that.

I am really desperate to get this conversion done so that I can work on my model :(

Waymo validation issue

Hi there, thanks helping me out previously with the Waymo dataset creation. I did managed to create the dataset and have successfully tested the training for pointpiller model as an example, but in the evaluation phase i an get error KeyError: 'pred_instances_3d', in my evaluation configuration and its something like this

val_evaluator = dict(
    type='WaymoMetric',
    ann_file="data/waymo/kitti_format/waymo_infos_val.pkl",
    waymo_bin_file= "data/waymo/waymo_format/gt.bin",
    data_root='data/waymo/waymo_format',
    backend_args=backend_args,
    convert_kitti_format=True,
    idx2metainfo='data/waymo/waymo_format/idx2metainfo.pkl'
    )

I have tried to follow the instruction on this path

The error is out it
11/28 00:21:27 - mmengine - INFO - Epoch(test) [39900/39987] eta: 0:00:07 time: 0.0922 data_time: 0.0021 memory: 368
11/28 00:21:31 - mmengine - INFO - Epoch(test) [39950/39987] eta: 0:00:03 time: 0.0963 data_time: 0.0030 memory: 368

Converting prediction to KITTI format
[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 39987/39987, 111.9 task/s, elapsed: 357s, ETA:     0s
Result is saved to /tmp/tmpki56cozu/results/pred_instances_3d.pkl.
Start converting ...
[                                                  ] 0/39987, elapsed: 0s, ETA:Traceback (most recent call last):
  File "tools/test.py", line 151, in <module>
    main()
  File "tools/test.py", line 147, in main
    runner.test()
  File "/homesdatasets7/mmdet/lib/python3.8/site-packages/mmengine/runner/runner.py", line 1823, in test
    metrics = self.test_loop.run()  # type: ignore
  File "/homesdatasets7/mmdet/lib/python3.8/site-packages/mmengine/runner/loops.py", line 438, in run
    metrics = self.evaluator.evaluate(len(self.dataloader.dataset))
  File "/homesdatasets7/mmdet/lib/python3.8/site-packages/mmengine/evaluator/evaluator.py", line 79, in evaluate
    _results = metric.evaluate(size)
  File "/homesdatasets7/mmdet/lib/python3.8/site-packages/mmengine/evaluator/metric.py", line 133, in evaluate
    _metrics = self.compute_metrics(results)  # type: ignore
  File "/mmdetection3d/mmdet3d/evaluation/metrics/waymo_metric.py", line 182, in compute_metrics
    result_dict, tmp_dir = self.format_results(
  File "/mmdetection3d/mmdet3d/evaluation/metrics/waymo_metric.py", line 391, in format_results
    converter.convert()
  File "/mmdetection3d/mmdet3d/evaluation/functional/waymo_utils/prediction_to_waymo.py", line 368, in convert
    convert_func(i)
  File "/mmdetection3d/mmdet3d/evaluation/functional/waymo_utils/prediction_to_waymo.py", line 285, in convert_one_fast
    if len(self.results[res_index]['pred_instances_3d']) > 0:
KeyError: 'pred_instances_3d'

I have also tried removing the idx2metainfo and using the normal one

val_evaluator = dict(
    type='WaymoMetric',
    ann_file="data/waymo/kitti_format/waymo_infos_val.pkl",
    waymo_bin_file= "data/waymo/waymo_format/gt.bin",
    data_root='data/waymo/waymo_format',
    backend_args=backend_args,
    convert_kitti_format=False)

and i use to get the below erro

11/29 15:30:08 - mmengine - INFO - Epoch(test) [2350/2500]    eta: 0:04:16  time: 1.2338  data_time: 0.0519  memory: 12030  
11/29 15:31:10 - mmengine - INFO - Epoch(test) [2400/2500]    eta: 0:02:49  time: 1.2408  data_time: 0.0340  memory: 12138  
11/29 15:32:10 - mmengine - INFO - Epoch(test) [2450/2500]    eta: 0:01:24  time: 1.1888  data_time: 0.0437  memory: 12139  
11/29 15:33:19 - mmengine - INFO - Epoch(test) [2500/2500]    eta: 0:00:00  time: 1.3789  data_time: 0.0367  memory: 12162  
Traceback (most recent call last):
  File "tools/test.py", line 151, in <module>
    main()
  File "tools/test.py", line 147, in main
    runner.test()
  File "/python3.8/site-packages/mmengine/runner/runner.py", line 1823, in test
    metrics = self.test_loop.run()  # type: ignore
  File "/python3.8/site-packages/mmengine/runner/loops.py", line 438, in run
    metrics = self.evaluator.evaluate(len(self.dataloader.dataset))
  File "/python3.8/site-packages/mmengine/evaluator/evaluator.py", line 79, in evaluate
    _results = metric.evaluate(size)
  File "/python3.8/site-packages/mmengine/evaluator/metric.py", line 133, in evaluate
    _metrics = self.compute_metrics(results)  # type: ignore
  File "/mmdetection3d/mmdet3d/evaluation/metrics/waymo_metric.py", line 182, in compute_metrics
    result_dict, tmp_dir = self.format_results(
  File "/mmdetection3d/mmdet3d/evaluation/metrics/waymo_metric.py", line 381, in format_results
    converter = Prediction2Waymo(
  File "/mmdetection3d/mmdet3d/evaluation/functional/waymo_utils/prediction_to_waymo.py", line 108, in __init__
    self.get_file_names()
  File "/mmdetection3d/mmdet3d/evaluation/functional/waymo_utils/prediction_to_waymo.py", line 117, in get_file_names
    if 'path_mapping' in self.backend_args:
TypeError: argument of type 'NoneType' is not iterable

Did you encounter such errors while you where running your models ? Would really appreciate your help with this if possible!