aim-uofa / dyco3d Goto Github PK
View Code? Open in Web Editor NEWLicense: Other
License: Other
Hi, when I tried to run the inference code, I met a problem that the variable proposal_idx was empty. Could you give me any suggestion to debug the code? Thank you very much! The location of code and error information are listed below.
DyCo3D/model/pointgroup/pointgroup.py
Line 454 in 17d22c2
Traceback (most recent call last):
File "test_s3dis.py", line 276, in
test(model, model_fn, data_name, cfg.test_epoch)
File "test_s3dis.py", line 70, in test
preds = model_fn(batch, model, epoch)
File "/home/xiaodchen/Generalization/DyCo3D/model/pointgroup/pointgroup.py", line 613, in test_model_fn
ret = model(input_, p2v_map, coords_float, coords[:, 0].int(), batch_offsets, epoch, ins_sample_num=-1, training=False)
File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/xiaodchen/Generalization/DyCo3D/model/pointgroup/pointgroup.py", line 574, in forward
proposal_idx, proposal_len, scores = self.generate_proposal(mask_logits.squeeze(dim=0).sigmoid(), object_idxs,
File "/home/xiaodchen/Generalization/DyCo3D/model/pointgroup/pointgroup.py", line 454, in generate_proposal
proposal_idx = torch.cat(proposal_idx, dim=0)
RuntimeError: There were no tensor arguments to this function (e.g., you passed an empty list of Tensors), but no fallback function is registered for schema aten::_cat. This usually means that this function requires a non-empty list of Tensors. Available functions are [CPU, CUDA, QuantizedCPU, BackendSelect, Named, AutogradOther, AutogradCPU, AutogradCUDA, AutogradXLA, AutogradNestedTensor, UNKNOWN_TENSOR_TYPE_ID, AutogradPrivateUse1, AutogradPrivateUse2, AutogradPrivateUse3, Tracer, Autocast, Batched, VmapMode].
Thank you for your outstanding work!
Where can I find your supplementary materials? I want to learn about the difference between your clustering algorithm and pointgroup~
Thank you for sharing the code. When I re-trained the model on the training set of ScanNetv2 (masking head size of 16) with your default config then tested it on the validation set, the result is lower than the result of your provided pretrained model (see image below).
So I have some concerns about the implementation details:
Could you give me some advices? Thanks.
I met several problems when I tried to compile spconv.
List them below for someone encounters the same problem.
Environment:
sudo apt-get install libboost-filesystem-dev
sudo apt-get install libboost-dev
In this case, it seems that the
Note: I reference from here and here
Other issue when compiling pointgroup_ops :
apt-get install libsparsehash-dev
Update:
conda install pytorch==1.1.0 torchvision==0.3.0 cudatoolkit=10.0 -c pytorch
Then, recompile the spconv and pointgroup_ops.
Other issues solving reference 1
Other issues solving reference 2
pointgroup_ops/src/bfs_cluster.h includes <THC/THC.h> that leads to the failure of pointgroup_ops installation.
When I was validating on ScanNetv2, I encounterd the problem in the function generate_proposal().
[2021-07-27 11:48:44,183 INFO test.py line 44 41515] >>>>>>>>>>>>>>>> Start Evaluation >>>>>>>>>>>>>>>> [585/15021]
[2021-07-27 11:48:53,470 INFO scannetv2_inst.py line 108 41515] Testing samples (val): 312
/home/sysadmin/.fei/DyCo3D/model/pointgroup/pointgroup.py:432: UserWarning: This overload of nonzero is deprecated:
nonzero()
Consider using one of the following signatures instead:
nonzero(*, bool as_tuple) (Triggered internally at /opt/conda/conda-bld/pytorch_1603729047590/work/torch/csrc/utils/python_arg_parser.cpp:882.)
proposal_id_n = ((mask_logits[n] > threshold) + (seg_pred == inst_pred_seg_label[n].item()) > 1).nonzero().squeeze(dim=1)
Traceback (most recent call last):
File "test.py", line 265, in <module>
test(model, model_fn, data_name, cfg.test_epoch)
File "test.py", line 64, in test
preds = model_fn(batch, model, epoch)
File "/home/sysadmin/.fei/DyCo3D/model/pointgroup/pointgroup.py", line 605, in test_model_fn
ret = model(input_, p2v_map, coords_float, coords[:, 0].int(), batch_offsets, epoch, ins_sample_num=-1, training=False)
File "/home/sysadmin/.conda/envs/mm3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/sysadmin/.fei/DyCo3D/model/pointgroup/pointgroup.py", line 568, in forward
min_pts_num=50)
File "/home/sysadmin/.fei/DyCo3D/model/pointgroup/pointgroup.py", line 445, in generate_proposal
proposal_idx = torch.cat(proposal_idx, dim=0)
RuntimeError: There were no tensor arguments to this function (e.g., you passed an empty list of Tensors), but no fallback function is registered for schema aten::_cat. This usually means that this function requires a non-empty li
st of Tensors. Available functions are [CPU, CUDA, QuantizedCPU, BackendSelect, Named, AutogradOther, AutogradCPU, AutogradCUDA, AutogradXLA, AutogradPrivateUse1, AutogradPrivateUse2, AutogradPrivateUse3, Tracer, Autocast, Batched
, VmapMode].
The problem is in the for loop that all proposal_id_n is a tensor of zero. However I think my dataset is well processed.
for n in range(n_inst):
proposal_id_n = ((mask_logits[n] > threshold) + (seg_pred == inst_pred_seg_label[n].item()) > 1).nonzero().squeeze(dim=1)
score = mask_logits[n][proposal_id_n].mean()
seg_label = inst_pred_seg_label[n]
if proposal_id_n.size(0) < min_pts_num:
continue
proposal_id_n = batch_id[proposal_id_n.long()].unsqueeze(dim=1)
id_proposal_id_n = torch.cat([torch.ones_like(proposal_id_n)*num, proposal_id_n], dim=1)
num += 1
tmp = proposal_len[-1]
proposal_len.append(proposal_id_n.size(0)+tmp)
proposal_idx.append(id_proposal_id_n)
scores.append(score)
Could you give me some advice? Thanks.
I want to ask how to calculate the single AP as there is no dirct score. Is that use classification and the mask quality as SOLOv2? (I haven't read the code of CondIns)
Hello, in the article it said the model has been tested in the S3DIS dataset, could you add the S3DIS preprocessing and testing code?
The some error occurs when I execute the command
python setup.py bdist_wheel
I see README.md that
Add the $INCLUDE_PATH$ that contains boost in lib/spconv/CMakeLists.txt. (Not necessary if it could be found.)
But I don't understand boost?
running bdist_wheel
running build
running build_py
running build_ext
/home/ubuntu/de/dyco3d/lib/spconv/build/lib.linux-x86_64-3.7
Release
-- The CXX compiler identification is GNU 5.4.0
-- The CUDA compiler identification is unknown
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - failed
-- Check for working CXX compiler: /home/ubuntu/Downloads/yes/envs/dyco3d/bin/c++
-- Check for working CXX compiler: /home/ubuntu/Downloads/yes/envs/dyco3d/bin/c++ - broken
CMake Error at /home/ubuntu/Downloads/yes/envs/dyco3d/lib/python3.7/site-packages/cmake/data/share/cmake-3.22/Modules/CMakeTestCXXCompiler.cmake:62 (message):
The C++ compiler
`/home/ubuntu/Downloads/yes/envs/dyco3d/bin/c++`
is not able to compile a simple test program.
It fails with the following output:
Change Dir: /home/ubuntu/de/dyco3d/lib/spconv/build/temp.linux-x86_64-3.7/CMakeFiles/CMakeTmp
Run Build Command(s):/usr/bin/make -f Makefile cmTC_0ead2/fast && /usr/bin/make -f CMakeFiles/cmTC_0ead2.dir/build.make CMakeFiles/cmTC_0ead2.dir/build
make[1]: Entering directory '/home/ubuntu/de/dyco3d/lib/spconv/build/temp.linux-x86_64-3.7/CMakeFiles/CMakeTmp'
Building CXX object CMakeFiles/cmTC_0ead2.dir/testCXXCompiler.cxx.o
/home/ubuntu/Downloads/yes/envs/dyco3d/bin/c++ -DVERSION_INFO=\"1.0\" -o CMakeFiles/cmTC_0ead2.dir/testCXXCompiler.cxx.o -c /home/ubuntu/de/dyco3d/lib/spconv/build/temp.linux-x86_64-3.7/CMakeFiles/CMakeTmp/testCXXCompiler.cxx
Linking CXX executable cmTC_0ead2
/home/ubuntu/Downloads/yes/envs/dyco3d/lib/python3.7/site-packages/cmake/data/bin/cmake -E cmake_link_script CMakeFiles/cmTC_0ead2.dir/link.txt --verbose=1
/home/ubuntu/Downloads/yes/envs/dyco3d/bin/c++ -DVERSION_INFO=\"1.0\" CMakeFiles/cmTC_0ead2.dir/testCXXCompiler.cxx.o -o cmTC_0ead2
/usr/bin/ld: cannot find crt1.o: No such file or directory
/usr/bin/ld: cannot find crti.o: No such file or directory
collect2: error: ld returned 1 exit status
CMakeFiles/cmTC_0ead2.dir/build.make:98: recipe for target 'cmTC_0ead2' failed
make[1]: *** [cmTC_0ead2] Error 1
make[1]: Leaving directory '/home/ubuntu/de/dyco3d/lib/spconv/build/temp.linux-x86_64-3.7/CMakeFiles/CMakeTmp'
Makefile:127: recipe for target 'cmTC_0ead2/fast' failed
make: *** [cmTC_0ead2/fast] Error 2
CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
CMakeLists.txt:2 (project)
-- Configuring incomplete, errors occurred!
See also "/home/ubuntu/de/dyco3d/lib/spconv/build/temp.linux-x86_64-3.7/CMakeFiles/CMakeOutput.log".
See also "/home/ubuntu/de/dyco3d/lib/spconv/build/temp.linux-x86_64-3.7/CMakeFiles/CMakeError.log".
Traceback (most recent call last):
File "setup.py", line 88, in
zip_safe=False,
File "/home/ubuntu/Downloads/yes/envs/dyco3d/lib/python3.7/site-packages/setuptools/init.py", line 155, in setup
return distutils.core.setup(**attrs)
File "/home/ubuntu/Downloads/yes/envs/dyco3d/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 148, in setup
return run_commands(dist)
File "/home/ubuntu/Downloads/yes/envs/dyco3d/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 163, in run_commands
dist.run_commands()
File "/home/ubuntu/Downloads/yes/envs/dyco3d/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 967, in run_commands
self.run_command(cmd)
File "/home/ubuntu/Downloads/yes/envs/dyco3d/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
cmd_obj.run()
File "/home/ubuntu/Downloads/yes/envs/dyco3d/lib/python3.7/site-packages/wheel/bdist_wheel.py", line 299, in run
self.run_command('build')
File "/home/ubuntu/Downloads/yes/envs/dyco3d/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/home/ubuntu/Downloads/yes/envs/dyco3d/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
cmd_obj.run()
File "/home/ubuntu/Downloads/yes/envs/dyco3d/lib/python3.7/site-packages/setuptools/_distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/home/ubuntu/Downloads/yes/envs/dyco3d/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/home/ubuntu/Downloads/yes/envs/dyco3d/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
cmd_obj.run()
File "setup.py", line 41, in run
self.build_extension(ext)
File "setup.py", line 71, in build_extension
subprocess.check_call(['cmake', ext.sourcedir] + cmake_args, cwd=self.build_temp, env=env)
File "/home/ubuntu/Downloads/yes/envs/dyco3d/lib/python3.7/subprocess.py", line 328, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '/home/ubuntu/de/dyco3d/lib/spconv', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/ubuntu/de/dyco3d/lib/spconv/build/lib.linux-x86_64-3.7/spconv', '-DCMAKE_PREFIX_PATH=/home/ubuntu/Downloads/yes/envs/dyco3d/lib/python3.7/site-packages/torch', '-DPYBIND11_PYTHON_VERSION=3.7', '-DSPCONV_BuildTests=OFF', '-DCMAKE_CUDA_FLAGS="--expt-relaxed-constexpr"', '-DCMAKE_BUILD_TYPE=Release']' returned non-zero exit status 1.
Dear author:
Thanks for your marvelous job and the code shared in github. I just run your code, and the result is lower than that stated in the paper, and the validation result is shown in the image below:
I find out that in the present train.py, the code that realize lr decay is missing:
I wonder if the code is not right, or you put the lr decay in other place. Looking forward to your reply.
Sincere.
ERROR conda.core.link:_execute(700): An error occurred while installing package 'daleydeng::gcc-5-5.4.0-2'
Thank you for sharing the code. I would like to ask about some of the problems I encountered when running your code. What is the upper limit for the number of iterations of the code? For data of different batch sizes, it seems that the upper limit of iteration is 30k times. Does it mean that the larger batch size will lead to more training times? I would like to ask what your specific settings are during training.
I am sorry to disturb you again because I am confused about two sentences below in this paper.
(1) During training, the ground truth for
What is the meaning of largets number of points? For a predicted mask, doesn’t it only contain one specific category of mask?
(2)
From the paper, I think a cluster obtains the semantic prediction from Seg Head
and contains the points which belong to same semantic label
. So it does not need an indicator function to determine whether a point have the identical semantic labels with group? But it seems I misunderstood. Could you give me some advice? Thanks.
Hi!
Thanks for open sourcing this amazing work!
I might be wrong, but I think several hardcoded variables should have been passed in through the config file. e.g. in the following example, the min_pts_num=50
should have been passed in via the cluster_npoint_thre: 50
in the config file.
DyCo3D/model/pointgroup/pointgroup.py
Lines 574 to 576 in 17d22c2
This took me quite a moment, as small objects with fewer than 50 points were missing in the outputs, even if I turned cluster_npoint_thre in the config file to 5 : )
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j4']' returned non-zero exit status 2
How can I solve this error? Anyhelp would be greatly appreciated please.
Hi, similar to #9 , I noticed the types of loss mentioned in the paper (L_ctr, L_mask, L_dice) are not implemented/used in the released code. Instead, the code is using exactly the same loss as PointGroup (semantic loss, offset loss, score loss).
When I checked the value in mask_logits, I found it's almost random values that are much smaller than the threshold=0.5. Therefore, I think the current code is NOT the right version that we can see the effect of the mask head design.
Could you please take a look and update the correct version? Thanks
Code to prepare the ScanNet dataset can not be found. Any code to prepare the train/val/test set?
build
dir in spconvpip uninstall cmake && pip install cmake==3.14.4
export LD_LIBRARY_PATH=/home/ubuntu/anaconda3/envs/dyco3d/lib/python3.7/site-packages/torch/lib/:LD_LIBRARY_PATH
to solve error I forgot about, referencecd dist
and pip install
the generated .whl
file.CUDA_VISIBLE_DEVICES=0 python -m torch.distributed.launch --nproc_per_node=1 --master_port=$((RANDOM + 10000)) train.py --config config/dyco3d_multigpu_scannet.yaml --output_path work_dirs --use_backbone_transformer
ImportError: /home/ubuntu/3D_MODEL/DyCo3D/lib/pointgroup_ops/PG_OP.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZN6caffe26detail36_typeMetaDataInstance_preallocated_7E
This error should not be important.
Here is the own data guide:
x y z r g b semantic_label instance_label
if you got some error like CUDA kernel failed : invalid configuration argument
maybe your data is empty, try to debug this line to see if args is empty
idx_shift, start_len_shift = pointgroup_ops.ballquery_batch_p(coords_ + pt_offsets_, batch_idxs_, batch_offsets_, self.cluster_radius, self.cluster_shift_meanActive) # in pointgroup.py
below is my env.
Thank you for your code. When I trained the model using Scannet v2 on my server (batch size 16, four 3090 GPUs), I found a reduction on the peformance, which is shown below. When I tried to load your trained model for evaluation, I realized that your model is trained using 50k iterations. So, I changed the iter to 5k in the config file. However, the results is still unsatisfactory. Could you give more details about your experiments to produce the results, like the number of prepare iterations using PointGroup (i.e., the training itr for semantic segmentation), batch size and corresponding learning rate? Thank you!
My point cloud data is in pcd format, which contains many workpiece. Can I use DyCo3D to segment each workpiece with pretrained model and how to do that? I‘m a new guy in this field and looking forward to your reply.
As title
Hi,
First of all, I highly apreciate your work. It has great influence on the subject and have multiple sue posibilities.
Though, I would like to ask you regarding possible instructions on how to prepare s3dis dataset to allow your model train on it and evaluate. Could you share such instruction? From what I saw in article, this method was already tested on s3dis dataset.
Hope for your reply.
best regards
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.