dylanwusee/pointconv

pointconv's Issues

about supplementary material

thank you for the great job.
where can i found the supplementary material?

About scannet model

Could you please share the pretrained model on Scannet?
Thanks for help !

Training scannet with RGB data

Hi,
Is the provided code ready to use RGB data for training? If i'm not wrong, multiple things need to be modified.
Thanks in advance.

BN's effect on Updated ScanNet Results

Dear authors,

Hi, I notice that you remove BN since it may lead to overfitting on the updated ScaNet results, as described in issue 22. Does this mean you remove all the BN in the network?

More specifically, current implementation uses BN in both PointConv modules (feature aggregation) and the mask prediction branch. I am curious about which part would result in overfitting.

Best,
Jianyuan

tf_interpolate_so.so

Hi,

Thank you for sharing code.

I found a problem of "tf_interpolate_so.so" when I compile your source code. This is error message it throw out: "undefined symbol: _ZN10tensorflow15OpKernelContext10CtxFailureEPKciRKNS_6StatusE"

I couldn't figure out the problem, have you see this kind of error?

Thanks

About the optimization of memory consumption

In the section 4 of the paper, an efficient PointConv implementation is discussed. From the analysis, particularly the comparison between Fig.3 and Fig.5, we know that the naive implementation costs K x C_in x C_out memory, while the optimized solution costs only C_in x C_mid. Therefore, after optimization, the memory footprint can be reduced to C_min / (K x C_out).

Here I have a question. In Fig. 5, the input to the 1x1 conv is a feature of size 1 x (C_in x C_mid), and the kernel size of 1 x 1 conv is C_mid x C_in x C_out. So does this mean that the "multiplication" applied here for the convolution induces a memory consumption C_mid x C_in x C_out (which is for the storage of such convolution kernel parameters)? If so, the improvement ratio should be C_mid / K. And since the parameters set in the experiment are C_mid = K = 32, there is no memory consumption reduction.

I wonder if my analysis is incorrect.

transfer the classification code in tensorflow shows low results.

Thanks for your great job.
I tried to transfer the classification part in tensorflow,however only achieving around 85%.
I noticed that you released the classification part in pytorch.
Why did you choose two frameworks?
Have you tried the classification part in tensorflow?

feature_decoding_layer_depthwise

Hi @DylanWusee,

Thank you for your interesting work!

Would you mind if you can explain the difference between "feature_decoding_layer_depthwise" and "feature_decoding_layer" functions? I didn't get the hints from your published paper. Also, do you have any ideas about further improvement, especially for large-scale LiDAR datasets, i.e., KITTI.

Cheers,
Lingfei

Kernel density calculation

Hi @DylanWusee,

I am a little confused when you calculate the kernel density, why did you choose "N_points = 128" (in the line of 37 in the pointconv_util.py file), how do you determine this value? Or what is the point density/spacing of your ScanNet dataset?

Thanks for your kind assistance again.

Cheers,
Mark

About Data Augmentation

@DylanWusee i notice that you want to do data augmentation, but the code

"#batch_data, batch_label, batch_smpw = get_batch_wdp(TRAIN_DATASET, train_idxs, start_idx, end_idx)
batch_data, batch_label, batch_smpw = get_batch(TRAIN_DATASET, train_idxs, start_idx, end_idx)
# Augment batched point clouds by rotation
aug_data = provider.rotate_point_cloud_z(batch_data)
#aug_data = provider.rotate_point_cloud(batch_data)"
means that you just rotate your data, so i am curious why you do it? is there any tricks for to train the model?

some error occurred when run train_scannet_IoU.py

IternalError: Blas xGEMMBatched launch failed.

how to solve it, thanks

test.py and predict.py

Hi @DylanWusee ,
Could you please release your test.py and predict.py to me ?
Cause I don't know how to modify the evaluate.py to test.py and predict.py.
Thanks a lot!

Efficient PointConv

Hi. Is there an efficient version of PointConv available? Or some documents for the implementation will be very helpful. Thanks!

how to understand the method in scanner evaluation

In your paper, it said:
evaluate the model with exhaustively choose all points in the 3m ⇥ 1.5m ⇥ 1.5m cube in a sliding window fashion through the xy-plane with different stride sizes.

how to understand the above sentence?

thank you

can not download scannet dataset.

supplementary material

Hi, @DylanWusee , thanks a lot for sharing your great work, could you please share the supplementary material or give me its link?

problem in load pb model

i change the ckpt model into pb, but when i load the pb model, i met a problem about

"ValueError: callback pyfunc_7 is not found
[[node fa_layer4/PyFunc (defined at /home/fussen/Work_Area/02_DGASIS/test_pb.py:141) ]]
Caused by op 'fa_layer4/PyFunc', defined at:"

i didn't kown the reason result to this, and had no idea to solve it, could you help me to slove it?
looking for you reply!

cannot find rgb info in placeholder

Hi,
Thanks for sharing this work!

in the paper, you said, the input is six channels, x, y, z, r, g, b.
but I find in the code, it's

   pointclouds_pl = tf.placeholder(tf.float32, shape=(batch_size, num_point, 3))

it seems the input is three channels?

Thanks!

providepython 3.6？

question

I would like to ask where there is a detailed code for pointconv.py some conde can not understand thanks

What is difference about two models in /models

Could you explain what is the difference between pointconv_weight_density_n16.py and pointconv_weight_density_n16_dp.py?

Thanks

Hi,I have seen that your solution had been tested on ModelNet40 from your paper,can you public your test code??

can you provide some codes about segmentation on s3dis dataset?

hello,how do you download scannet dataset，http://www.scan-net.org/　do not have download link ,

Question about the mIoU on ScanNet validation set

Hi @DylanWusee ,

Thanks for your contribution!

I tried to reproduce your results on ScanNet with default configurations, and there seems a gap between mine and yours.

If I understand your released code correctly, the default configurations is : input-only xyz, evaluation stride-0.5, use Density with MLP.
Table 5 in your paper shows that mIoU is 61.0 under this configurations. However, my reproduced result is 0.596368.

The gap is not ignorable, how can I get a better result? I use the default batch_size 8, should I modify this param?

My log_evaluate.txt is as below:

Namespace(batch_size=8, dump_dir='dump', gpu=0, model='pointconv_weight_density_n16', model_path='pointconv_scannet_raw/best_model_epoch_500.ckpt', num_point=8192, num_votes=5, ply_path='../../data/scannet/scans', with_rgb=False)
Model restored.
2020-06-08 19:28:12.735647
---- EVALUATION WHOLE SCENE----
eval point avg class IoU: 0.596368
Each Class IoU:::
Class 1 : 0.7410
Class 2 : 0.9473
Class 3 : 0.5330
Class 4 : 0.7170
Class 5 : 0.8281
Class 6 : 0.7131
Class 7 : 0.6537
Class 8 : 0.3413
Class 9 : 0.4584
Class 10 : 0.7243
Class 11 : 0.1304
Class 12 : 0.5907
Class 13 : 0.5174
Class 14 : 0.5421
Class 15 : 0.4015
Class 16 : 0.5021
Class 17 : 0.8202
Class 18 : 0.5903
Class 19 : 0.7632
Class 20 : 0.4123

ShapeNet model?

Can you upload ShapeNet model file?

On Updated ScanNet Results

Dear authors,

I have noticed that you submitted a version to ScanNet benchmark recently, and the result (mIoU=66.6%) is significantly better than the CVPR'19 version (55.6%). Is it possible to describe what change you have made to the original CVPR'19 model to achieve such large performance boost? Thanks a lot for your response.

Best,
Haotian

scannet dataset

Can you upload scannet_train_rgb21c_pointid.pickle, scannet_val_rgb21c_pointid.pickle, and scannet_test_rgb21c_pointid.pickle?

Thank you !

question feature_decoding_layer_depthwise and feature_decoding_layer?

@DylanWusee @Lawrence444 Thanks for you sharing. I am curious about the feature decoding. feature_decoding_layer_depthwise is a function for depthwise conv, but feature map would lack the information of space loaction after this operation. So why do you use the feature_decoding_layer_depthwise, have you compared the results between the feature_decoding_layer_depthwise and feature_decoding_layer on the same dataset?

Which on is the code of classification on modelnet dataset?

runtime problem

HI, thank you for the work
when i running the training code, i encounter error
Traceback (most recent call last): File "train_scannet_IoU.py", line 55, in <module> MODEL = importlib.import_module(FLAGS.model) # import network module File "/home/muzhengyang/anaconda3/envs/pointconv/lib/python2.7/importlib/__init__.py", line 37, in import_module __import__(name) File "/unsullied/sharefs/muzhengyang/dataroot1/pointconv/pointconv/models/pointconv_weight_density_n16.py", line 10, in <module> from PointConv import feature_encoding_layer, feature_decoding_layer File "/unsullied/sharefs/muzhengyang/dataroot1/pointconv/pointconv/PointConv.py", line 20, in <module> from tf_interpolate import three_nn, three_interpolate File "/unsullied/sharefs/muzhengyang/dataroot1/pointconv/pointconv/tf_ops/3d_interpolation/tf_interpolate.py", line 8, in <module> interpolate_module=tf.load_op_library(os.path.join(BASE_DIR, 'tf_interpolate_so.so')) File "/home/muzhengyang/anaconda3/envs/pointconv/lib/python2.7/site-packages/tensorflow/python/framework/load_library.py", line 56, in load_op_library lib_handle = py_tf.TF_LoadLibrary(library_filename) tensorflow.python.framework.errors_impl.NotFoundError: /unsullied/sharefs/muzhengyang/dataroot1/pointconv/pointconv/tf_ops/3d_interpolation/tf_interpolate_so.so: undefined symbol: _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringEv

I have compiled the pointnet custom operation.

how to fix it?

thanks

is there any pytorch version for this?

feature_encoding_layer

Hi @DylanWusee
Thank you for you excellent code. I'm implementing a Pytorch version on your work.
I have a question about the feature_encoding_layer.
In your cvpr paper fig.5. The local feature F_in(KxC_in) will mutiply with the Density(KxC_in) directly.
But in your code, the grouped_feature will conv before muliply with Density.
Am I right?

for i, num_out_channel in enumerate(mlp):
            if i != len(mlp) - 1:  
                grouped_feature = tf_util.conv2d(grouped_feature, num_out_channel, [1,1],  
                                            padding='VALID', stride=[1,1],  
                                            bn=bn, is_training=is_training,  
                                            scope='conv%d'%(i), bn_decay=bn_decay, weight_decay = weight_decay) 
weight = weight_net_hidden(grouped_xyz, [32], scope = 'weight_net', is_training=is_training, bn_decay = bn_decay, weight_decay = weight_decay)
density_scale = nonlinear_transform(density_scale, [16, 1], scope = 'density_net', is_training=is_training, bn_decay = bn_decay, weight_decay = weight_decay)
new_points = tf.multiply(grouped_feature, density_scale)

Cheers,
ziyu

Any plan to upload semantic segmentation model ?

hi, thank you for the quality code.

But, are you having plan to upload a model for semantic segmentation which consumes 8192 points?

Or Can you say how many encoding layers and decoding layers did you choose?
Thank you.

Modify evaluate_scannet.py to test

when I modify the ply_path with 'scannetv2/scans_test' and the file_list = "./scannet/scannetv2_test.txt" and scannet_dataset_sw_rgb.ScannetDatasetWholeScene_evaluation(root=DATA_PATH, split='test', with_rgb = WITH_RGB),
there was something wrong happened.
Could you help me? Thks! @DylanWusee