nikitakaraevv/pointnet

PyTorch implementation of "PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation" https://arxiv.org/abs/1612.00593

pointnet's Issues

AttributeError: module 'utils' has no attribute 'cent_norm'

Pointnet regression

@nikitakaraevv

Thank you very much for this nice repository. I found it really insightful when learning about Pointnet.

Suppose I want to do regression instead of classification. An example is to estimate the object's weight from a 3D point cloud.

Can regression be achieved by "simply" changing the output activation layer to a linear one (instead of softmax) and implementing a different loss function (for example mean absolute error instead of negative log likelihood loss).

I'm pretty new to Pointnet, so maybe you have some nice suggestions...
Thanks in advance!

Part Segmentation on all classes

Is it possible to train the part segmentation on all the classes of the dataset?

modelnet40

Hi,
can you help me to read modelnet40 for pointnet?
thanks.

module 'utils'

When I try to run the notebook

AttributeError                            Traceback (most recent call last)
<ipython-input-16-6f9310cafa62> in <module>
     12 print('######### Dataset class created #########')
     13 print('Number of images: ', len(dset))
---> 14 print('Sample image shape: ', dset[0]['image'].shape)
     15 #print('Sample image points categories', dset[0]['category'], end='\n\n')
     16 

<ipython-input-15-a0ffe3b5dad8> in __getitem__(self, idx)
     36         category = self.files[idx]['category']
     37         with open(img_path, 'r') as f:
---> 38             image1 = read_pts(f)
     39         with open(category, 'r') as f:
     40             category1 = read_seg(f)

<ipython-input-13-6a351433de44> in read_pts(file)
      3 def read_pts(file):
      4     verts = np.genfromtxt(file)
----> 5     return utils.cent_norm(verts)
      6     #return verts
      7 

AttributeError: module 'utils' has no attribute 'cent_norm'

Initially I had this error

ModuleNotFoundError: No module named 'utils'

So i installed this package but I think that it is not the correct one. Which one do I have to install?

This part seems to be a typo, so it would be better to correct it.

torch.save(pointnet.state_dict(), "save_"+str(epoch)".pth") -> torch.save(pointnet.state_dict(), "save_"+str(epoch)+".pth")

Prediction with one object

Dear Nikita,

thanks again for your great help, I am trying to classify a single object

with open(path/"chair/test/chair_0890.off", 'r') as f:
  verts, faces = read_off(f)
  

pointcloud = PointSampler(3000)((verts, faces))

with torch.no_grad():
  inputs = data['pointcloud']
  outputs, __, __ = pointnet(inputs.transpose(1,2))
  _, preds = torch.max(outputs.data, 1)
  print(preds.numpy())

Then I receive this error,

RuntimeError                              Traceback (most recent call last)

<ipython-input-68-a9df877ae490> in <module>()
      1 with torch.no_grad():
      2   inputs = data['pointcloud']
----> 3   outputs, __, __ = pointnet(inputs.transpose(1,2))
      4   _, preds = torch.max(outputs.data, 1)
      5   print(preds.numpy())

7 frames

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/conv.py in forward(self, input)
    257                             _single(0), self.dilation, self.groups)
    258         return F.conv1d(input, self.weight, self.bias, self.stride,
--> 259                         self.padding, self.dilation, self.groups)
    260 
    261 

RuntimeError: expected scalar type Double but found Float

Any help would be much appreciated.

Airborne Lidar Data

Will minor tweakings of this implementation work for Airborne Lidar data to classify power lines and vegetation which will have sparse point clouds?

.las files to learn

Hello, is there any way to use own dataset from .las files?

different results

I trained with also 15 epochs
then I modify the save_14.pth to get into the evaluation and I got this confusion matrix completely bias with the monitor
.

Is there any reason why? what would you recommend me to do?! I am following you Colab https://colab.research.google.com/github/nikitakaraevv/pointnet/blob/master/nbs/PointNetClass.ipynb#scrollTo=y2i_0ECIcR1X

Can't reproduce the classification results

Hi, @nikitakaraevv Thanks a lot for your code! But I can't reproduce the classification results. Could you kindly give some suggestions? Thanks a lot!

I download the save.pth and run the test code in PoinetNetClass.ipyb, but get really different results with the reported, especially for desk and night stand, as shown in the following table.

method	Bathtub	Bed	Chair	Desk	Dresser	Monitor	Night stand	Sofa	Table	Toilet	Avg
reported	93.4	92	97.2	81.5	71	89.4	56	86.9	93.4	95.9	82
save.pth	82	97	100	40	49	73	90	95	93	83	80.2

I retrain the model with default hyper parameters. Except for comment one line of code, since it prevents me from setting num_workers in dataloader as 4.
# random.seed = 42
The results is still different with the reported rsults, as shown in the following table.

method	Bathtub	Bed	Chair	Desk	Dresser	Monitor	Night stand	Sofa	Table	Toilet	Avg
reported	93.4	92	97.2	81.5	71	89.4	56	86.9	93.4	95.9	82
save.pth	82	97	100	40	49	73	90	95	93	83	80.2
reproduced	80	98	93	38	52	92	88	86	98	88	81.3

I check your training process in the PoinetNetClass.ipyb, the valid accuracy is still not stable near 15th epoch, around 4% gap between adjacent epochs. Any suggestions to make it stable?

Can't train with bacth size = 1

Hi there,

I set

  parser.add_argument('--batch_size', default=1, type=int,
                          help='training batch size')

And I got the following error:

  ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1, 512])

My env:

• Torch == 1.7.1
• torchvision == 0.8.2
• CUDA == 10.2
• OS = Win 10

Advice: Using `x, y, z` points stored in CSV

Hello! Thank you for sharing this implementation of PointNet! I found your tutorial super clear and easy to follow through! I am currently working on neutrino vs background classification for my masters research and the team requires me to use PointCloud NN. I was unsure about how my data format would fit this NN implementation.

The dataset (around 5GB) has x, y, z positions and a label indicating whether it is a neutrino or background in csv format. I can group the neutrinos into muon events that will represent point clouds.

dom_id,  pos_x ,pos_y, pos_z, dir_x ,dir_y, dir_z, tot, time, label
321.0,  -17.660999999999994, 32.245,65.23100000000001, -0.46, -0.266, -0.847, 26.0, 0.0, 0

I am unsure if my data will work here since it is just plain x, y, z points and does not have meshes (like in the example used). Is it possible to use this data and format with the implementation? The pointnet paper does mention that simple x,y,z coordinates can be used at the very least but I'm not very confident. Any advice would be quite appreciated!

[Question] Loss Function

Hello buddy, I just started learning PointNet and am working on my own implementation currently as well. Your code has offered a great insight into my work but at the moment I don't quite understand your loss function setup somehow.

def pointnetloss(outputs, labels, m3x3, m64x64, alpha = 0.0001):
    criterion = torch.nn.NLLLoss()
    bs=outputs.size(0)
    id3x3 = torch.eye(3, requires_grad=True).repeat(bs,1,1)
    id64x64 = torch.eye(64, requires_grad=True).repeat(bs,1,1)
    if outputs.is_cuda:
        id3x3=id3x3.cuda()
        id64x64=id64x64.cuda()
    diff3x3 = id3x3-torch.bmm(m3x3,m3x3.transpose(1,2))
    diff64x64 = id64x64-torch.bmm(m64x64,m64x64.transpose(1,2))
    return criterion(outputs, labels) + alpha * (torch.norm(diff3x3)+torch.norm(diff64x64)) / float(bs)

According to the PointNet paper Page 4 Joint Alignment Network section and Supplementary Part C "A regularization loss (with weight 0.001) is added to the softmax classification loss to make the matrix close to orthogonal.", shouldn't the loss function be something like below:

# Regularization function from the official pytorch implementation repo 
# Ref. https://github.com/fxia22/pointnet.pytorch/blob/master/pointnet/model.py
def feature_transform_regularizer(trans):
    d = trans.size()[1]
    batchsize = trans.size()[0]
    I = torch.eye(d)[None, :, :]
    if trans.is_cuda:
        I = I.cuda()
    loss = torch.mean(torch.norm(torch.bmm(trans, trans.transpose(2,1)) - I, dim=(1,2)))

def pointnetloss(outputs, labels, m3x3, m64x64, alpha=0.001):
    criterion = torch.nn.NLLLoss()
    return criterion(outputs, labels) + alpha * feature_transform_regularizer(m64x64)

I newly just stepped into this area of research and tbh I am still not confident to say that I fully understand the paper. If you can show me some pointers, that would be more than appreciated!
Cheers~