zsef123/pointrend-pytorch

A PyTorch implementation of PointRend: Image Segmentation as Rendering

PointRend

A PyTorch implementation of PointRend: Image Segmentation as Rendering

[arxiv] [Official Implementation: Detectron2]

---

This repo for Only Semantic Segmentation on the PascalVOC dataset.

Many details differ from the paper for feasibilty check.

---

Reproduce Fig 5.

Sampled Points showing from different strategies on A Dog image.

See test_point_sampling.ipynb

Original Figure

Reference : Pytorch Deeplab Tutorial

---

How to use:

First, fix data path in default.yaml

Multi GPU Training See details in Single GPU Training

➜ python3 -m torch.distributed.launch --nproc_per_node={your_gpus} main.py -h

Sinle GPU Training

➜ python3 main.py -h
usage: main.py [-h] config save

PyTorch Object Detection Training

positional arguments:
  config      It must be config/*.yaml
  save        Save path in out directory

optional arguments:
  -h, --help  show this help message and exit

e.g.)

python3 main.py config/default.yaml test_codes

---

pointrend-pytorch's Issues

question:loss=0,seg=0,point=0

Thank you for your contribution, I in the use of the network to train their own data set, data format shall be carried out in accordance with the format of cityscapes only, use the command: python3. Main py configs/default yaml/output, the code can run, but the training of the output loss value, seg, point from the first epoch start all is 0 (I got the five categories, semantic segmentation), want to excuse me, what should be the problem?

About "points"

Excuse me, in "points", does p[:,:,0] represent the x-coordinate or the y-coordinate?

What does "topk uncertainty points" mean?

When running test_point_sampling.ipny in the test file, the "topk uncertainty points" in the file do not know what it means. Can you tell me？Thank you!!!

so many bugs in this project

so many bugs in this project, eg : the label is wrongly normalized to 0-1 ..... and the mlp is a single 1*1 conv even without a relu , Hope author repair soon

In Class-PointHead: “During inference, subdivision uses N=8096” Why is 8096？

Thanks for your contribution.
Here are the code-comments in pointrend.py class: PointHead-inference:
"""
During inference, subdivision uses N=8096
(i.e., the number of points in the stride 16 map of a 1024×2048 image)
"""
i found this N=8096 in paper: "5.Experiments: SemanticSegmentation" and as same as your code variable N. I don't understand how to get this N. Is it related to the size of the input image？or maybe something else?

what‘s the vision of torchvision in this code?

About validation code.

Thx a lot for the code but I didn't find the validation code. Are they released?

如何在Mask-rcnn上尝试PointRend

Whether the implementation's result is the same as original code of FAIR?

Could you give me some comparison of your implementation's result VS original code?

is this project done?

is this project done or in process your project?

I want to run this your project but not smooth..

RuntimeError: Dataset not found or incomplete. Please make sure all required folders for the specified "split" and "mode" are inside the "root" directory

how to process dataset?

about the resnet53

I saw in your code that the first conv of resnet53 should be replaced by three 3x3 convs, but your code only wrote one conv.

There is a bug in inference stage

In the training stage, the samplepoint func return a tensor while in the inference stage it is asked to return a tuple with point_idx and point

hello Here is my suggestion! Using " torch_scatter()" rather than "for loop" during inference .

Example:
B, D, H, W = pred.shape
rend_pred = pred.view(B, C, -1).scatter_(2, rend_result['points'].unsqueeze(1).expand(-1, C, -1),
rend_result['rend'].float())
rend_pred = rend_pred.reshape((B, C, H, W))

Necessary modification

Hi~
I made some necessary modification to run your code. The most important part is in pintrend.py:

def inference(self, x, res2, out):

    B = x.shape[0]
    _, C_res2, H_res2, W_res2 = res2.shape

    while out.shape[-1] != x.shape[-1]:
        # N = out.shape[-2] * out.shape[-1]
        out = F.interpolate(out, scale_factor=2, mode="bilinear", align_corners=True)

        _, C_out, H_out, W_out = out.shape
        points = sampling_points(out, training=False, N=4048)
        coarse = torch.gather(out.view(B, C_out, -1), 2,
                              points.unsqueeze(1).expand(-1, C_out, -1))

        stride_y = H_out // H_res2 
        stride_x = W_out // W_res2
        points_index_x = points // W_out // stride_x
        points_index_y = points % W_out // stride_y
        res2_points = (points_index_x * W_res2 + points_index_y).long()

        fine = torch.gather(res2.view(B, C_res2, -1), 2,
                            res2_points.unsqueeze(1).expand(-1, C_res2, -1))
        feature_representation = torch.cat([coarse, fine], dim=1)

        rend = self.mlp(feature_representation)

        out = out.view(B, C_out, -1).scatter_(2, points.unsqueeze(1).expand(-1, C_out, -1), rend)
        out = out.view(B, C_out, H_out, W_out)

Can u grant me to the acccess rights, so I can make a PR. thx a lot
I am interested in implementing it in maskrcnn
@zsef123