why one SparseSequential cost much less time comparing with several SparseSequentials' combination?

[traveller59]

why one SparseSequential cost much less time comparing with several SparseSequentials' combination?

[bigsheep2012]

Update
I remove all .dense() in-betweens in forward function and the speed becomes normal.
So if I transfer a sparse tensor to dense like the following example, the time cost of middle_conv_block2 would be increased. The dense() return an inference of ret1 and turn ret1 into dense format inplace, which lead to an dense input for next sparse convolution?

def forward(self, voxel_features, coors, batch_size):
    coors = coors.int()
    ret0 = spconv.SparseConvTensor(voxel_features, coors, self.sparse_shape, batch_size)

    ret1 = self.middle_conv_block1(ret0)
    ret1_dense = ret1.dense()
    
    ret2 = self.middle_conv_block2(ret1)
 return ret2

[traveller59]

dense isn't a inplace operation, it convert sparse tensor to regular torch dense tensors without modify origin sparse tensor, this operation use pure pytorch implementation and may cost lots of time, check spconv/__init__.py for more details.
So the time cost increase shouldn't happen. How do you measure time? do you add torch.cuda.synchronize()?

[bigsheep2012]

Nope. But I test the time outside the forward function.
Like
start = time.time()
out=model(input)
end=time.time()
print end-start
torch.cuda.synchronize() would affect this duration a lot?
Anyway, the time cost becomes much less after I remove all dense() operations. I guess the dense operations is time costly.

[traveller59]

you must use torch.cuda.synchronize() to measure device code time:

torch.cuda.synchronize()
t = time.time()
# device code
torch.cuda.synchronize()
print(time.time() - t)

otherwise the measured time is wrong.

[bigsheep2012]

@traveller59
I followed your instructions and got the same results.
That is, with .dense() operations in-betweens, the time cost is around 200ms:
0.196189165115
0.204177856445
while, without (comment out) .dense() operations in-betweens, the time cost is around 70-90ms:
0.0651390552521
0.0862791538239

The example code is as follows:

def forward(self, voxel_features, coors, batch_size):
        coors = coors.int()
        ret0 = spconv.SparseConvTensor(voxel_features, coors, self.sparse_shape, batch_size)
        # ret0_dense = ret0.dense()
        torch.cuda.synchronize()
        t = time.time()
        
        ret1 = self.middle_conv_block1(ret0)  ## SparseSequential
        ret1_dense = ret1.dense()
        ret2 = self.middle_conv_block2(ret1) ## SparseSequential
        ret2_dense = ret2.dense()
        ret3 = self.middle_conv_block3(ret2) ## SparseSequential
        ret3_dense = ret3.dense()
        ret4 = self.middle_conv_block4(ret3) ## SparseSequential
        ret4_dense = ret4.dense()
        ret5 = self.middle_conv_block5(ret4) ## SparseSequential
        ret5_dense = ret5.dense()
        ret6 = self.middle_conv_block6(ret5) ## SparseSequential

        print time.time() - t
        return ret6.dense()