adavoudi/spdnet

Implementation of Deep SPDNet in pytorch

spdnet's Issues

Why is gradient orthogonaly projected with 0_n matrix?

In lines:

spdnet/spdnet/optimizer.py

Lines 37 to 39 in 2a15e90

	p.data.fill_(0)
	trans = orthogonal_projection(p.grad.data, p.data)
	p.grad.data.fill_(0).add_(trans)

Weight matrix (p.data) is set to 0 before computing the orthogonal projection with it's gradient, why is this the case?

Kullback leibler divergence

def distance_kullback(A, B):
    """Kullback leibler divergence between two covariance matrices A and B.
    :param A: First covariance matrix
    :param B: Second covariance matrix
    :returns: Kullback leibler divergence between A and B
    """
    dim = A.shape[0]
    logdet = numpy.log(numpy.linalg.det(B) / numpy.linalg.det(A))
    kl = numpy.trace(numpy.dot(numpy.linalg.inv(B), A)) - dim + logdet
    return 0.5 * kl

Determinant of a SPD matrix

torch.potrf(a).diag().prod()

https://discuss.pytorch.org/t/how-to-calculate-the-determinant-of-a-variable/1393/5

UnTangentSpace behavior different from scipy.linalg.expm

I performed some neural network training and produced a 50x50 SPD matrix called X which I compared to another SPD Y (ground truth), by putting both of them into Tangent space and computing a standard MSELoss between them. After some successful iterations during which the loss decreased, I decided to transform X back to "normal space" using untangent_space = SPDUnTangentSpace(unvectorize=False). I found that the result of untangent_space(X) is very different from the true Y. To check again, I then performed the back conversion by taking scipy.linalg.expm(X) instead of applying untangent_space, and then the result was very close to the true Y, as expected.

I looked at spd.py, and the forward function for SPDUnTangentSpace seems ok... nothing suspicious.
Another interesting thing - when I applied scipy.linalg.logm to the result of untangent_space(X), I obtained a version of X in which all the signs were flipped, as if scipy.linalg.logm(untangent_space(X)) = -X. Very puzzling. As if it was a different leaf or something.

I cannot give you the exact matrices now for experimenting, but I'll try to come up with a more simple example (say, a 5x5 matrix). Maybe it has something to do with numerical stability of the SVD, or there's something in tangent_space in utils.py?.. I'll look to it, but meanwhile will refrain from using SPDUnTangentSpace.

UPD: Basically, X has nothing to do with the problem. It boils down to two observations:

untangent_space(tangent_space(Y)) != Y
scipy.linalg.logm(untangent_space(tangent_space(Y)) = -scipy.linalg.logm(Y)

By the way, you are using the SVD of a matrix and its diagonal part for computing the log_ and the exp_, while the original paper by Huang and Van Gool used the eigendecomposition. Is it equivalent?

GPU利用率低

在我的服务器运行您的代码，GPU利用率只有个位数，导致训练非常慢，请问怎么解决

It is time consuming to calculate the svd decomposition of each sample one by one. Is there a vectorization version？

bug in SPDIncreaseDim, when SPD matrix size increases in SPDTransform

I tried to create a somewhat unconventional SPDnet, where the size of SPD matrix first diminishes, and then increases back to original, in the manner of U-nets used for image segmentation. Here's my net that does 52x52->36x36->52x52:

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.trans1 = SPDTransform(52, 36)  #  first do some reduction
        self.trans2 = SPDTransform(36, 52)  #  then expand back to original size
        self.relu1 = nn.ReLU()
        self.relu2 = nn.ReLU()

    def forward(self, x):
        x = self.trans1(x)
        x = self.relu1(x)
        x = self.trans2(x)
        x = self.relu2(x)
        return x

When doing a forward pass with a batch of size 8x52x52, I get this error:

File "/home/anaconda3/lib/python3.6/site-packages/spdnet/spd.py", line 49, in forward
output = torch.baddbmm(eye, torch.bmm(input, eye.transpose(1,2)), add)

RuntimeError: The expanded size of the tensor (52) must match the existing size (36) at non-singleton dimension 2. Target sizes: [8, 36, 52]. Tensor sizes: [8, 52, 36]

This happens in the forward part of SPDIncreaseDim.
Could you please point me out what I'm doing wrong, or how should I change my code or your source code to get correct behaviour? Thanks!

Retraction operation

Hi,

I'm new to geometric deep learning and I'm wondering why you chose to realize the retraction operation with:

data = A + ref
Q, R = data.qr()
sign = (R.diag().sign() + 0.5).sign().diag()
out = Q.mm(sign)

instead of

data = A + ref
Q, R = data.qr()
out = Q

as is suggested in equation 4.8 page 59 of Optimization Algorithms on Matrix Manifolds (Absil, Mahony, Sepulchre), which is the reference of the original SPDNet paper for this exact operation.

In any case, I take advantage of this post to thank you for your public PyTorch implementation of the paper !

Cheers,
Blupon.

Missing SPDTransformFunction

Hi;
We are facing an error when runing the tests/test_modules.py.
A (torch.autograd.)Function is missing (not defined): Although the nn.Module SPDTransform is well defined, we can not find the SPDTransformFunction.
Is it a critical issue or the remaining of the code can be executed without difficulties?
Best regards;
A. Falaize

size of weights of SPDTransform incorrect when dimension increases

Hi again,
I have a neural net with these modules:

self.trans1 = SPDTransform(52, 26)
self.trans2 = SPDTransform(26, 13)
self.trans3 = SPDTransform(13, 26)
self.trans4 = SPDTransform(26, 52)

When I take a look at my model.trans3.weight.shape, I norice that it is torch.Size([26, 26]), and model.trans4.weight.shape returns torch.Size([52, 52]). I don't know how this is treated in the code, but it seems to be a mistake. For trans1 and trans2, i.e. when dimenstion decreases, the shape of the weights is correct.

RuntimeError: Lapack Error gesdd : 1 superdiagonals failed to converge.

I get a RuntimeError when running augment_data.py, which traces back to SVD (matrix_operator function invoked from tangent_space function). Possibly due to a badly-conditioned matrix?

Traceback (most recent call last):
  File "augment_data.py", line 35, in <module>
    tangent = tangent_space(data, data)
  File "/home/reddwarf/.local/lib/python3.6/site-packages/spdnet/utils.py", line 43, in tangent_space
    middle = matrix_operator(middle, 'expm')
  File "/home/reddwarf/.local/lib/python3.6/site-packages/spdnet/utils.py", line 20, in matrix_operator
    u, s, v = A.svd()
RuntimeError: Lapack Error gesdd : 1 superdiagonals failed to converge. at /opt/conda/conda-bld/pytorch_1556653183467/work/aten/src/TH/generic/THTensorLapack.cpp:406

HTTP Error 404 when runing the demo

Hi!
We tried the demo in examples, and got line 48 of dataset.py in download:
urllib.error.HTTPError: HTTP Error 404: Not Found
This appens after the file data.zip is successfully downloaded.

mat file for running examples/demo.py

Hello,
Many thanks for your Riemannian network for SPD matrices implementation!
How do you obtain spddb_afew_train_spd400_int_histeq.mat ?
I downloaded AFEW SPD data from Zhiwu Huang's website at ETH, but I don't know how to process the many SPD matrices in order to get your demo.py running.

RuntimeError:_th_diag not supported on CUDAType for Bool

I get a runtime error when I run demo.py. the version of my Pytorch is 1.2.0.what's the problem?
1747
371

Epoch: 1
0it [00:00, ?it/s]Traceback (most recent call last):
File "demo.py", line 133, in
train_loss, train_acc = train(epoch)
File "demo.py", line 73, in train
loss.backward()
File "/home/yxbu/.conda/envs/torch/lib/python3.7/site-packages/torch/tensor.py", line 118, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/home/yxbu/.conda/envs/torch/lib/python3.7/site-packages/torch/autograd/init.py", line 93, in backward
allow_unreachable=True) # allow_unreachable flag
File "/home/yxbu/.conda/envs/torch/lib/python3.7/site-packages/torch/autograd/function.py", line 77, in apply
return self._forward_cls.backward(self, *args)
File "/home/yxbu/code/spdnet/spdnet/spd.py", line 296, in backward
Q = max_mask.diag().float()
RuntimeError: _th_diag not supported on CUDAType for Bool
0it [00:00, ?it/s]

SPDRectified() makes the model output consistent

When I added SPDRectified() to my model, I found that the output of the model remained consistent for all samples. But after I remove the layer, the output is normal.

My code is as follows:

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.trans = SPDTransform(16, 10)
        self.rect = SPDRectified()
        self.tangent = SPDTangentSpace(10)
        self.linear = nn.Linear(55, 2, bias=False)

    def forward(self, x):
        a = self.trans(x)
        b = self.rect(a)
        c = self.tangent(b)
        d = self.linear(c)
        return d


if __name__ == '__main__':
    data = scipy.io.loadmat('./db.mat')['data']
    data = torch.from_numpy(data).to(torch.float)

    model = Net()
    pred = model(data)
    print(pred)

When I added SPDRectified(), the ouput is:

tensor([[0.6507, 4.6896],
        [0.6507, 4.6896],
        [0.6507, 4.6896]], grad_fn=<MmBackward0>)

When I delete SPDRectified(), the ouput is:

tensor([[-3.0920,  1.4963],
        [-3.1109,  1.5340],
        [-3.0174,  1.5908]], grad_fn=<MmBackward0>)

Why does this happen, is it because of the data?

Whether the line 172 of spd.py is wrong?

It is a good implementation on pytorch. But I'm a little confused about spd.py.

Whether the line 172 of spd.py is wrong? It might need to be

u.mm(2*symmetric(P.t() * (u.t().mm(dLdV)))+dLdS).mm(u.t())

In the original paper, this was multiplied by 2. But in this code, probably you ignored this. Would you mind answering a question for me? @adavoudi